I ran across an interesting article from Quartz today that serves as an interesting warning for those attempting to compare cross-cultural survey results.

People from multiple countries were asked the same question “Would you personally accept a refugee into your own home?”, and the results were compared to find the “most welcoming” country.  China came out ahead by a large margin: 46% of residents said yes, as compared to 15% of US residents.

However, when the question was more closely examined, it was discovered that the English word “refugee” does not have an exact translation in Chinese. While in the US “refugee” almost always refers to someone from another country, in Chinese the word has a more neutral “person who has experienced a calamity” definition. Depending on the situation, it is then modified with either “domestic” or “international”.  The survey question did not contain either modifier, so it was up to the respondent’s personal interpretation.

So basically, people in different countries were answering different questions and then the results were compared. Surveys are already prone to lots of bias, and adding inexact translations into the mix can obviously heighten that effect. Interesting thing to be aware of when reading any research that compares international responses.