Home > Department of Media and Communications > Research > EU Kids Online > Best Practice Guide > FAQ 30: What are the key issues when collecting data in more than one country?

 

FAQ 30: What are the key issues when collecting data in more than one country?

What's the issue?

Potentially, any and all dimensions of a research project may take on a different meaning when conducted in a different country - including the questions asked, the terms used, the population studied and the position of the researcher. There is a persistent tension between the attempt to standardise the research conducted in different countries (e.g. using exactly the same sampling technique, questionnaire survey, approach to analysis) and the attempt to recognise and reflect cultural or social differences across research contexts.

Common practice

  • It is often asserted that the standardisation of methodological tools and conceptual frameworks is more easily achieved in quantitative research. Conversely, qualitative methods are arguably better at reflecting and responding to specific cultural contexts.
     
  • However, both approaches can be adjusted to comparative research, and both require considerable effort in both research design and data interpretation, so as to understand where the data are, or are not, directly comparable.
     
  • While efforts in comparative research are often concentrated on the construction of samples, the recruitment of respondents, the design of survey questionnaires or interview schedules and so forth, researchers must also attend to the challenges of data interpretation and analysis. Comparing questionnaire responses across countries (and languages) is easier than comparing interview transcripts, but ensuring that the questionnaire means the same thing in different languages is not easy. Ideally, questionnaires and interview schedules should be translated and then back translated to check the back translation against the original.

Pitfalls to avoid

There are very many of these, and they arise mainly from either the fact that the researcher will be more familiar with one country than another, or from the fact that researchers from different countries must collaborate together. Typically, one takes one's own context for granted, not perceiving its distinctive features, and sees the other context as unusual, not understanding how it makes sense to those who live there. While the major differences between countries are obvious (e.g. language), more subtle differences can easily be overlooked (e.g. expectations regarding parenting). Too often, it is convenience rather than the research rationale that directs the project (e.g. having access to researchers, or respondents, in another country, even though that country may not provide the optimal point of comparison).

Questions to consider

Why are you undertaking cross-national research? For instance, do you expect to find similarities or differences, and why might these be interesting? Which countries do you want to compare and why (what are their interesting and relevant points of similarity and difference?)? What are the practical issues to be addressed in comparing across countries? These might include the means of contacting children or obtaining their consent. Are there significant differences also within countries (e.g. the two language communities within Belgium, or the north/south divide that characterises many countries)? Even if words can be translated, do they have a different meaning in a different cultural context? Are findings typically disseminated differently in the countries you are working in?

Further resources

Hammersley, M., & Atkinson, P. (1983). Ethnography: Principles in Practice. London: Tavistock.
Livingstone, S. (2003). On the challenges of cross-national comparative media research. European Journal of Communication, 18(4), 477-500.
Mante-Meijer, E., & Haddon, L. (2005). Working in International Research Groups. In L. Haddon (Ed.), International Collaborative Research. Cross-cultural Differences and Cultures of Research. Brussels: COST.
Stald, G. (2004). International Co-operation in Research: Children and their Changing Media Environments'. In M. Grosse-Loheide & U. Hasebrink (Eds.), Netzwerke für die Informationsgesellschaft. Bielfeld: GMK.

A researcher's experience

In our research, we translated questionnaires used in the 'Young People, New Media' (Livingstone & Bovill, 1999) and SAFT (SAFT (Safety Awareness Facts and Tools) Project, 2004-2006) projects, to be answered in a self-completion survey by Portugese children aged 9-14. We found that expressions such as "stepmother" or "stepfather" are sensitive for Portuguese children, as the Portuguese words ("madrasta", "padrasto") have a derogatory meaning, associated with "unkind people", so we found alternative words. Also, questions about media use in children's bedrooms (or 'own rooms') did not fit the reality of children of very low SES. Last, the designation of the place where the child lives and play outdoors may also be ambiguous in different cultures. In Portugal, a large number of children live in flats and don't have access to private gardens. The experience of playing outdoors is mostly associated with public spaces. Houses with private gardens are mostly associated with high SES, and they are called "vivendas". However, a child who lives in an illegal house self-made by their parents (in a slum, for instance) may use the word "vivenda" to describe the place where he/she lives. In a survey that named different kinds of places to live, children's answers showed that their naming of those places is appropriated in their own socio-cultural terms.
(José Alberto Simões, Portugal)

Share:Facebook|Twitter|LinkedIn|