Home > Department of Media and Communications > Research > EU Kids Online > Best Practice Guide > FAQ 1: When is it better to do qualitative or quantitative research?

 

FAQ 1: When is it better to do qualitative or quantitative research?

What's the issue?

To some extent all questions may be approached either quantitatively or qualitatively. It all depends on what is our chief goal. Are we interested in a systematic approach, in order to produce comparable, generalisable data, or do we want to produce a "thick" description of a particular case/group/situation/context? Each option involves different kinds of planning, which may best be followed by a particular research design. Nevertheless, combination or mixed method approaches prove to be very useful in many situations, and seem to solve many of the problems, which arise from adopting a single methodological approach.

Common practice

  • Surveys are highly formal and standardised (we should be able to anticipate all pertinent questions); while field work/ethnographic methods are informal and open to unexpected data (indicating little control over events).
  • Quantitative methods are best when you want to compare data in a systematic way, make generalisations to the whole population or test theories with hypothesis. This is particularly so when you want to compare or generalise information extensively within and from a specific population or between different populations (some of them configured within particular geographical or socio-spatial units - like countries, regions, etc).
  • A qualitative approach is best when you are exploring a subject about which you don't know much in advance or, for the opposite reason, when you want to grasp the meanings, motives, reasons, patterns, etc, usually unnoticed in standardised approaches, like those you would get with a survey.
  • In short, to find quantitative differences in children's behaviour, beliefs, attitudes, we employ quantitative methods, but to find and illuminate meanings related to these differences, we employ qualitative methods.

Questions to consider

What kinds of questions should be translated into which research strategy? Are all problems quantifiable? Or should some be presented only qualitatively? Do we want to generalise our findings to the whole population? Are we after deep meanings rather than numbers?

Pitfalls to avoid

  • We need to try to avoid going after quantitative methods just because they provide generalisable results, which many consider as more appropriate and valid.
  • Try not to use a particular method just because it seems like a part of your 'research tradition'.
  • Think carefully about what the research problem is and go for the method that particular research question 'dictate' to use.

Further resources

Creswell, J. W. (2008). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Thousand Oaks: Sage
Bryman, A. (2004). Social research methods (2nd ed.). Oxford: Oxford University Press.

Example of good practice: Special Eurobarometer 250

First of all, let's consider quantitative strategies independently.

In 2005 a special Eurobarometer Survey on issues related to "Safer Internet" was conducted in 29 countries (25 EU member states, two candidate states - Bulgaria, Romania, and two accession countries - Croatia, Turkey). Respondents were adults who had a child under 18 living in their household for whom they were responsible (- for this reason, respondents were not necessarily the child's parents but could be older siblings or other carers). They were asked several questions regarding child internet uses. Although this didn't provide access to the children's actual behaviour but to others' perception of their behaviour, we might say that the research strategy chosen had the same general implications as it would have had with a different population: to produce comparable data about internet uses. This aim was achieved through a questionnaire, delivered to a representative sample from each country involved in that study.

Not all quantitative studies mobilise these kinds of resources; nonetheless their objectives remain identical: to obtain large amounts of information, under the same standardised conditions, in order that they can be treated, analysed and interpreted statistically. One of the main advantages of quantitative methods is precisely the possibility of making comparisons and enabling generalisations. This explains the popularity of surveys. But they also present some limitations. The number of questions is always limited, not to mention their scope. Some subjects may be difficult to translate into "closed questions", especially if we are dealing with sensitive subjects or when we are searching for meaning and understanding.

A researcher's experience: qualitative research on youth and the internet

For some researchers, the benefits of using a qualitative research strategy exclusively are considerable, depending on the purpose of the research. If you don't have a pre-defined theoretical model or if you want to capture "freely" (that is, with minimum intervention) what is on your subject's mind, qualitative methodologies may be most helpful to attain that goal. This seemed to be the view of two researchers from the SAFT project, referring to their experience with a qualitative approach:

We chose to relate to the subject with as much openness and inquisitiveness as possible without formulating any clear theses until we were in contact with the field. We found that qualitative methodology was best suited to such an approach. In this way, we hoped to capture, as far as possible, the themes that were important to the youngsters whom we interviewed, and about whom this study is based, rather than focusing upon subjects influenced by our own theoretical observations as to what would be relevant. However, such an approach does place great demands of openness on us as researchers. Ideally, one should be open and objective to all the factors one comes across and treat all subject matter with the same degree of interest and understanding.
(Bjørnstad & Ellingsen, SAFT - Onliners. A report about youth and the internet, 2004)

Examples of good practices: combined approaches

Only at an abstract (or purist) epistemological level are quantitative and qualitative approaches likely to be presented as completely incompatible. In most cases, a combination of methods may prove to be more useful. Under different research circumstances both strategies can be (and usually are) combined. In fact, quantitative and qualitative mean different things in different situations. The actual form this combination will take depends, on the one hand, on the objectives and, on the other hand, on research development.

The data resulting from the above "free" qualitative methodology (Bjørnstad & Ellingsen, 2002) were used to formulate questions for the quantitative SAFT study, and to provide explanations and insights for the interpretation of the quantitative data.

In the project Children and their Changing Media Environment (Livingstone, 2002; Livingstone & Bovill, 2001), a qualitative study preceded a quantitative one, which proved to be very helpful when interpreting the quantitative data (Livingstone S. & Lemish, 2001); the same happened with the UK Children Go Online research project. As the authors of the study noticed, "Though often insightful in suggesting themes or trends, qualitative research is best complemented by quantitative research in order to judge the scale and significance of the findings" (Livingstone & Bober, 2004).

A researcher's experience: children's use of computer games

Aiming to analyse how children (7-12) use computer games in their lives and how this activity changes over time, a researcher decided to conduct a survey with the same children at two different moments (Malheiro, 2007). A questionnaire was designed with closed and open questions, the latter oriented to capture feelings and personal evaluation (for instance, "Do you prefer computer games or other things? What kinds of things?", "Do you think you are a good player? Why?"). The questionnaire was pre-tested on ten children.

The pre-test showed that the open questions did not allow the identification of any significant trend. Different concepts were used by the children and it was impossible to estimate patterns. Some open questions were also not answered, maybe because they were focused on the processes and required them to write a lot (such as, "Did you learn quickly? How was it?", "Did you learn new things with computer games?" "What?", "Do you think games are a good way of spending your leisure time?" "Why?").

Based on these results, the researcher decided to use qualitative approaches, mixing interviews with topics (where, when, how or why they play games, what kinds of games, experiences and expectations they have experienced, what were their "unforgettable moments"...) and observations of children playing games.

As the starting point for a new topic, a qualitative approach like this proved to be more productive for the design of questions that were less difficult for children to answer.(Cristina Ponte, Portugal)

A researcher's experience: study on children's reading of animation

Quantitative research is of better use when looking for the general features of a population. It can be particularly limiting when young respondents are involved because of literacy and social gaps between the researcher and the subjects, which make it a challenge to organise a questionnaire capable of motivating active participation. I found a combination of quantitative and qualitative techniques more useful - the first helping to determine similarities and differences in socio-economoc status, and to trace routines and practices, while the second aids the exploration of social dynamics and contextual variations. In my study on children's readings of animation ((Leitão, 2005). I used both methods, employing two research techniques - the questionnaire and the small-group interview - in a school context, working with a class of twenty-two children from the first year of primary school education and a class of twenty-four fourth year children. (Sofia Leitão, Portugal)

 

Share:Facebook|Twitter|LinkedIn|