Home > Department of Media and Communications > Research > EU Kids Online > Best Practice Guide > FAQ 16: What are the best ways to construct a survey questionnaire?

 

FAQ 16: What are the best ways to construct a survey questionnaire?

What's the issue?

Writing a survey questionnaire requires care and attention to the design and wording, as well as to the means of administering the survey and recording responses, especially when the respondents are children. The answers should be reliable (i.e. they provide consistent measures in comparable situations) and valid (i.e. they correspond to what they are intended to measure). In that sense, a good questionnaire maximises the relationship between the answers given with respect to a particular question and what the research wants to measure through that question (facts, perceptions, experiences, etc).

Common practice

  • Once survey objectives are stated explicitly, the questions to be asked should be clear.
     
  • Almost all questions in a questionnaire should be asked using a standardised format for both question and answer, in order to produce answers that can be readily compared and that the child can produce answers reliably.
     
  • For each section, state whether single or multiple answers are permitted. Try to convey the same type of information in the same way throughout the questionnaire (Dillman, 2000) and use answer spaces consistently.
     
  • Questions may be asked using either closed questions (i.e. a list of acceptable responses is provided) or open questions (i.e. no list of acceptable questions is provided). Although open questions permit the researcher to obtain unanticipated answers or answers in the respondent's own words, they take a long time to complete. Moreover, the closed questions produce more analytically useful and reliable data.
     
  • Standard response options include agree/disagree questions (these are generally preferable to yes/no questions), and a scale is often used. A five point scale suffices for most purposes, and it is useful to code the negative pole as '1' and the positive pole as '5': for example - 'strongly disagree', 'disagree', 'partly agree and partly disagree', 'agree', 'strongly agree'.

Pitfalls to avoid

  • Format and wording pitfalls must be avoided especially when the self-completion questionnaire is employed (common in research with children). In self-completion surveys, the formatting is even more important than in other data collection procedures, as in this case there are no trained interviewers to guide and encourage the respondents.
     
  • It works best if a self-completion questionnaire is self-explanatory (no further instructions required), if only closed questions are included, and if there are few question formats (to reduce confusion). It is important that the question is interpreted in the same way by all respondents, so avoid words that are ambiguous or may be understood in different ways.
     
  • A questionnaire will be poorly designed if it is cluttered, gives too many instructions, or does not leave enough space between questions. The layout should clearly differentiate instructions, questions and response options.
     
  • Complex skip patterns (i.e. occasions where the question flow varies depending on the responses given) are a common fault and should be kept to a minimum (if necessary, use arrows and boxes that communicate skips without verbal instructions).
     
  • If a researcher fails to establish a conversational style in the sequence of questions, children in particular may feel distant from the context and subject matter of the research. However, the tone should be fairly neutral, not judgemental or patronising.
     
  • For each question, any ambiguous words and concepts need to be clarified. Yet at the same time, questions need to be short and simple. Long complex questions are best broken down into a series of short simple questions. Yet at the same time, a "multi question approach" lengthens the questionnaire which can lead to non-response, so consider what counts as the right amount of questions.
     
  • In order to ensure good measurement, unless measuring the knowledge is the goal of the question, all respondents should have access to the information needed to answer the question from their experience. What constitutes an adequate answer should be consistently communicated.
     
  • Try to avoid strong negative words (forbid, ban, restrain, oppose).
     
  • Try to avoid long list of response choices in order not to confuse respondents.
     
  • Overall, lengthy questionnaires should also be avoided when children are participants. It can be tiring and lower the response rates or even affect the accuracy of the answers.

Questions to consider

After a pilot test, why are some questions not answered? Are all response options used appropriately? Do some answers suggest response biases that could be corrected? How long does the questionnaire take to complete? Did all respondents understand what they were meant to do? Are all the questions really needed? What exactly is being measured with each question and how will the data be analysed?

Further resources

Fowler, F. J. (1993). Survey Research Methods (2nd ed.). Newbury Park; London; New Delhi: Sage Publications
Frey, J. H. (1989). Survey Research by Telephone (2nd ed.). Newbury Park; London; New Delhi: Sage Publications
The SAFT (Safety Awareness Facts and Tools) Project. SAFT 2006 Parent and Children survey. 2004-2006. Norwegian Action Plan for Children, Youth and the Internet and the European Commission Safer Internet Action Plan: Norwegian Media Authority.
UK Children Go Online Project. UK Children Go Online: Emerging Opportunities and Dangers. London: London School of Economics and Political Science.

Good practice

A golden rule, when constructing a survey questionnaire, is to ask yourself three questions: A) Can the respondent understand the questions? B) Is the respondent able to answer the questions? C) Is the respondent willing to answer the questions? We need to be cautious of using common words/expressions. To the question: "What proportion of your evening viewing time do you spend watching news programmes?", Belson (1981) found in his research that only &/188; of respondents interpreted "proportion" as a "part", "fraction", "percentage". About ⅓ saw it as quantitative such as "how long", "how many hours", "how often". A larger group tapped other dimensions entirely such as "when they watch", "which programmes", "which channels". Therefore, try to avoid such common words or try to be as specific as possible about what we mean to ask.
(Bojana Lobe, Slovenia)

Researcher's experience

Mainly because of budget and time constraints our questionnaire was designed and piloted in the country of residence of the researcher (Belgium) instead of in the country where the data collection had to take place (Chile). Moreover, the questionnaire was piloted with 1st year bachelor students instead of with school children (the actual sample population). As a consequence, the English pilot questionnaire was not really useful in revealing essential problems such as language issues present in the Spanish version. Moreover, and probably due to the fact that the questionnaire was piloted with an older university population, we were not able to detect on time that our questionnaire was too lengthy for a secondary school population.
(Verónica Donoso, Belgium)

Share:Facebook|Twitter|LinkedIn|