Skip to content

Asking questions

Module items

Assignment

Sample assignment

Reading


Fink, Arlene. 2017. “The Survey Form: Questions, Scales, and Appearance.” Pp. 35–66 in How to conduct surveys: A step-by-step guide. Los Angeles: Sage.

Learning outcomes

  1. Identify four different types of administering surveys (face to face, phone, mail, and online)
  2. Learn behavior, attitude, and attribute survey questions
  3. Identify the principles of question design (reliability and validity)
  4. Learn question types (closed-ended and open-ended)
  5. Learn the principles of wording questions (short, single-barreled, and neutral)

Administering Questionnaires

  • FACE-TO-FACE SURVEY
    • Longer questionnaire
    • Higher response rate
    • Costly
  • PHONE SURVEY
    • Rapid data collection
    • Quality control
    • Sampling problems
  • MAIL SURVEY
    • Large sample size
    • No interviewer effect
    • No opportunity to clarify questions
  • ONLINE SURVEY
    • Reduced cost
    • Automatic data entry
    • Lower response rate

Question content

  1. Behavior
    1. WHAT PEOPLE DO?
    2. Global Warming/Renewable Energy Survey - 2008 (ICPSR 36368)
  2. Attitude
    1. WHAT PEOPLE BELIEVE?
    2. European Social Survey – 2014 (ICPSR 24190)
  3. Attribute
    1. WHO ARE THEY?
    2. New Family Structures Study – 2012 (ICPSR 34392)

Principles of question design

  1. RELIABILITY
    1. Do people answer a question the same way on repeated occasions?
    2. Consistency
    3. National Social Life, Health and Aging Project (ICPSR 36873)
  2. VALIDITY
    1. Does the question measure what it is intended to measure?
    2. Content

Question types

Closed-ended

  • Answers are provided from which respondents are to select one or more responses.
  • Latino National Survey - 2006 (ICPSR 20862)
  • Easier to answer
    • Requires less effort and less facility with words
  • Easier to analyze
    • Enhances standardization and comparability
  • Forcing and manipulating the respondents
    • Forces to give simple responses to complex issues

Open-ended

  • Respondents answer in their own words.
  • Latino National Survey - 2006 (ICPSR 20862)
  • Explores a topic in depth
    • Yields responses the designer may not have contemplated
  • Builds rapport
    • Increases the level of rapport between respondent and researcher
  • Coding problem
    • Time consuming
    • Introduces the variability in the coding of answers

Question types - “Other” option

  • Latino National Survey - 2006 (ICPSR 20862)
  • B8. WHYUS What would you say is the main reason you came to live in the United States? 1. Education 2. Family reunification 3. Escape political turmoil 4. My parents brought me as a child 5. Improve economic situation 6. Other (SPECIFY)

Question types - Data entry errors and “Other” option issues

  • Closed-ended
  • Open-ended

Wording questions (1)

  1. LONG?
    1. On a scale of 1 to 10, with 1 being not concerned at all and 10 being extremely concerned, how concerned are you about the wide-ranging and multifaceted impact of climate change on the environment, including but not limited to its effects on biodiversity, natural resources, ecosystems, weather patterns, sea level rise, and overall planetary health, considering the potential implications for current and future generations, and the need for urgent action to mitigate and adapt to these impacts?
  2. DOUBLE-BARRELED?
    1. Do you believe that poverty and inequality are caused by economic policies and lack of access to education and healthcare?
  3. LEADING?
    1. Don't you agree that stricter gun control laws are necessary to reduce gun violence in our communities?

Wording questions (2)

  1. LONG?
    1. If someone is caught with a small amount of illegal drugs for personal use, in your opinion, what should happen to them: They should be evaluated for drug issues, offered treatment but not be arrested or face jail time; they should spend no more than 3 months in jail; they should spend up to 1 year in jail; they should spend up to 3 years in jail; they should spend up to 5 years in jail; or they should spend 10 or more years in jail?
  2. DOUBLE-BARRELED?
    1. How important to you is addressing prescription drug and other drug abuse in Maine and the recent surge in overdose deaths: very important, somewhat important, not too important, or not important at all?
  3. LEADING?
    1. There are more than 2 million Americans in prison, more people by far than any other country. How important to you is substantially reducing the number of Americans in prison: very important, somewhat important, not too important, or not important at all? - Drug Policy Alliance, 2016. Public Policy Polling (Link)

Response formats

  • Do you agree that people should recycle? 1. Yes 2. No
  • Do you agree that people should recycle? 1. Strongly disagree 2. Mostly disagree 3. Somewhat disagree 4. Neither agree or disagree 5. Somewhat agree 6. Mostly agree 7. Strongly agree