Survey modes

Module items¶

Assignment¶

• Face to face surveying
  • See: [[How to submit an assignment]]

Sample assignment¶

• No sample

Reading¶

Learning outcomes¶

1. Distinguish interviewer-administered surveys from self-completion surveys
2. Compare face-to-face, telephone, postal, and online surveys
3. Explain how the survey mode can affect who responds and what answers they give
4. Identify the interviewer effect and social desirability bias
5. Evaluate the advantages and disadvantages of each survey mode
6. Explain how survey modes can increase or reduce different sources of total survey error

[[Survey modes]]¶

• Writing good survey questions is only one part of survey research.
  • We must also decide how respondents will receive and answer those questions.
  • There is no single best survey mode.
    • The best mode depends on the particular study.
      • This decision is called the survey mode.

• Surveys can be organized into two broad groups:

  • flowchart TD
        A[Survey modes] --> B[Interviewer-administered]
        A --> C[Self-completion]
        B --> D[Face-to-face survey]
        B --> E[Telephone survey]
        C --> F[Postal survey]
        C --> G[Online survey]

• In an [[interviewer-administered survey]], an interviewer asks the questions and records the answers.

  1. [[Face-to-face surveys]],
  2. [[Telephone surveys]],
• In a [[self-completion survey]], respondents read the questions and record their own answers.
  1. [[Postal surveys]], and
  2. [[Online surveys]].

[[Face-to-face surveys]]¶

• A trained interviewer meets the respondent in person.
• The interviewer:
  • Reads each question,
  • Records each answer,
  • Follows the required question order, and
  • Clarifies instructions when necessary.

Advantages of [[face-to-face surveys]]¶

• The interviewer can build rapport with the respondent.
• The interviewer can explain the task and clarify misunderstandings.
• The researcher has more control over:
  • Who answers the survey,
  • The order of the questions, and
  • The completeness of the questionnaire.
• Longer and more complex questionnaires are possible.
• Respondents with limited reading skills can still participate.
• Visual materials and complex response categories can be shown to respondents.
• Response rates are often higher than in other survey modes.

Disadvantages of [[face-to-face surveys]]¶

• They are expensive.
  • Interviewers must be hired, trained, supervised, and paid for travel time.
• They take more time to organize and complete.
• A geographically scattered sample increases travel costs.
• Interviewers may influence respondents' answers, which is called [[interviewer effect]].
• Respondents may hide sensitive or socially undesirable behavior, which is called [[social desirability bias]].
• Interviewer safety and access to homes can become practical problems.

[[Interviewer effect]]¶

• The presence or characteristics of an interviewer can affect the answers respondents give.
• Possible influences include the interviewer's:
  • Age,
  • Gender,
  • Race or ethnicity,
  • Accent,
  • Clothing,
  • Facial expressions, and
  • Tone of voice.
• Different interviewers may also read questions or record answers differently.
• These differences create [[interviewer variance]].

[[Social desirability bias]]¶

• Respondents may give an answer that appears respectable or socially acceptable.
• They may:
  • Overreport socially approved behavior, or
  • Underreport socially disapproved behavior.
• Social desirability bias can occur in any survey.
• It is often stronger when respondents give answers directly to another person.
• Sensitive questions may therefore work better in a private self-completion section.
• To reduce the interviewer effect:
  • Train interviewers to:
    • Read every question exactly as written,
    • Keep a neutral tone and expression,
    • Avoid approving or disapproving answers,
    • Use only standardized explanations, and
    • Record answers accurately.
  • Monitor interviewers and check their completed surveys.
  • Training can reduce the interviewer effect.
    • It cannot guarantee that the effect disappears.

[[Telephone surveys]]¶

• A trained interviewer asks questions by telephone.
• Most large telephone surveys use computer-assisted telephone interviewing.
• The computer can:
  • Display the next question,
  • Apply skip patterns,
  • Check whether an answer is valid, and
  • Record the response directly.
• Telephone surveys are interviewer-administered, but the interviewer and respondent do not see each other.

Advantages of [[telephone surveys]]¶

• They are usually faster and less expensive than face-to-face surveys.
• A geographically scattered sample can be reached without travel.
• Interviewers can clarify questions and encourage respondents to continue.
• Centralized interviewing makes supervision easier.
• Question order and skip patterns can be controlled.
• The absence of face-to-face contact may make some sensitive questions easier to answer.

Disadvantages of [[telephone surveys]]¶

• Respondents cannot see:
  • Maps,
  • Images,
  • Long lists, or
  • Complicated response categories.
• People can screen or reject unfamiliar calls.
• A telephone sampling frame may exclude or underrepresent some groups.
• The interviewer effect and social desirability bias are still possible.

• Questions and response categories must be short and easy to remember.

  • Telephone questions are heard only once

    • Respondents must listen, understand, remember the response categories, and answer.
    • A question that works on paper may be too complicated by telephone.
    • Long questions and long response lists should be shortened or divided.

[[Postal surveys]]¶

• A questionnaire is mailed to a sampled person or household.
• The respondent completes and returns it without an interviewer.
• The package usually includes:
  • A cover letter,
  • Clear instructions,
  • The questionnaire, and
  • A prepaid return envelope.
• Follow-up reminders and replacement questionnaires are often necessary.

Advantages of [[postal surveys]]¶

• They cost less than face-to-face surveys.
• A geographically scattered sample can be reached.
• There is no interviewer effect.
• Respondents can answer at a time that is convenient for them.
• Respondents can check personal or household records.
• Questions and response categories can be presented visually.
• Greater privacy may improve answers to sensitive questions.

Disadvantages of [[postal surveys]]¶

• Data collection is slow because mailing and follow-up take time.
• The questionnaire must explain itself.
  • No interviewer is present to clarify a confusing question.
• Respondents can skip questions or answer them out of order.
• The researcher cannot be certain that the sampled person completed the survey.
• People with limited reading or writing skills may be excluded.
• Returned questionnaires may require manual coding and data entry.
• Nonresponse can be high when respondents are not interested in the topic.

[[Online surveys]]¶

• Respondents complete a digital questionnaire without an interviewer.
• The survey may be distributed through:
  • Email invitations,
  • Text messages,
  • A website,
  • A course or workplace system, or
  • A survey panel.
• A link does not create a sample.
  • The researcher must still decide who should receive the invitation.

Advantages of [[online surveys]]¶

• Data collection can be fast and inexpensive.
• Large and geographically scattered samples can be contacted.
• Answers enter the dataset automatically.
  • This saves time and reduces data-entry errors.
• The survey can apply:
  • Automatic skip patterns,
  • Response validation,
  • Randomized question order, and
  • Personalized question wording.
• Images and other visual materials can be included.
• There is no interviewer effect.

Disadvantages of [[online surveys]]¶

• People without reliable internet access or digital skills may be excluded.
• Email invitations can be ignored, filtered, or treated as spam.
• The researcher may not know who completed the survey.
• Respondents may quit when the survey is long, slow, or difficult to use.
• Questions may appear differently across phones, tablets, and computers.

• Respondents may worry that their answers can be connected to their identity.

  • An online survey is not automatically representative

    • A large number of responses does not fix a poor sample.
    • A public survey link usually produces a volunteer sample.
    • People who notice the link and choose to respond may differ from the target population.
    • Sampling and survey administration are separate decisions.

Comparing the four [[survey modes]]¶

• Survey mode	Interviewer	Presentation	Questionnaire
  Face-to-face	Present	Visual and auditory	Can be long and complex
  Telephone	Present	Mainly auditory	Must use short questions and response lists
  Postal	Absent	Visual	Must be completely self-explanatory
  Online	Absent	Mainly visual	Can use automatic skips and validation

• Survey mode	Relative cost	Relative speed	Common threat
  Face-to-face	High	Slow	Interviewer and social desirability effects
  Telephone	Medium	Fast	Contact difficulties and auditory limitations
  Postal	Low	Slow	Nonresponse and manual processing
  Online	Low	Very fast	Coverage, breakoff, and device differences

[[Mode effect]]¶

• Survey mode can affect how respondents understand and answer the same question. When the mode changes the answer, this is called a [[mode effect]].
• Respondents may react differently because of:
  • The presence or absence of an interviewer,
  • Whether the question is heard or read,
  • The amount of privacy,
  • The response categories they can see, and
  • The amount of time they have to answer.
• Face-to-face, postal, and online surveys can show response categories.
• Telephone surveys usually present response categories by sound only.
• Visual presentation allows respondents to:
  • Read categories more than once,
  • Compare categories, and
  • Use long lists more easily.
  • Visual lists may produce a [[primacy effect]]:
    • Respondents may be more likely to choose categories near the beginning of the list.
  • Auditory lists may produce a [[recency effect]]:
    • Respondents may be more likely to choose categories heard near the end of the list.
  • Therefore, the same response list may produce different answers across modes.

Main [[survey errors]]¶

• (1) [[Coverage error]]: some members of the target population cannot be reached through the sampling frame.
  • Each survey mode can create a different coverage problem:
    • Face-to-face: people in inaccessible buildings or distant locations may be difficult to reach.
    • Telephone: people without usable contact numbers may be missed.
    • Postal: people without stable or accurate addresses may be missed.
    • Online: people without reliable internet access or digital skills may be missed.
• (2) [[Nonresponse error]]: people who do not respond differ from those who respond.
  • The [[response rate]] is the percentage of eligible sampled people who complete the survey.
    • A higher response rate is desirable.
  • Nonresponse bias occurs when people who do not respond differ in an important way from those who do respond.
  • There are two forms of [[nonresponse error|nonresponse errors]]:
    • [[Unit nonresponse]]: a sampled person does not participate at all.
    • [[Item nonresponse]]: a respondent participates but skips a particular question.
• (3) [[Measurement error]]: the answer recorded is different from the answer the question was intended to measure.
• (4) [[Processing error]]: mistakes occur while answers are coded, entered, edited, or prepared for analysis.