M303 Introduction to Survey Methods
Where we are
Last class
Qualitative and observational methods
Today
Survey methods and mode choice
Coming next
Attitude measurement and scaling
Learning Objectives
You should be able to
Understand the advantages of surveys and the issues associated with them
Understand different types of surveys and pros and cons of each type
Problem
Formulation
Research
Design
Data
Collection
Data Analysis &
Recommendation
Secondary Data
Primary Data
Observational and Qualitative Methods
Surveys
Sampling
Prior-class review
All behaviors can be observed.
In order to answer "why" questions, one can use observation rather than questioning.
Cognitive phenomena can’t be observed.
A systematic process of recording behaviors without normally communicating with the people involved is called observation.
One can use qualitative methods in order to get a better idea about things that can’t be directly observed and/or measured.
Nondirective interviews are especially effective with busy executives, technical experts, and thought leaders.
In semi-structured or focused interviews, the respondent is given the maximum freedom to respond.
In symbolic analysis, the interviewer probes step by step to the question of interest.
When direct questions fail, one might use projective techniques.
What information can be acquired from surveys?
Knowledge
Attitudes
Assessments
Decisions
Demographics & Lifestyles
Social contacts and interactions
Why Use Survey Research?
Surveys are the overwhelming choice of researchers for collecting quantitative primary data.
Surveys have many advantages such as
Cost & speed
Accuracy
Efficiency
Flexibility
Qualitative and observational methods are more likely to be used to improve and/or supplement the survey methods rather than take their place.
What are the potential issues in surveys?
To obtain meaningful results:
The population must be defined correctly
The sample should represent the population
Selected respondents must be able and willing to cooperate
Respondents must have the required knowledge, opinions, attitudes, or facts
Respondents must understand the questions
The interviewer must understand and record responses correctly
What are the types of surveys?
Survey method: Telephone Interviews
Telephone interviewing is a long-established method for obtaining information from large samples.
Selecting telephone numbers
Pre-specified list
A directory
Random dialing procedure
Random digit dialing
Systematic random digit dialing (SRDD)
Plus-one dialing
Listed number 314-327-2989 → you call 314-327-2990
What are the pros of Telephone Interviews?
More interviews can be conducted in a given time period
Travel time is saved
More hours of the day/week are productive
Less sample bias due to nonresponse
Ease of callbacks at lower cost (WATS)
What are the cons of Telephone Interviews?
Inability to employ visual aids or complex tasks
Interviewer must rely solely on verbal cues to judge the reaction and understanding of respondents
Cannot be longer than 5–10 minutes, or the respondent may lose interest
Relatively little data can be collected
A capable interviewer is essential
Potential for sample bias
Coverage gaps: unreachable people, unlisted numbers, and mobile-only or screened-call populations
Survey method: Self-Administered Surveys
No interviewer involved
The survey is distributed by email, mail, fax, or by hand
How to conduct Self-Administered Surveys?
The method of contact should be determined
Cover letter should be prepared
Questionnaire length, content, layout, color, and format should be chosen
Incentives might be given to increase the response rates
Submitting responses should be easy for the respondent
What is the response rate in Self-Administered Surveys?
Low survey quality => low response rate => sample bias
Example: If you contacted 1,000 participants and the response rate is 5%, then what is the number of complete surveys?
What are the factors affecting the Response Rate?
Perceived amount of work required.
Long surveys → low response rate
Difficult surveys → low response rate
Intrinsic interest in the topic
Low-interest topics → low response rate
Level of motivation
Monetary incentives → high response rate
Characteristics of the sample
What are the pros of self-administered surveys?
Lower cost
Larger samples
Time savings
No interviewer bias
Survey answered at the respondent’s discretion
What are the cons of self-administered surveys?
The absence of an interviewer provides limited control over
Understanding the identity of respondents
Whom the respondent might consult for help in understanding the questions
The speed of the response
The order in which the questions are presented and answered
Survey method: Personal Interviews
A face-to-face survey method.
The personal interview process is characterized by the interaction of four entities.
What are the types of Personal Interviews?
Door-to-door interviewing
Interviews with respondents in their homes
Executive interviewing
Interviews with business people at their offices
Mall-intercept interviews
Interviews with mall shoppers in shopping centers
What are the types of Personal Interviews?
Purchase-intercept technique
Interviews with customers after observing them make purchases in stores
Omnibus surveys
Regularly scheduled interviews/surveys with questions provided by a number of separate clients
What are the pros and cons of different personal-interview types?
Door-to-Door interviewing
Executive interviewing
Omnibus surveys
Purchase intercept technique
Mall intercept surveys
Advantages and Disadvantages of Each Interview Type
Respondents’ buying experiences are salient
Respondents may not be a representative sample
Very expensive to set up meetings
The interviewer has to go to the respondent’s office
Long waiting time and cancellations are common
Relatively lower costs
Respondents may not be a representative sample
Low costs – clients share costs
Good solution if questions are limited and more standard
Suitable for tracking periodically and before-after studies
Relatively expensive
Comfortable environment for respondents
Extensive illustrations and in-depth explanations are possible
Viable way for in-home product tests
What are the pros of Personal Interviews?
Can arouse and keep interest
Ask complex questions with the help of visual and other aids
Clarify misunderstandings
High degree of flexibility
Good for neutral questions
What are the cons of Personal Interviews?
Respondent and interviewer biases
Lack of respondent anonymity
Embarrassing/personal questions
Time Requirements
Cost per completed interview is high
Trained interviewers (geographically near the sample) are required
When should you use Personal Interviews?
A large amount of information per respondent
Complex questions
Required tasks
Product test
No other choices
No phone number
No mailing address
No email address
Survey method: Combination of Survey Methods
Each survey method has different pros and cons, therefore, a combination of survey methods can retain the best features while minimizing the limitations.
Example: Telephone pre-notification approach
What is Telephone Pre-notification?
Call to ask permission before mailing the survey
Advantages
Gain agreement
Signals that the respondent is serious about cooperating
Disadvantages
Requires both telephone numbers and mailing addresses for respondents.
Agreement
What is Lockbox Approach?
Mail a locked box containing the questionnaire and a gift; the lock combination or PIN is provided immediately before the interview.
The resulting curiosity can produce response rates as high as 50% to 70%.
How to choose a survey method?
Type of population – reading and writing skills, motivation to cooperate
Sampling – how to draw a representative sample
Questions – complex vs. easy, general vs. sensitive
Response rates
Costs
Available facilities and trained interviewers
Duration of data collection
Today’s Class: Quiz
Surveys can be used to understand consumers' perceptions about products and services.
Surveys can be used to collect information about consumer demographics and lifestyles.
Four advantages of surveys are their low cost, high accuracy, efficiency and flexibility.
Qualitative methods can easily be used to replace surveys.
In order to obtain meaningful results from a survey, one has to define the population of interest correctly.
In a telephone survey, one might use phone directories to select the telephone numbers to call.
The cost of telephone interviews is typically higher than face-to-face interviews.
If one wants to employ visual aids or complex tasks, face-to-face interviews become the best survey method.
Today’s class: quiz (continued)
In telephone interviews, the interviewer must rely solely on verbal cues to judge the reaction and understanding of respondents.
If you contacted 1,000 participants and the response rate is 10%, then the number of complete surveys should be 100.
The response rate in a self-administered survey decreases as the survey becomes longer.
Door to door interviewing refers to interviews with mall shoppers in shopping centers.
In an omnibus survey, multiple clients have sections of questions in the survey; this way the cost of conducting the survey is shared among these clients.
Among the personal interview methods, the most costly one is the mall intercept surveys.
If you do not have any contact information such as phone numbers, mailing and email addresses, personal interviews become the feasible survey method.
In the telephone pre-notification method, you call to ask the respondent's permission before mailing the survey.
To sum up
Surveys
Pros
Limitations
Types of surveys and their pros and cons
Telephone interviews
Self-administered surveys
Personal interviews
Door-to-door, executive, mall intercept, purchase intercept, omnibus
Combination of survey methods
Next class
Exam 1 — check Canvas for the schedule
Next topic — Attitude Measurement and Scaling