SOC 220 - October 1

Overview

Jeff Brassard explains practical survey-design choices, including balanced response scales, vignette and neighbor questions, pilot testing, and validated measures such as the 20-item UCLA Loneliness Scale. He compares questionnaires, web surveys, researcher-driven diaries, and secondary data analysis, using examples from the 2016 U.S. election, Qualtrics routing, religious-attendance research, and Ryan Burge’s work; he closes with requirements for a 10-question survey assignment due October 11.

Key takeaways

Pilot every survey before launch: testing can reveal misunderstood wording, inaccurate completion-time estimates, and broken Qualtrics routing that could prevent respondents from finishing.
Indirect question formats such as the neighbor question can reduce the social pressure of reporting a contentious personal opinion, but they may still capture projection rather than an independent estimate of others.
Validated instruments such as the 20-item UCLA Loneliness Index provide established response formats and scoring procedures, making findings easier to interpret and compare across studies.
Online questionnaires can collect 1,500 responses for an estimated $12,000, far below Brassard’s approximate $120,000 estimate for structured interviews, but cost savings do not remove sampling or access limitations.
Researcher-driven diaries can capture sensitive, time-sequenced experiences that interviews may miss, as Donna Freitas’s online study of about 800 college students’ sex lives illustrates.
Secondary analysis can be inexpensive and support decades-long comparisons, but researchers cannot control the original data quality or recover variables that the survey never collected.

Chapters

0:00 Course Logistics: October 15 Midterm and Mini-Assignment Deadlines

The October 15 midterm is in Education South, room 129, on the first floor near the hospital—not in the regular classroom.
Mini-assignment three is due October 11; students can ask Jeff Brassard about feedback on the second assignment during office hours or by appointment.
Brassard warns students to complete every template field, including consent-letter blanks, rather than submitting an unfinished assignment at the last minute.

6:50 Survey Question Basics: Balanced Scales, Recall, and Question Order

Keep response options symmetrical and balanced; a common scale has two positive choices, a neutral choice, and two negative choices.
Avoid asking respondents to recall events too far in the past; short recall periods generally yield more reliable answers.
Use “don't know” selectively—especially as a knowledge filter—and consider “prefer not to answer” or no opt-out for other questions.
Keep survey question order consistent unless the research specifically tests order effects; put topical questions early and sensitive questions after respondents are engaged.

10:28 Vignette Questions Create Distance but Still Invite Social-Desirability Bias

Vignettes describe hypothetical scenarios and ask how a respondent would react, separating the answer from a real personal event.
A scenario about encountering a person begging outside a grocery store can probe moral judgments without asking respondents to recount an actual interaction.
Vignettes may encourage more candid answers on sensitive topics, but respondents can still choose the socially approved response—for example, claiming they would give money.

13:50 The Neighbor Question and the 2016 U.S. Election Polling Miss

Instead of asking whom respondents planned to vote for, one pollster asked whom they thought their neighbors would vote for—the “neighbor question.”
While many election forecasts put Hillary Clinton’s chances near 80%, the neighbor-question poll indicated Donald Trump had a stronger chance of winning.
The method can reduce pressure to report a socially stigmatized personal opinion, though answers may still reflect respondents projecting their own views onto others.
Brassard suggests considering this indirect approach for contentious topics such as abortion or medical assistance in dying.

19:13 Pilot Surveys Catch Confusing Questions, Broken Routing, and Bad Time Estimates

Test surveys with a small sample before paying to recruit a large one; friends or family can help identify unclear, embarrassing, jargony, or uninteresting questions.
For online tools such as Qualtrics, test every routing path: a screening answer might send someone to question four or skip ahead to question 15.
Researchers may need to run a survey repeatedly for every possible response path; Brassard describes groups of about 60 graduate students testing an iPad survey.
Pilot testing establishes realistic completion times; overstating a 30-minute survey as 15 minutes can prompt participants to quit.

26:28 Validated Scales: Reuse Established Measures Like the UCLA Loneliness Index

Existing validated scales let researchers measure constructs such as depression, loneliness, ADHD, or gambling problems without inventing new measures.
The UCLA Loneliness Index uses 20 questions on a four-point never-to-always scale, including items about feeling left out or lacking companionship.
Researchers reverse-code positively worded items and average responses so that higher scores represent greater loneliness.
Cite the scale when using it; some measures require a fee, but many publicly available instruments only require proper attribution.

34:09 Questionnaires and Web Forms: Keep Survey Design Short and Easy to Follow

A questionnaire is a structured interview without an interviewer; the Canadian census is a familiar example that has shifted from mailed forms to online completion.
Closed questions are generally easier to complete than open responses, which require more typing and can discourage participation.
Questionnaires should be simple, clear, and relatively short because frustration and effort can lead respondents to abandon them.
Survey platforms such as SurveyMonkey and Qualtrics support online forms and data exports for analysis in tools such as SPSS or R.

40:04 Questionnaire Advantages: Lower Cost, Faster Fieldwork, and Less Interviewer Pressure

Online questionnaires can be much cheaper and faster than structured interviews: Brassard estimates that 1,500 online responses might cost about $12,000, versus roughly $120,000 for interviews.
A 500-person Edmonton poll reached its quota in about 18 hours, illustrating how quickly online samples can be collected.
Respondents may answer sensitive questions more honestly without an interviewer present, reducing—but not eliminating—social-desirability bias.
A phone-location study of visits to places of worship estimated weekly attendance closer to 5%, compared with roughly 25% of U.S. adults reporting weekly attendance in surveys.

45:11 Questionnaire Limitations and the Shift to Web Surveys

Without an interviewer, researchers cannot clarify questions or ask follow-ups; open-ended responses and nonresponse are also persistent challenges.
Online questionnaires may exclude people with limited internet access or populations such as unhoused people, for whom structured interviews can work better.
Long-running surveys such as the U.S. General Social Survey and the British Social Attitudes Survey have used phone methods; Brassard notes the GSS moved online around 2015.
Web forms support mobile completion, skip logic, and automatic database collection, but their sampling limitations still need to be considered.

50:03 Questionnaire Presentation: Keep Questions and Response Options Together

Use clear layouts that work on phones and tablets, where many respondents now complete surveys.
Horizontal rows of response options are common online, but repeated grids can encourage respondents to select the same answer down a series of questions.
Keep each question with its response options; page breaks that separate them can confuse respondents, a problem Brassard has encountered in multiple-choice exams.

51:38 Researcher-Driven Diaries Track Behavior and Sensitive Experiences Over Time

Diaries record behaviors, feelings, or perceptions shortly after they occur, often once a day for a short period such as a week.
Quantitative examples include tracking food intake or media use; one media diary asked Brassard and his daughter to log streaming and other habits, with escalating incentives.
Researcher Donna Freitas used online diaries to study college students’ sex lives after interviews proved awkward, generating detailed accounts from about 800 students across evangelical, Catholic, public, and private colleges.
Diaries can capture timing and behavioral sequences, but participants may tire of daily entries, provide sparse details, or fill in missed days from memory.

59:21 Secondary Survey Analysis: Build Research from Existing Datasets

Researchers can build substantial careers without collecting new survey data by analyzing publicly available datasets and asking new questions of existing results.
Brassard highlights political scientist Ryan Burge, whose research on religion relies heavily on secondary data analysis and whose Substack is called Graphs about Religion.
Existing datasets can reduce research costs to near zero, support longitudinal and subgroup comparisons, and allow researchers to revisit decades of findings.
Limitations include learning complex datasets, ecological fallacy when applying group-level patterns to individuals, lack of control over data quality, and missing variables that cannot be added retroactively.

1:04:22 Survey Assignment: Write Ten Questions Without Demographic Shortcuts

Mini-assignment three requires a 10-question survey built around the student’s quantitative research question, with possible responses supplied.
Include a mix of question types where appropriate; submitting only open questions earns at most five out of ten.
Do not use generic demographic items such as age or gender unless the research topic itself calls for them.
The assignment is due October 11, and the midterm remains scheduled for October 15 in Education South room 129.

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Jeff Brassard.