SOC 220 - Sept 10

Overview

Jeff Brassard explains how research designs are evaluated through measurement, internal, and external validity, then compares experiments, cross-sectional studies, longitudinal designs, and case studies. Examples ranging from unreported property crime and Hawthorne effects to North Dakota’s shale boom show how measurement choices, sampling, time, and research settings shape what researchers can conclude and generalize.

Key takeaways

Measurement validity depends on whether an indicator represents the intended concept: police reports can miss unreported property crime, while a single term-end GPA may not capture changing academic performance.
External validity is strengthened by representative samples and weakened by artificial settings or reactive effects; Hawthorne workers changed behavior when they knew they were being observed.
Natural experiments such as North Dakota’s 2011–2020 shale boom let researchers compare communities exposed to a major economic change with nearby areas that did not experience the same boom, but researchers cannot control the event.
Cross-sectional surveys efficiently compare social characteristics at one moment, but a single measurement cannot establish causal direction or show whether findings hold in other years.
Longitudinal panels clarify change by following the same people, but attrition can substantially shrink samples: the National Study of Youth and Religion fell from about 3,000 participants to roughly 1,500 over 20 years.
Case studies prioritize deep, detailed understanding of one social unit over broad generalization; critical, extreme, and revelatory cases serve different research purposes.

Chapters

0:00 Research Design Review and the First Quantitative Assignment

The Sunday mini-assignment asks students to identify a social phenomenon, write a quantitative question, and propose a suitable research method.
Questions can use a relationship format, such as how social-media use relates to depression, and methods may include surveys, experiments, or existing data.
Internal validity concerns whether a proposed cause, X, actually produces an observed effect, Y.

4:08 Measurement Validity: Does the Measure Capture the Intended Concept?

Measurement, or construct, validity asks whether a study’s indicators actually measure its intended concepts; operationalization is central to this problem.
Official police statistics can undercount property crime because victims may not report incidents, such as a car break-in with no significant loss.
GPA is a blunt measure of academic performance over a term; tracking quizzes and assignments alongside sleep or stress can capture change more precisely.
Counting weekly showers as a measure of depression or daily smiles as a measure of happiness risks measuring something other than the intended concept.

11:14 External Validity and Generalizing Beyond the Research Sample

External validity asks whether findings apply beyond the specific research setting and participants.
A well-constructed survey of 3,000 Albertans could support broader conclusions about Alberta public opinion.
Quantitative surveys often prioritize generalizability, while qualitative research and artificial lab experiments commonly have more limited external validity.

14:12 External Validity Threats: Sampling, Settings, and Hawthorne Effects

Nonrepresentative participants and settings unlike everyday life weaken generalizability; a controlled lab task may not reflect behavior outside the lab.
The Hawthorne studies at Hawthorne Electric Works found that workers’ productivity rose during changes to workplace conditions, in part because workers knew they were being observed.
Reactive effects occur when awareness of observation changes behavior, making it difficult to generalize findings to situations where participants are not being studied.
Pretests can affect both internal and external validity if they condition participants or make the experiment unlike the experience of people who were not pretested.

19:44 Replicability and the Social Sciences’ Replication Crisis

A replicable study provides enough detail for another researcher to repeat its procedures and obtain broadly similar results.
Brassard describes a replication crisis in psychology and sociology, where repeated studies sometimes produce results different from the original findings.
Limited replication and auditing leave room for unsupported or fraudulent results, while academic pressure rewards novel, highly cited findings.

24:07 Lab, Field, and Natural Experiments Compared

Lab experiments offer strong control over participants and conditions but often have low external validity because the setting is artificial.
Field experiments, such as staging an event in a campus quad, take place in more natural settings but give researchers less control.
Natural or quasi-experiments use changes that occur without researchers creating them; the North Dakota shale boom of roughly 2011–2020 provides a case.
Researchers can compare North Dakota’s sharp increases in oil-and-gas employment and income with nearby states to study outcomes such as marriage, fertility, and life expectancy.

32:16 Cross-Sectional Design: A One-Time Snapshot of Society

A cross-sectional design measures observations at one point in time, with no before-and-after comparison.
One-off surveys and structured interviews—surveys administered in person using a fixed script—are common cross-sectional methods.
Researchers measure multiple variables, such as age, education, and income alongside health, happiness, or loneliness, to identify patterns and associations.
Cross-sectional analyses can use independent- and dependent-variable language, but researchers do not manipulate those variables.

37:08 Cross-Sectional Limits and Strengths: Causation, Time, and Sampling

A single snapshot may reveal an association without identifying its direction; adult-content use and religiosity could be linked because use reduces religiosity or because less-religious people use more content.
The same ambiguity applies to self-esteem and income: one-time data cannot establish whether confidence raises income or higher income increases confidence.
A 2024 survey can describe its sampled population at that time but cannot automatically establish what was true in 2023 or will be true in 2025.
Representative or random sampling strengthens generalization, and cross-sectional studies are useful for examining attributes such as age, ethnicity, religion, or education that cannot be experimentally assigned.

44:21 Longitudinal Design: Tracking Change and Temporal Order

A longitudinal design measures cases at two or more times; it can last three weeks, several months, or decades.
Repeated measurements at T1 and T2 can show whether one change precedes another, helping researchers assess causal direction.
Brassard describes a University of Alberta project that began in 1972 and repeatedly surveyed the same participants.

48:21 Panel Studies and Cohort Studies: Same People or Similar Groups

Panel studies re-examine the same people, households, or organizations at multiple waves, making individual change easier to track.
The National Study of Youth and Religion followed roughly 3,000 young people from 2002 to 2022, attempting to survey and interview the same participants over 20 years.
Cohort studies repeatedly sample people who share an experience or age range—for example, Albertans born between 1982 and 1992—without necessarily retaining the same individuals.
Cohort designs are easier to maintain than panels but provide less direct evidence about how specific individuals changed.

51:55 Longitudinal Study Challenges: Attrition and Panel Conditioning

Attrition occurs when participants cannot be reached or leave a study; a prison-release project intended to re-interview participants six times over two years lost many after its first waves.
The National Study of Youth and Religion reportedly declined from about 3,000 participants to roughly 1,500 over 20 years as people moved, changed phone numbers, or died.
Researchers must choose wave intervals that balance resources and participant burden against the risk of losing track of people.
Panel conditioning is the possibility that repeated participation changes how respondents think or act, although Brassard says it is not generally a major concern for sociological studies.

55:16 Case Study Design: In-Depth Analysis of a Single Case

A case study conducts an in-depth investigation of one social unit, which could be a person, family, organization, event, country, city, or education system.
Case studies are common in qualitative research, including ethnography, interviews, focus groups, and content analysis, but can also use quantitative statistics.
A case study of Alberta’s education system, for example, could combine statistics from Alberta Education with other sources to explain that particular system.
Case studies generally have limited external validity, but their strength is detailed knowledge of a case that broad cross-sectional or longitudinal designs may not provide.

1:02:15 Critical, Extreme, and Revelatory Cases; Assignment Instructions

A critical case tests where a hypothesis holds or fails; an extreme case examines an unusual instance, such as someone who has been married seven times, to illuminate broader patterns.
A revelatory case becomes possible when researchers gain access to previously unavailable evidence, such as newly opened KGB archives after the collapse of communism in Eastern Europe.
The mini-assignment is due Sunday, September 13, at 11:59 p.m.; students should submit a quantitative research question and name a plausible tool, such as a survey, experiment, or secondary data.
The assignment can be brief: identify the phenomenon, state the question, and choose a research tool; teaching assistants will provide general feedback.

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Jeff Brassard.