Intro to Statistics at IU Indy - Probability and Nonprobability Sampling
Overview
Andy Baker explains how sampling methods shape what researchers can legitimately conclude about a population, using grocery-store food samples, classroom polls, and the General Social Survey (GSS) as examples. He distinguishes systematic bias from random sampling error, compares probability methods such as simple random, stratified, and cluster sampling with non-probability methods, and shows how the research question and available sampling frame determine the appropriate approach.
Key takeaways
- A probability sample supports population-level estimates because each person has a known, non-zero chance of selection, allowing researchers to quantify sampling error.
- Confidence intervals describe random sampling error, not systematic bias; researchers must prevent bias through sample design and an adequate sampling frame.
- Stratified sampling is useful when comparisons between groups matter, because randomly sampling within each group helps ensure enough participants from each group.
- Cluster sampling can make large-area surveys more practical by selecting places first, but similarities among people in the same cluster can increase standard errors.
- The GSS uses probability sampling, so its 2024 results can estimate values for US adults; a first-day class poll only describes the participants who answered.
- Purposive or snowball sampling may suit research on hard-to-reach groups, such as struggling first-generation students, even though those samples do not justify the same population-wide claims.
Chapters
- A sample is the subset of a population researchers collect data from; a sample of students attending class cannot automatically represent every IU Indy student.
- A Chick-fil-A food sample is biased in two ways: the store chooses which item to promote, and shoppers decide whether to stop.
- Confirmation bias can make people notice favorable reactions while overlooking shoppers who pass by or reject the sample.
- Convenience samples and class polls describe the people who participated, not necessarily the wider population.
- The population is the full group of interest, such as US adults for the GSS; the sampling frame is the list used to select participants, such as an enrollment roster.
- An incomplete frame can produce bias even when names are randomly selected, because eligible people may be missing or ineligible people may remain on the list.
- Selection bias results from systematically restricting participants, such as surveying only people under 30 or only people with bachelor's or graduate degrees.
- Sampling error is the ordinary difference between a sample and its population; confidence intervals describe this error but cannot correct bias.
- Probability sampling gives every population member a known, non-zero selection chance, supporting confidence intervals and hypothesis tests about a defined population.
- Simple random sampling gives everyone an equal chance; stratified sampling selects within groups, such as first-generation and continuing-generation students, to ensure group representation.
- Cluster sampling selects groups such as counties or neighborhoods before sampling individuals, reducing fieldwork costs but potentially increasing standard errors because people in a cluster may be similar.
- Convenience, purposive, and snowball sampling can help with exploratory research or hard-to-reach groups, but do not support the same population-level inference; method choice depends on the research goal and available lists.
Transcript excerpt
0:00 Now let's talk about how researchers choose who ends up in the data. We almost never study every person in a population. We study a sample. The method we use to choose that sample decides what we can later say about the larger group. We will cover two big families of methods, probability sampling and non-probability sampling. We're going to use the GSS data that we already know and by the end we'll be able to tell the difference and know which is more useful given our interests. If I'm only talking to the students that hear me, I cannot claim that I've learned about every single IU Indy student. Those of you who are here with me and listening may be different than the people who aren't here and not listening. Sampling is the decision about which people end up in the data. A good sampling plan does not remove all
0:49 error. It makes the error random so we can measure it later with confidence intervals. A poor sampling plan can hide whole groups and give us a misleading picture of the population we care about. Think about the free food sample at a grocery store or at a Chick-fil-A. The person handing out the sample is not giving a random bite of everything in the store. They are offering one specific item, usually something new or something they want you to buy. The people who stop are not a random sample of every shopper either. They are the people who have time, who like free food, or who are already walking down that aisle. If you tasted that sample and then said, "Hmm, this represents all of the food in the store." you'd obviously be wrong. That is the same problem we have with a convenience
1:38 sample. The people who end up in the
This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.
Related notes
Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Andy Baker.