EDF9850_Activity1_Part1
Overview
Lesa Hoffman uses Activity 1 Apps 1 and 2 to explain how cluster size and the intraclass correlation coefficient (ICC) shape design effects, effective sample size, and the reliability of cluster estimates. She connects these ideas to multilevel-model decisions, contrasting random effects with fixed cluster indicators, and emphasizes that sampling goals and data signal matter more than universal rules of thumb.
Key takeaways
- The design effect is jointly shaped by cluster size and ICC, so an ICC threshold cannot be interpreted without knowing how many people are sampled per cluster.
- Effective sample size is total sample size divided by the design effect, and the design effect equals 1 only when ICC is exactly zero.
- Small clusters produce less reliable cluster means, while a small number of clusters also makes cluster-level variation harder to estimate precisely.
- Estimated random effects are shrunk toward the overall center, especially when a cluster is small or has an extreme mean, so they are not equivalent to directly observed cluster data.
- A clustered sampling design must be accounted for, but researchers can choose among approaches such as random effects or fixed cluster indicators according to the research question and number of clusters.
- Sample-size planning should match the level of the research question: add people within clusters for person-level questions, and add clusters for questions about differences between clusters.
Chapters
- The design effect depends on both the ICC and the number of people per cluster; no nonzero ICC is universally safe to ignore.
- Dividing total sample size by the design effect gives effective sample size, and a design effect of 1 occurs only when the ICC is exactly zero.
- One person per cluster would minimize the design effect, but would defeat the purpose of collecting clustered data.
- Level-one sample size means people per cluster, while level-two sample size means the number of clusters; they affect power and precision in different ways.
- Estimated cluster effects, or Us, are pulled toward the distribution center, especially for clusters with small sample sizes and extreme means.
- Because of this shrinkage, estimated Us should not be treated as observed data, particularly for the most extreme and least reliable clusters.
- A school mean based on 100 students is generally more reliable than one based on 8, just as a 100-question exam provides more information than an 8-question exam.
- For multilevel data, a conditional or adjusted ICC can show how much cluster-level variation remains after adding predictors.
- Sampling design determines that clustering must be addressed, but not necessarily with random effects; fixed cluster indicators are another option.
- With only 3 clusters, treating cluster membership as a categorical predictor may be preferable to estimating a noisy between-cluster variance; with 50 clusters, fixed indicators would require 49 slopes.
- Fixed cluster effects suit questions about specific schools, but do not explain why schools differ and can limit analysis of cluster-level predictors.
- There is no universal 20-by-20 rule: prioritize more people per cluster for person-level questions and more clusters for questions about cluster-level differences.
- For a course project, Hoffman recommends two levels when possible; a three-level model, such as students within classrooms within schools, requires two random intercepts.
Transcript excerpt
0:00 So, what do you think the too long didn't read is for app number one? What is the take home message that it is trying to tell us? Please raise your hands in the room or the Zoom that way we don't talk over each other. >> I don't know what you're talking about. >> You don't know what I'm talking about? Okay. So, rather than going through each of the questions one by one, I thought it would be more efficient just to sort of like get to like what what is the message that you that it's trying to illustrate for you? >> Oh. >> Like that that's what I mean. So, poorly worded. >> There's no ICC that's safe to ignore. >> There is no ICC that's safe to ignore. Yeah. What two factors drive design effects?
0:50 Cluster size. That's one. And ICC, yeah. So, it's literally a multiplier. And so, only when the ICC is exactly zero would you have a design effect of one, which means your total sample size divided by your design effect works out to be your effective sample size. So, if you wanted to design the most efficient study possible, you need one person per cluster. Which defeats the whole purpose, but like from a from a design effect perspective, that's what would be optimal. So, that is why blanket advice like if an ICC is less than whatever cannot be used for all situations. It depends on the cluster size.
1:38 And it changes after you explain variance. So, those are the main things I think that um app one was trying to illustrate. So, anything else? >> Yeah, I have a question about that. >> Yeah, hit me. >> My hand is like virtual hand. I don't know. >> Yes, yes, it is. I saw it.
This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.
In this course
Related notes
Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Lesa Hoffman.