STAT 638 (Fall 2026), Lecture 26

Overview

Samiran Sinha reviews collapsed Gibbs sampling as an option for reducing MCMC autocorrelation, then develops Bayesian count regression using Poisson, zero-inflated Poisson, negative binomial, and zero-inflated negative binomial models. Using an extramarital-affairs dataset and R's brms package, he explains log links, coefficient interpretation, marginal effects, model comparison by log marginal likelihood, and posterior predictive checks.

Key takeaways

Collapsed Gibbs sampling can reduce autocorrelation by integrating out selected parameters, but some update schemes yield valid marginal draws without preserving the complete joint posterior.
For Poisson regression, λᵢ = exp(β₀ + Xᵢᵀβ) guarantees a positive rate and makes the log of the conditional mean a linear predictor.
Marginal effects can be computed as posterior contrasts between covariate settings and averaged over observed values of other predictors, but these associations are not automatically causal effects.
The example's zero-inflated Poisson estimate ω ≈ 0.48 and log marginal likelihood -54.25 improve on the ordinary Poisson model's -59.45.
Among the four fitted count models, the negative binomial has the highest reported log marginal likelihood, -49.84, outperforming the zero-inflated negative binomial at about -51.
Posterior predictive checks assess model adequacy by comparing observed summaries, such as the proportion of zeros, with summaries generated from the posterior predictive distribution.

Chapters

0:00 Collapsed Gibbs Sampling and MCMC Autocorrelation

Sinha contrasts ordinary Gibbs sampling, which draws each parameter from its full conditional, with collapsed sampling, which integrates out selected parameters.
Collapsing can reduce dependence between successive draws, but the resulting parameter vector may not represent the full joint posterior when updates use incompatible marginal distributions.
The choice of block or collapsed sampling depends on which marginal distributions are practical to derive and whether marginal inference is sufficient.

7:55 Poisson Regression: Log Link, Likelihood, and Count Outcomes

For independent observations, Sinha models each count outcome Yᵢ given covariates Xᵢ as Poisson with rate λᵢ.
The model λᵢ = exp(β₀ + Xᵢᵀβ) keeps the rate positive and gives the canonical Poisson log link: log(λᵢ) = β₀ + Xᵢᵀβ.
Bayesian inference combines the product of Poisson probabilities across observations with priors on β₀ and β; the Poisson conditional mean and variance both equal λᵢ.

13:37 Affairs Data and Writing the Model in brms

The example predicts counts of extramarital affairs using the Affairs dataset available through the R package AER.
Covariates include gender, age, years married, children, religiousness, education, and marital happiness.
Sinha demonstrates fitting the model with the brms package's brm() function and emphasizes that R formula syntax is code, not a full mathematical description of the model.

19:26 Poisson Model Priors, Diagnostics, and Coefficient Interpretation

The example uses normal priors for regression coefficients and an intercept prior; brms also supplies defaults, with four chains and 2,000 iterations, including 1,000 warm-up iterations, in the displayed setup.
Categorical predictors are represented by indicator variables: female is the gender reference category, and male receives a 1.
R-hat values near 1 indicate good chain convergence; posterior credible intervals that include zero provide little evidence of a nonzero coefficient effect in this example.
Coefficient magnitudes are not directly comparable across predictors measured on different scales, such as binary gender and age in years.

24:20 Marginal Effects and Bayesian Model Comparison

To assess age, Sinha compares predicted outcomes at ages such as 30 and 40 while holding other covariates fixed, or averages those contrasts over observed covariate values.
Each posterior draw of the regression coefficients yields a draw of the marginal contrast, allowing posterior means and credible intervals to be calculated.
These model-based contrasts describe associations, not causal effects; effects can also be defined for probabilities or tail events rather than only the mean.
Bridge sampling estimates log marginal likelihood for model comparison; the Poisson model's reported value is -59.45.

30:52 Checking Poisson Fit with Zero Counts and Tail Probabilities

For a Poisson model, the probability of a zero count for observation i is exp(-λᵢ), so the expected number of zeros is the sum of these probabilities across observations.
A substantial mismatch between predicted and observed zero counts suggests the Poisson model may not capture the data adequately.
Sinha also illustrates defining age effects for tail probabilities, such as the probability of more than five affairs, and notes an observed count of 12 in the dataset.

34:14 Zero-Inflated Poisson Regression for Excess Zeros

A zero-inflated Poisson model adds an inflation parameter ω for extra zeros alongside the Poisson regression coefficients β₀ and β.
Its mean is (1 - ω)λᵢ, and its variance exceeds the mean, relaxing the Poisson model's equality of mean and variance.
The brms zero-inflated Poisson fit estimates ω at about 0.48, indicating a meaningful zero-inflation component in the example.
Its log marginal likelihood is -54.25, better than the Poisson model's -59.45 because the less-negative value is higher.

38:48 Negative Binomial Regression and Overdispersed Counts

The negative binomial model handles overdispersion with mean μᵢ and variance μᵢ(1 + ωμᵢ), which can exceed the mean.
Sinha models μᵢ = exp(β₀ + Xᵢᵀβ), again using a log link to connect the mean to the linear predictor.
The notation varies across references and software; in the parameterization shown, 1/ω is the shape parameter.
The brms negative binomial fit has log marginal likelihood -49.84, the best value among the four models compared.

43:34 Zero-Inflated Negative Binomial and Comparing Four Models

A zero-inflated negative binomial adds an extra-zero parameter to the negative binomial's regression and dispersion parameters.
The four reported log marginal likelihoods are -59.45 for Poisson, -54.25 for zero-inflated Poisson, -49.84 for negative binomial, and about -51 for zero-inflated negative binomial.
The ordinary negative binomial is preferred by this comparison because -49.84 is the highest, or least-negative, log marginal likelihood.
Sinha notes that modern software such as brms handles MCMC computation and uses advanced sampling algorithms, avoiding the need to write a Metropolis-Hastings sampler from scratch.

47:46 Posterior Predictive Checks for Count-Model Fit

The posterior predictive distribution describes a new outcome given the observed covariates and data, averaging the outcome model over the posterior distribution of parameters.
Sinha checks whether replicated data resemble observed data using summary statistics, including the proportion of zero counts.
He recommends running the provided code on the Affairs dataset or another count dataset and comparing observed statistics with their posterior predictive distributions.

Transcript excerpt

0:01 Sinha, Samiran: So today, we will talk about the last part of Chapter 10, which is MCMC methods, and then we will talk about some problems of generalized linear model. Sinha, Samiran: All right. So, MCMC techniques is a very broad technique, and in this lecture, in this class, we are getting some glimpses of these MCMC methods. Sinha, Samiran: I talked about The general metropolis strings technique. Sinha, Samiran: Metropolis H strings algorithm. Sinha, Samiran: And then we talked about Sinha, Samiran: High correlation among the observation sample values. Sinha, Samiran: And to reduce high correlations, there is no optimal or unique technique, There are different techniques.

1:01 Sinha, Samiran: And we need to try out those techniques and see which one works. Sinha, Samiran: One of them… is block sampling, we talked. And the other one we started is about co-op sampling. Sinha, Samiran: So, in the co-op sampling, in the Gibb sampling, what happens? We sample each parameter. Sinha, Samiran: From its full conditional distribution. Sinha, Samiran: And in co-op sampling, For some parameters, we will sample Prompt… Sinha, Samiran: A distribution which is marginalized over some components of the parameter. Sinha, Samiran: For instance, if you have two parameters, theta1 and theta2, we can sample Theta 1… from… Sinha, Samiran: This distribution, theta 1 given theta, And theta 2 from…

2:03 Sinha, Samiran: The conditional distribution, conditioning on theta 1 and theta. Sinha, Samiran: So, what was the advantage? Advantage is… is that… The sample values, theta1, theta2, Are no longer correlated. Sinha, Samiran: from t and t minus 1 at the iteration. Sinha, Samiran: If we do collapsing. So here we collapsed. Sinha, Samiran: Theta 2 when we sample theta 1. Sinha, Samiran: Now, this concept can be generalized from two components to multiple components of a parameter. So, suppose we have three components, theta 1, theta2, theta 3.

This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.

Related notes

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Samiran Sinha.