IEE 475: Lecture K2 (2026-11-19): Variance Reduction Techniques, Part II (AVs & Importance Sampling)
Overview
Ted Pavlic explains how common random numbers, control variates, and antithetic variates reduce simulation-estimation variance by controlling or exploiting correlation, often tightening confidence intervals without adding replications. He demonstrates Arena seed-stream settings and paired analysis, derives a control-variate estimator, and introduces importance sampling as a way to estimate rare-event probabilities by sampling from a deliberately altered input distribution and reweighting results; its derivation is deferred to the next lecture.
Key takeaways
- Common random numbers reduce variance in a system comparison when corresponding replications share inputs and produce positively correlated outputs; a pilot experiment should verify that paired-difference variance is lower.
- Arena supports CRN through a Seeds block with Common initialization, but analysts must select the appropriate shared streams and perform paired statistical analysis separately.
- A control variate Z = Y - C(X - E[X]) remains unbiased for E[Y]; choosing C = Cov(X,Y) / Var(X) can reduce variance when X and Y are correlated.
- Antithetic variates pair Uniform(0,1) draws U with 1 - U and analyze pair averages, aiming for negatively correlated outputs that improve one-sample estimates.
- Importance sampling makes rare outcomes more frequent under a modified input distribution, then uses likelihood reweighting to recover estimates for the original distribution.
Chapters
- The remaining formal lab sessions and homework end with Arena-based Homework J2, which involves several small optimization steps and is not droppable.
- Final project presentations and four-page reports are due Wednesday of the last class week; each student reviews two reports and two presentations.
- The cumulative final uses two stages, with a 90-minute window for the first stage and an autograded second stage during finals week.
- Simulation estimates have standard errors that depend on output variability and sample count; increasing replications reduces error but costs more.
- Variance reduction instead designs experiments to control for sources of variation that are not relevant to the performance question.
- Common random numbers reduce noise in system comparisons by giving both systems the same inputs and analyzing paired differences.
- Independent replications of the greedy and nearly-full muffin policies can leave too much input noise to estimate their performance difference precisely.
- Pairing each policy's output on the same input stream makes the comparison focus on within-pair differences rather than variation in absolute output levels.
- A paired confidence interval for A minus B that excludes zero indicates a statistically detectable difference between systems.
- The example compares an M/M/1 queue with one server at one customer per minute against an M/M/2 queue with two servers each at half that rate.
- Independent replications produce overlapping estimates and a wide Welch two-sample interval that includes zero.
- Using the same interarrival streams and scaled service times creates paired comparisons; the resulting interval indicates the two-slow-server system has slightly lower queue waiting time.
- The paired estimate puts the difference at roughly 0.35 to 0.45 minutes, information a stakeholder can weigh against implementation cost.
- Separate groups testing orthopedic and running shoes may differ in activity levels, confounding shoe durability with participant demographics.
- Having each of 100 people wear both shoes controls for person-level differences, much like pairing simulation systems on common inputs.
- Randomizing which foot gets each shoe helps avoid a systematic left-versus-right-foot effect while enabling per-person wear differences.
- In Arena, add a Seeds block from the Elements panel and choose the Common initialization option to align random-number streams across replications.
- Common initialization advances seeds by 100,000 between replications, so corresponding replications across models can receive matching random inputs.
- Specify streams for inputs shared by the systems, such as interarrival times and service times, rather than forcing every random draw onto one stream.
- Export outputs to Arena Output Analyzer or another statistical tool and explicitly select paired analysis; Arena does not calculate the paired comparison automatically.
- CRN helps when both systems respond to shared input difficulty in the same direction, creating positive output correlation.
- Compare pilot-run variance of paired differences with the sum of the systems' individual variances before committing to a large experiment.
- If systems respond oppositely to inputs, their outputs may be negatively correlated and CRN can increase, rather than reduce, variance.
- For an output Y and a related variable X with known mean, define Z = Y - C(X - E[X]); Z has the same expected value as Y.
- The variance of Z depends on Var(Y), Var(X), and Cov(X,Y), so a correlated control variable can remove irrelevant variation.
- The variance-minimizing coefficient is C = Cov(X,Y) / Var(X), yielding a variance reduction proportional to the squared correlation between X and Y.
- The method is a simplified case of linear regression: a measured nuisance predictor, such as temperature, can sharpen the estimate of the effect of interest.
- For X uniformly distributed from 0 to 1 and Y = e^X, the known expected value is e - 1, approximately 1.7183.
- A MATLAB run with 1,000 samples gives a standard error near 0.01 for the raw Y mean.
- Using X as a control variate with an estimated coefficient of about 1.686 reduces the standard error to about 0.002 without generating more samples.
- Measuring relevant nuisance factors can allow experimental analysts to adjust for their effects even when those factors cannot be physically controlled.
- Antithetic variates target one-sample estimation, such as estimating a single system's mean waiting time, rather than comparing two systems.
- A short run can be misleading when several early replications fall on the same side of the true mean, even if many replications eventually average out.
- Pairing outputs with negative correlation makes high and low outcomes tend to offset, reducing the variance of pair averages.
- Simulation inputs are generated from Uniform(0,1) draws, so an antithetic replication can use 1 - U for each draw U in its paired replication.
- For example, a draw of 0.25 is paired with 0.75, and a draw of 0.33 with 0.67.
- When a model responds monotonically to its inputs, complementary draws often produce negatively correlated outputs and lower-variance pair averages.
- Pairing 500 random Uniform(0,1) draws with their 500 complements produces 500 antithetic pairs from 1,000 total draws.
- In the MATLAB demonstration, antithetic averaging sharply reduces the standard error compared with 1,000 independent draws.
- The plotted paired averages vary less and lie closer to the known mean of about 1.7183 for the e^X example.
- In Arena's Seeds block, select Antithetic initialization so even replications complement the random numbers from the preceding odd replications.
- Assign the antithetic stream to the relevant input expressions, such as an Expo(5) interarrival or service-time draw.
- Arena does not automatically create pair averages; separate odd and even outputs in Excel or use Output Analyzer's antithetic-pairs option, then analyze the pair means.
- Common random numbers create positive correlation for relative system comparisons, while antithetic variates seek negative correlation for absolute estimates.
- Directly estimating a rare event as a fraction of simulation replications may require enormous runs; 10 million standard-normal draws produced no observations above five standard deviations in the example.
- For a reactor, a flood, or an overloaded Starbucks queue, the event of interest may be too rare to observe often enough for a tight confidence interval.
- Importance sampling deliberately uses a different input distribution that makes the event more common, then reweights outcomes to estimate its probability under the original distribution.
- The lecture gives the intuition but defers the reweighting mathematics to the next class; the closing check asks whether antithetic pairs should be negatively or positively correlated.
Transcript excerpt
0:03 Okay. So, uh last lecture of the last main unit. So, the um uh otherwise next week which is going to be a short week due to the Thanksgiving holiday uh we'll have lecture L which is kind of a wrap-up lecture uh for uh the course. So, we'll review a little bit of the variance reduction techniques and then kind of pull everything to together. Uh then there'll be Thanksgiving and then we'll come back after Thanksgiving. We'll have the uh review lecture and I think I'll have this on the next slide and um and then we'll do the the exam. So I'll talk about that in a second. uh labs. Um there are no more formal lab experiences, but I still asking the TA to be available at the lab times so that there's when you're working on your final project, if you're debugging things or need help I coming
0:52 up with ideas on how to put things together, the the lab TA will be there. Of course, you can also contact me. You can come to my office hours on Mondays. You can uh you set up an appointment. Um those are all options as well but uh but no formal lab then so it also gives the TA time to grade the remaining labs that are out and uh gives you time to focus on that project. Uh no more homeworks as well. So I I I so this is uh there is a homework due this weekend that homework uh J2 that's your last homework. It's not droppable it is because we captures an AET matrix and you won't get a solution set. It is an arena based homework. So it has a a few more steps like the other homeworks have been kind
This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.
In this course
Related notes
Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Ted Pavlic.