STAT 638 (Fall 2026), Lecture 34
Overview
Samiran Sinha develops a Gaussian process model for river-basin zinc concentrations, using log-transformed observations, a squared-exponential spatial covariance kernel, and independent measurement noise. He explains how generalized least squares and marginal-likelihood optimization estimate the model, demonstrates fitting and validation in R, discusses prediction uncertainty, and previews forecasting DAX stock levels and Kalman filtering.
Key takeaways
- A spatial Gaussian process for log zinc concentration separates a baseline β₀, a latent spatial surface F(Sᵢ), and independent measurement noise, with covariance V = K + σ²εI.
- The squared-exponential kernel σ²F exp(−distance²/(2η²)) uses σ²F to control spatial variation and η to control how quickly spatial correlation decays with distance.
- The baseline β₀ has a generalized least-squares estimate, while σ²ε, σ²F, and η can be fitted by optimizing the marginal likelihood after integrating out the latent Gaussian process.
- A fitted map and training-data RMSE do not establish out-of-sample accuracy; holding out about 30% of observations provides a direct test of prediction performance.
- Gaussian process predictions can include uncertainty: with predictive mean μ₀ and standard deviation σ₀, a 95% interval is approximately μ₀ ± 1.96σ₀.
- For the DAX example, forecasting future closing levels requires both predicted values and uncertainty; a Kalman filter offers an online alternative by updating forecasts with each new observation.
Chapters
- Upload one reproducible project file, preferably a ZIP, to Sinha’s shared TAMU folder rather than large data files.
- For external datasets, provide specific access instructions or code; small datasets can be linked directly online.
- Groups present over Zoom for 8 to 10 minutes, followed by individual questions; each student must attend their group’s presentation.
- Slides should cover the problem, goals, general methods, results, and any methodological, data, or scientific novelty.
- Sinha introduces Gaussian processes as a useful modeling tool after completing chapters 1 through 10.
- Applications include spatial modeling and, with modifications, stock-price modeling.
- The preceding lecture covered Gaussian process definitions, Brownian motion, and Brownian bridges.
- The example uses zinc concentration observations, Zᵢ, at river-basin sampling locations Sᵢ, represented by longitude and latitude.
- The response is log concentration, Yᵢ = log(Zᵢ), which Sinha describes as more stable than the original measurements.
- The model adds a location-dependent, nonparametric function F(Sᵢ) to baseline β₀ and independent white noise εᵢ.
- F is assigned a zero-mean Gaussian process prior, while εᵢ is normally distributed with variance σ²ε and independent of F.
- The spatial covariance kernel is σ²F exp(−distance²/(2η²)), where σ²F controls spatial variation and η controls the correlation length.
- Integrating out the Gaussian process gives a multivariate normal distribution for Y with mean β₀ and covariance V = K + σ²εI.
- Because the design matrix for the baseline is a column of ones, the generalized least-squares estimate of β₀ is (1ᵀV⁻¹Y)/(1ᵀV⁻¹1).
- The same generalized least-squares estimate is the maximum-likelihood estimate of β₀ when the covariance parameters are fixed.
- The covariance matrix depends on unknown σ²ε, σ²F, and η, so the GLS estimate of β₀ also depends on these parameters.
- Sinha substitutes the GLS estimate of β₀ into the multivariate normal likelihood to form a marginal likelihood for the covariance parameters.
- Maximizing the likelihood estimates the three covariance parameters; R and Python optimization routines can perform this numerical search.
- Cholesky decomposition supports likelihood calculations without explicitly computing V⁻¹.
- The Gaussian process is Bayesian because F receives a prior distribution that represents uncertainty about the unknown spatial surface.
- Sinha integrates out F and estimates β₀ and the covariance parameters through the marginal likelihood.
- A full Bayesian analysis could assign explicit priors to β₀, σ²ε, σ²F, and η; the example instead uses priors proportional to 1.
- He characterizes the procedure as close to empirical Bayes, rather than a full Bayesian analysis, because parameters are estimated by marginal-likelihood optimization.
- The R example prepares log zinc values, sampling locations, a vector of ones for the design matrix, and pairwise distances for the covariance matrix.
- The objective is the negative marginal log likelihood, which the R optimization routine minimizes over σ²ε, σ²F, and η.
- The example uses BFGS and notes that sensible initial values matter because the objective is nonlinear.
- The likelihood calculation uses the Cholesky factor to obtain the log determinant and quadratic residual term; the constant 0.5n log(2π) can be omitted during optimization.
- After estimating β₀, σ²F, η, and σ²ε, Sinha calculates fitted spatial effects and predicted log zinc values.
- A fine-grid map displays predicted concentrations, with darker colors for lower values and lighter colors for higher values; observed locations are overlaid as circles.
- An observed-versus-fitted plot checks whether points lie near the 45-degree line, providing a visual assessment of fit on the training data.
- A stronger validation can hold out about 30% of observations, predict them without training on them, and compare predictions with observed values; an RMSE of 0.3010 is meaningful mainly against competing models.
- The displayed spatial surface shows predicted means but omits uncertainty; the predictive variance σ²₀ can also be calculated using fitted covariance parameters.
- For a Gaussian predictive distribution, a 95% prediction interval is approximately μ₀ ± 1.96σ₀.
- For a Gaussian distribution, mean and median predictions coincide; Sinha notes that medians can be useful for non-Gaussian alternatives.
- Smaller η means spatial correlation decays more slowly with distance; optimizing log η keeps the search parameter unrestricted, while explicit bounds are also possible.
- Sinha introduces historical DAX closing levels, a German stock-market index, as a financial forecasting example.
- The dataset contains about 1,600 trading-day observations, with each time step running from one recorded close to the next regardless of weekends or holidays.
- The forecasting goal is to predict future closing levels, such as 1, 3, 5, or 20 trading days ahead, while quantifying forecast uncertainty.
- The next lecture will compare fitting from all 1,600 observations with a Kalman filter that updates forecasts as each new daily price arrives.
Transcript excerpt
0:06 Sinha, Samiran: In today's class, we'll discuss… Sinha, Samiran: Last night, emails. Sinha, Samiran: and the Gaussian process. Last night, I sent an email Sinha, Samiran: To you all about some predictions, where to keep your projects, So, please put those projects Sinha, Samiran: in a folder, I gave you the folder link, and you'll be able to access that folder using your TAMU credentials. Sinha, Samiran: And, so the, the point is… Do not overload the folder.
1:06 Sinha, Samiran: Okay? So, upload zip file, that is the reason I said, upload one file, preferably, most preferably, one file, which, anyone can use it for the reproducibility purpose. Sinha, Samiran: And, if you have a very big data, megabytes after megabytes data, do not upload it. Rather, you can give very specific instructions Sinha, Samiran: on how to access that data. Very specific instruction, but do not… do not upload it. Sinha, Samiran: The reason is, you have seen already, yesterday, Everything crashed, right? Sinha, Samiran: From Microsoft, and until today morning, I was not able to, see my email. Sinha, Samiran: So, if you put so many things in my folder, this is… will be within my directory, keep in mind, this is my personal directory, and if my directory is overloaded with your
2:07 Sinha, Samiran: files, data files, then everything, once everything crashes, then I'll not be able to come to the class, teach anything. Everything will be blocked. So, so please be considerate. Do not upload a huge data pile. No data pile, essentially, needed. Sinha, Samiran: You can give very specific instructions. Sinha, Samiran: If you have external data, how to access that external data with specific code. That would be enough. Sinha, Samiran: And for a very small dataset. Sinha, Samiran: What you can do, you can put the direct link of the internet so that your code can directly access that data set.
This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.
In this course
Related notes
Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Samiran Sinha.