ece1756_lecture6_part1_2026_blackwell_fpga_arch_styles

Overview

Vaughn Betz outlines the midterm and Assignment 2, then explains how 3×3 image convolution maps to streaming FPGA hardware and how Quartus can infer or instantiate Agilex DSP and memory IP. He analyzes Nvidia’s AI-focused Blackwell B200, including its two-die package, HBM bandwidth, tensor cores, precision options, and NVLink scaling, showing how memory movement, workload specialization, and compute density shape accelerator architecture.

Key takeaways

A 3×3 image convolution requires a rolling neighborhood of pixels, boundary padding, and ready/valid flow control; a correct FPGA design must account for back pressure as well as arithmetic.
Quartus can infer basic RAMs and multipliers from SystemVerilog, but exploiting advanced Intel Agilex DSP features may require explicit IP instantiation.
Blackwell’s two-die B200 uses an interposer for about 10 TB/s die-to-die communication and HBM packaging for 192 GB at about 8 TB/s, at a 1,200 W power budget.
LLM prefill can use cheaper, lower-bandwidth Blackwell CPX hardware, while token-by-token decode benefits from B200 HBM bandwidth; combining both types can specialize serving by phase.
Blackwell tensor cores devote substantially more throughput to matrix multiplication than CUDA cores, while GPCs, scratchpads, and caches help reduce the energy cost of moving data.
FP8 and FP4 increase throughput and reduce bandwidth needs, but shared scaling and model-specific precision choices are necessary to limit accuracy loss.

Chapters

0:00 Midterm Format, Scope, and Expectations

The closed-book, paper midterm is worth 15% of the final grade and is planned for about one hour.
Coverage includes lectures, required readings, and Assignment 1; optional readings are excluded except possibly for bonus questions.
Likely formats include multiple choice, fill-in-the-blank, short answers, circuit analysis, and diagrams.

4:50 Concepts to Review: Retiming, Processor Trade-offs, and Verilog

Review conceptual differences among CPUs, GPUs, and FPGAs, including how CPU area favors decisions and single-thread performance while GPUs target many parallel threads.
Be able to reason about pipelining and retiming, including whether a proposed retimed circuit is valid.
Know basic SystemVerilog and FPGA topics from Assignment 1, such as reset styles, without preparing to write extensive Verilog from memory.

9:53 Required FPGA Architecture Reading and Assignment 2

Read the first half of Vaughn Betz and a University of Waterloo coauthor’s FPGA architecture survey; it introduces architecture trade-offs and full-custom implementation.
Assignment 2 optimizes 2D convolution on an Intel Agilex FPGA, with a throughput prize and a November 5 deadline announced as roughly three weeks away.
The assignment also compares provided CPU and GPU implementations using throughput per device, per watt, and per square millimeter.

13:45 One-Dimensional Convolution as a Digital Filter

A low-pass filter removes high-frequency content; its time-domain response is a sinc function.
Digital hardware samples the continuous response into filter coefficients applied to a stream of input values.
Each output is a weighted sum of the current and previous samples, with a new output computed as each input arrives.

17:36 2D Image Convolution, Zero Padding, and Stride

A 3×3 filter multiplies nine image pixels by corresponding coefficients and sums them to produce one output pixel.
Zero padding supplies missing pixels at image boundaries so the filter can produce outputs at the edges.
Stride 1 moves the filter window one pixel at a time; stride 2 skips positions and can reduce output rows and columns by about half.

21:42 Streaming Convolution Interface and Pixel Storage

Assignment 2 uses an 8-bit grayscale pixel stream with ready/valid back pressure, following the protocol used in Lab 1.
The FPGA must buffer pixels because each output depends on a 3×3 neighborhood; padded input is supplied, but boundary cycles may not yield valid outputs.
Filter coefficients remain constant for a test case but vary between cases, so they may be stored on chip but cannot be hard-coded.
Images always have 512 columns; row count is indicated by the end of the input stream, and small test images are recommended for debugging first.

28:16 Quartus Inference and Intel Agilex DSP and Memory IP

Quartus commonly infers gates, muxes, basic RAMs, and multipliers from suitable SystemVerilog patterns.
Advanced DSP-block features such as internal multiply-accumulate paths and pipeline registers may require instantiating Native Fixed-Point DSP IP through the Quartus IP Catalog.
FIFO and two-port RAM IP can provide explicit buffering or simultaneous reads and writes; neither is required for a basic working design.
Start with inferred RTL, then consider IP blocks if reports show that the design is not using resources or performance features as intended.

33:01 Blackwell B200: Two Reticle-Limited Dies and HBM Packaging

The AI-focused Nvidia Blackwell B200 combines two large GPU dies, each near the roughly 815 mm² reticle limit, with about 208 billion transistors reported across the device.
A silicon interposer provides dense, short wiring between the dies, enabling about 10 TB/s of die-to-die bandwidth.
Eight HBM stacks connect through very wide interfaces, described as roughly 1,000 bits per stack, to deliver 192 GB and about 8 TB/s of memory bandwidth.
The package has a 1,200 W power budget and targets data centers; HBM cost and supply constraints are major trade-offs.

43:33 Blackwell CPX and LLM Prefill Versus Decode

Blackwell CPX omits HBM and instead connects to conventional DDR on a circuit board, reducing cost and memory bandwidth.
LLM prefill processes a prompt’s tokens together, reusing model weights and placing less pressure on memory bandwidth.
Decode generates one token at a time and repeatedly loads weights, making high-bandwidth HBM valuable on the B200.
Nvidia’s proposed serving setup pairs CPX devices for prefill with B200 devices for decode, transferring state between them.

47:38 Blackwell SM: CUDA Cores and Tensor Core Throughput

A Blackwell streaming multiprocessor has 128 CUDA cores, which Betz describes as functional units operating in a SIMD or SIMT-style group.
Each SM also has four tensor cores for matrix multiplication, a dominant operation in large language models.
Betz estimates each tensor core can perform 1,024 FP16 multiply-accumulates per cycle, or 4,096 across four cores, far more than the 128 CUDA-core operations in the comparison.
Tensor cores accelerate tiled portions of large matrix multiplications; instructions launch substantial work while scratchpad memory stages operands.

53:17 Blackwell Data Locality: SMs, GPCs, and Caches

Each SM’s scratchpad lets programmers reuse local data and reduce energy spent fetching from farther memory.
Blackwell groups about 20 SMs into a Graphics Processing Cluster, exposing a locality level that CUDA programs can optionally use.
Keeping communication within a GPC can reduce data movement compared with transfers between distant SMs.
Hardware-managed caches help serve reused data that programmers did not keep in scratchpads, avoiding some HBM accesses.

59:19 FP16, FP8, and FP4 Trade Precision for Throughput

Smaller numeric formats reduce compute-unit size and let fixed-width data paths and on-chip memories carry more values.
Blackwell supports FP8 and FP4 modes with shared scaling factors, including MXFP8 scaling over groups of 32 values and FP4 scaling over groups of 16.
Betz describes tensor-core throughput as roughly doubling when moving from FP16 to FP8 and doubling again for FP4, while FP8 and FP4 also reduce memory-bandwidth demands.
Lower precision is often useful for inference, but rounding error limits its suitability; training generally needs more precision, and models may mix FP4 and FP8 layers.

1:04:30 NVLink Scales Blackwell Systems to 72 GPUs

NVLink is Nvidia’s high-speed serial interconnect, analogous in role to PCI Express but proprietary.
A rack-scale system connects up to 72 GPUs through high-speed links and switches, allowing large language models to be partitioned across devices.
The large aggregate compute and memory capacity makes multi-GPU Blackwell systems suitable for serving models that do not fit on one GPU.

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Vaughn Betz.