Save this video free

Lecture 7 In-Memory Computing

Shimeng Yu · 36:24 · Watch on YouTube

Part of Georgia Institute of Technology ECE 6465 Memory Device Technologies and Applications. See all 20 lectures and search them

These notes were made with YouTube Collector. Paste any YouTube link and get the same: a summary, chapters and a searchable transcript.

Overview

Shimeng Yu explains how in-memory computing reduces AI data movement by performing operations where model weights are stored, then compares analog SRAM and resistive-memory designs with newer digital approaches. Analog designs can deliver high efficiency, including a 7 nm SRAM demonstration that normalizes to about 88 TOPS/W for 8-bit MACs, but limited capacity and sensitivity to process, voltage, and temperature constrain their use; digital compute-in-memory and near-data HBM are presented as more practical paths for reliable products and large language models.

Key takeaways

Chapters

0:00 AI Accelerators Still Face the Memory Wall
4:40 SRAM Versus Nonvolatile Memory for AI Workloads
11:10 Analog Compute-in-Memory Uses Array Physics for Matrix Operations
14:45 7 nm SRAM Demonstration Reaches 88 TOPS/W at 8-bit Precision
22:35 RRAM Capacity Advantages Meet Analog PVT Accuracy Limits
29:40 Charge-Domain Arrays and Digital Compute-in-Memory Trade Accuracy for Efficiency
34:45 Near-Data HBM Is the Path Toward Large Language Models

Keep these chapters and the full searchable transcript in your own library.

Transcript excerpt

0:07 So we will briefly talk about the inmemory computing. This is more related to the research that MAB has been working on for the past 10 15 years now. So the motivation very simple right? So the AI right demand the hardware and especially the data movements is a bottleneck in many of those AI workloads. So initially the focus was the the computer vision problem right you know the annexet 2012 the image class imageet

0:57 classification started the machine learning deep learning wave and then 2023 chat GPT came out and then now we have the magic language model and so on. So in terms of hardware platforms of course GPU right was a workhorse all the time and uh we do see a lot of custom silicon these days from all kinds of companies right and one of that earlier pioneer in the custom silicon was Google TPU right tensor processing units So then we still face the memory wall problem

1:47 even in the TPU. It use a sys systoical array architecture where you you know can share the weights buffers and then just have the uh partial sound you know be moved from the labor PE which is a processing element or process engine which is a digital MAC but still then we need to load in the weights from the offd run and then the activation s and then the computed partial sound result needs to be stored back into the global buffer on the chip. So still a lot of data movements ongoing there but it's kind of near memory computing because the data movements

2:35 actually got reduced because of the sharing or caching of those uh partial sum and activations. So the metric that people are interested in is the energy efficiency of the accelerator

This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.

In this course

Related notes

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Shimeng Yu.

Want the full transcript?

Save this video in YouTube Collector to get its complete searchable transcript, your own AI summaries, and a library that keeps every video you collect in one place.