EfficientML.ai Lecture 10 - MCUNet and TinyML (MIT 6.5940 Fall 2026)

Overview

MCUNet brings deep learning to microcontrollers by jointly designing TinyNAS architectures and the TinyEngine inference system around severe SRAM and flash limits—typically about 300 KB of SRAM and 1 MB of flash. MCUNetV2 adds patch-based inference to lower peak activation memory, while managing halo-related redundant computation; the resulting techniques support vision, keyword spotting, and anomaly detection directly on low-power devices.

Key takeaways

TinyML deployment must satisfy two separate limits: model weights must fit in flash, while the maximum live activation footprint must fit in SRAM; a small parameter count alone does not ensure a model will run.
MCUNet's co-design connects TinyNAS architecture search with TinyEngine execution so that the network is optimized for the target microcontroller and its runtime together.
Patch-based inference lowers peak activation memory by processing spatial regions through early layers, but overlapping halos are needed to preserve convolution outputs and can introduce redundant computation.
MCUNetV2 controls patch overhead by limiting receptive-field growth in early layers, then using larger spatial operations after downsampling has reduced feature-map size.
Patch-based execution makes higher input resolutions practical on constrained devices, supporting tasks such as person detection where small or distant objects are missed at low resolution.
TinyML extends beyond vision: convolutional keyword spotters use time-frequency locality, while autoencoders can detect abnormal sensor behavior from reconstruction error without labeled anomalies.

Chapters

0:00 TinyML Moves Inference from Cloud and Phones to Microcontrollers

TinyML targets low energy, low latency, low cost, and improved privacy by processing data locally rather than uploading it to cloud servers.
Microcontrollers already power billions of IoT devices and can add local intelligence to sensors, healthcare monitors, and manufacturing equipment.
Unlike smartphones with gigabytes of memory, microcontrollers may have only kilobytes or hundreds of kilobytes for neural-network execution.

5:50 MCU SRAM and Flash Limits Make Peak Activations a Deployment Barrier

A representative microcontroller budget is about 300 KB of on-device SRAM for activations and 1 MB of flash for model weights and program code.
The lecture contrasts these limits with 32 GB of memory on a Tesla V100 and gigabytes of memory on smartphones.
Convolution weights can reside in nonvolatile flash, while input and output activations occupy SRAM; peak per-layer activation usage determines whether inference fits.
MobileNetV2 can have fewer parameters than ResNet-18 yet still require more peak activation memory, so mobile optimization does not guarantee MCU compatibility.

14:10 MCUNet Co-Designs TinyNAS Architectures with TinyEngine

MCUNet jointly optimizes the neural-network architecture and the inference engine instead of improving only the model or only its runtime.
TinyNAS searches for architectures that satisfy target-device constraints; TinyEngine compiles and executes the selected network efficiently.
This system-and-algorithm co-design expands the options beyond architecture search with a fixed library or compiler optimization for a fixed model.

18:30 TinyNAS Scales Resolution and Width for Each MCU Budget

Mobile-model scaling commonly uses input resolution R and width multiplier W; a smartphone starting point is often R=224 and W=1.0.
TinyNAS chooses resolution and width for each microcontroller's SRAM and flash limits rather than reusing a mobile search space that may not fit.
Its two-stage process first selects broad scaling variables, then searches details such as kernel sizes, expansion ratios, and block counts.
A useful search space packs more FLOPs and model capacity into the same memory budget, increasing the opportunity for higher accuracy.

24:30 Weight-Sharing TinyNAS Produces Hardware-Specific Subnetworks

One-shot NAS trains a shared supernetwork and samples nested subnetworks, allowing smaller or larger models to be selected for different device budgets.
The approach can produce multiple hardware-specific networks from one shared training process.
TinyNAS distributes activation memory more uniformly across blocks than MobileNetV2, reducing the gap between average and peak layer usage.
On Visual Wake Words, MCUNet achieved better accuracy with lower peak SRAM use and latency than earlier manually designed and NAS models.

27:05 Patch-Based Inference Cuts MCUNet's Peak Activation Memory

Ordinary layer-by-layer inference computes a complete feature map, which can create an oversized activation peak in early convolution layers.
Patch-based inference processes a portion of the feature map through several early layers before moving to the next portion, reducing the live activation footprint.
The lecture illustrates roughly a fourfold activation-size reduction and explains that patch size can be selected to meet a target SRAM limit.
The method enables models that otherwise exceed a 256 KB or 320 KB microcontroller SRAM budget to run locally.

35:55 Halo Overlap and Receptive-Field Scheduling Control Patch Overhead

Patch boundaries create a halo problem: convolution kernels need neighboring elements outside a patch, so correct outputs require overlapping input regions.
The halo grows with depth; repeated 3×3 convolutions can increase redundant computation as each patch must include a wider receptive-field margin.
MCUNetV2 reduces early receptive-field growth with fewer or smaller spatial convolutions, then uses more spatial processing after downsampling has made feature maps small.
Architecture search jointly selects kernels and inference scheduling, including patch count and how many early layers use patch-based execution.

42:04 MCUNetV2 Uses Patch Execution to Improve Accuracy per SRAM

MCUNetV2 reaches about 90% accuracy with roughly 30 KB of SRAM and uses about four times less SRAM than first-generation MCUNet at a fixed accuracy.
The improvements extend beyond Visual Wake Words to ImageNet classification and PASCAL VOC object detection on tightly constrained devices.
Search selects small expansion ratios while feature maps remain large, then allows larger expansions after downsampling reduces activation memory.
Patch inference supports a 160×160 input instead of 128×128, improving resolution-sensitive tasks such as detecting distant people.

47:35 Tiny Vision Runs ImageNet Classification and Visual Wake Words on MCUs

MCUNet demonstrated ImageNet-scale classification on commercial microcontrollers and was reported as the first system in the comparison to exceed 70% top-1 accuracy.
Once-for-all training can specialize different subnetworks for microcontrollers with different SRAM and flash capacities.
Visual Wake Words detects whether a person is present, enabling an always-on visual trigger without running a costly face-recognition model.
An elevator demo uses a microcontroller to turn a red light on when a person is present and a green light on when the area is empty.

50:50 Higher-Resolution MCU Detection and On-Device Fine-Tuning

Object detection is especially sensitive to input resolution because it makes dense spatial predictions; patch-based inference allows larger inputs within the same SRAM budget.
Mask-detection and person-detection demos show colored boxes for detected people and mask status.
Higher-resolution MCUNet results recover people missed at lower resolution and reduce false detections by retaining more spatial detail.
A microcontroller demonstration fine-tunes a classifier from camera examples to distinguish bananas from oranges.

54:55 TinyML Keyword Spotting Uses Convolutional Audio Features

Tiny audio workloads include keyword spotting, speech recognition, and noise cancellation, where local always-on processing avoids continuously sending microphone data to the cloud.
The keyword-spotting pipeline splits a waveform into overlapping frames and transforms each frame into frequency-domain features.
Convolutional networks exploit local correlations across nearby time frames and frequencies, handling small speaking-style shifts better than fully connected networks at similar compute budgets.
MCUNet applies hardware-aware search to speech commands to find accuracy, latency, and model-size trade-offs for constrained devices.

57:21 Autoencoders Detect Anomalies in MCU Time-Series Data

Anomaly detection learns normal patterns and flags inputs that deviate, with applications in manufacturing inspection, health monitoring, and attack detection.
An autoencoder compresses input into a latent code and reconstructs it; training minimizes reconstruction error on normal examples.
Because training can use normal data without anomaly labels, unusually large reconstruction error can serve as an anomaly signal.
A running-fan demo turns on a warning light when a screwdriver disrupts the fan or shaking changes its behavior, then turns it off when operation returns to normal.

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, MIT HAN Lab.