CMU Introduction To Deep Learning 11-785, Fall 2026: Lecture 1

Overview

CMU's Introduction to Deep Learning opens with course expectations and then traces neural networks from Alexander Bain's 1873 connectionist model through McCulloch–Pitts neurons, Donald Hebb's learning rule, and Frank Rosenblatt's perceptron. It builds to the central result that multilayer perceptrons can represent arbitrary Boolean functions and approximate continuous functions, making neural networks flexible function models for tasks such as classification, speech recognition, translation, and image captioning.

Key takeaways

A connectionist model stores knowledge in its network of connections: the lecture cites an estimated 80 billion neurons and 100 trillion connections in the human brain.
McCulloch–Pitts threshold neurons can implement Boolean gates, but their model does not provide a mechanism for learning the connections.
Hebb's rule strengthens a connection when its two neurons fire together, but an increase-only update can cause weights to saturate.
Rosenblatt's perceptron uses supervised error-driven weight updates and can learn linearly separable classes, but a single unit cannot represent XOR.
Multilayer perceptrons combine linear threshold boundaries to represent arbitrary Boolean functions and classification regions, including complex shapes.
With suitable activations and enough units, neural networks can approximate continuous functions as well as perform classification.

Chapters

0:00 11-785 Course Setup and Where to Find Materials

The Fall 2026 course has roughly 400 students and 25 teaching assistants.
Course materials, policies, quizzes, and recorded Recitation Zero tutorials are hosted at deeplearning.cs.cmu.edu.
Students are asked to review the logistics lecture and use Piazza, Canvas, Gradescope, and AutoLab.

3:00 Study Groups, TA Mentors, and Student Well-Being

The course encourages students to form study groups early; students who need help finding a group can be matched with one.
Study groups receive TA mentors, and the course offers dozens of office hours each week.
Students facing stress or other difficulties are encouraged to contact their mentor, Brad Warren, or the instructor.

6:30 Course Goals, Participation Rules, and Lecture Style

The course aims to teach neural-network theory, enable students to build network components, and prepare them to work on large-scale problems.
Homework Part Ones build an independent deep-learning toolkit; Part Twos apply techniques to state-of-the-art problems.
Attendance is generally mandatory and checked through Zoom or Piazza polls; personal devices should be limited to course tasks.

14:00 TPU Access, JAX Support, and Course Accommodations

Google provides substantial Google Cloud TPU access, but TPU workflows require JAX rather than the more commonly used PyTorch.
Homework is primarily designed for PyTorch; JAX support is available with fewer TA resources.
The instructor explains that physical limitations may affect classroom movement and asks students not to make derisive comments.

17:00 Lecture Roadmap and the Deep-Learning Breakthrough

Lecture 1 covers neural-network history, connectionism, early models, their limitations, and what modern networks can compute.
The slide deck contains 116 slides; about 90 are presented live, while additional slides may appear in weekly quizzes.
The lecture frames the mid-2010s as a turning point when neural networks began overtaking previous methods on major AI tasks.

22:00 Speech, Translation, Go, and Image Captioning

The lecture cites Microsoft's 2016 speech-recognition result as a milestone in outperforming people on a standard benchmark.
Google Translate's 2016 transition to neural translation models is presented as a shift from unreliable translations to practical professional assistance.
AlphaGo defeated the Go world champion in 2016; the lecture also highlights neural image captioning and later image analysis by ChatGPT.

27:30 Associationism: Learning by Connecting Experiences

The lecture traces associationism from Plato in 400 BC through David Hume and Ivan Pavlov.
An association links experiences, such as lightning with subsequent thunder, enabling prediction from one observation to another.
Machine-learning functions are described as storing associations between inputs and outputs.

31:40 Alexander Bain's 1873 Connectionist Network

In Mind and Body, Alexander Bain proposed that information is stored in connections between neurons.
Bain sketched networks of binary inputs and outputs in which different input combinations produce different output patterns.
His proposed learning mechanism strengthened pathways when impressions occurred together or in close succession.

37:20 Connectionist Brains Versus von Neumann Computers

The lecture contrasts the brain's estimated 80 billion neurons and 100 trillion connections with conventional computer architecture.
In a connectionist machine, knowledge and computation are encoded in the pattern of connections between simple processing units.
In a von Neumann architecture, processor and memory are separate, so changing a stored program changes what the processor does.

42:00 McCulloch–Pitts Neurons and Boolean Logic

Warren McCulloch and Walter Pitts introduced a mathematical neuron model in their 1943 paper on logical operations.
Their threshold units combine excitatory inputs, while inhibitory inputs can prevent a neuron from firing.
Networks of these units can implement Boolean gates such as AND, OR, and NOT, as well as simple timing and sensory circuits.

49:00 Heat and Cold Circuits Reveal a Learning Gap

McCulloch–Pitts circuits illustrate how brief cold contact can trigger a heat response while sustained contact activates a cold response.
The model offered a way to compose logic circuits but did not explain how a network learns its connections.
Donald Hebb addressed learning in The Organization of Behavior, published in 1949.

53:20 Hebbian Learning: Neurons That Fire Together Wire Together

Donald Hebb proposed that repeated activity from neuron A that helps fire neuron B strengthens their connection.
A simplified update is to increase weight wxy when both neurons x and y fire.
Because this rule only increases weights, connections can grow without bound and eventually saturate; later learning methods add ways to control or reduce weights.

59:00 Frank Rosenblatt's Perceptron and Supervised Updates

Frank Rosenblatt's perceptron models sensory units, association units, and response units, with selected connection weights learned from examples.
Each unit computes a weighted input sum and fires when that sum reaches its threshold.
Rosenblatt's supervised update changes weights using the error between desired and actual output, and converges for linearly separable classes.

1:03:00 Multilayer Perceptrons Overcome the XOR Limitation

A single perceptron can implement simple Boolean functions but cannot compute XOR.
Combining perceptrons in a multilayer perceptron (MLP) allows hidden units to build intermediate functions that an output unit combines.
The lecture shows that MLPs can compose arbitrary Boolean functions, with intermediate units called hidden-layer neurons.

1:08:00 Affine Neurons as Linear Classifiers

A real-valued perceptron computes a weighted sum plus a bias, then passes the result through an activation function such as a threshold.
The bias makes the computation affine rather than strictly linear and allows the decision hyperplane to move away from the origin.
A single threshold perceptron separates inputs on opposite sides of a hyperplane, so it acts as a linear classifier.

1:14:00 MLPs Approximate Functions and Model More Than Classes

Networks can combine half-space tests to represent polygonal regions and approximate arbitrary classification boundaries, including regions in high-dimensional image spaces such as MNIST's 784 dimensions.
Pairs of threshold units can create localized intervals; combining many such units gives piecewise-linear approximations to continuous functions.
The lecture closes by describing neural networks as layer-wise functions that can classify, approximate continuous mappings, store patterns in recurrent connections, and represent or generate probability distributions.

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Carnegie Mellon University Deep Learning.