Save this video free

Lecture 18: Linear neural network (cont); Fully connected neural network

Burton Ma · 49:29 · Watch on YouTube

Part of Queen's University CISC 371. See all 17 lectures and search them

These notes were made with YouTube Collector. Paste any YouTube link and get the same: a summary, chapters and a searchable transcript.

Overview

Burton Ma derives gradients for a layered linear network using the chain rule, showing why parameter gradients can be computed from the output layer backward. He then introduces fully connected networks with vector and matrix notation, and derives back-propagation equation BP1, including the output-layer error for least-squares loss; BP2 and BP4 are left for the next lecture.

Key takeaways

Chapters

0:00 Chain Rule for the Last Neuron's Weight Gradient
7:00 Bias Gradients and Propagating Activation Derivatives
13:00 Assembling Layer Gradients and Working Backward
17:00 Back-Propagation's Purpose and Historical Rediscovery
21:00 Fully Connected Networks and Layer Notation
29:00 Weight Vectors, Bias Units, and Layer Matrices
34:30 Vectorized Forward Pass and Hadamard Products
38:00 BP1 Defines the Output-Layer Error
43:50 Least-Squares Loss Gives the Output Error Vector

Keep these chapters and the full searchable transcript in your own library.

Transcript excerpt

0:02 All right. Uh I guess we'll get started. So um just to remind you what we're doing, um we're looking at this simple linear network of neurons. Um and eventually we want to train this thing, but to train this thing we need to compute its gradient. So that's what we're trying to do here. Um at the end of the last lecture we got to uh these sets of slides. So to compute the gradient, um one way that you can do it is you can uh visualize what's happening with a picture like this. Um and so this is showing you the neuron in the last layer. Right? So here's uh the input coming from the second last layer flowing into the neuron. Neuron calculates its uh its transfer function followed by its activation function.

0:50 Um what we want is the gradient of the overall network, but if we look at the gradient of just this one neuron, what we want what we're interested in is the quantity uh if these quantities change, how does the cost change at the end? So we're looking for dW dC and dB by dC. And so the basic idea is is if you tweak W, that causes a change in Z1, which causes a change in A1, which causes a change in C. Uh and so Right? So to get the uh so the the change in W causes a change in C. So that's just your derivative dC by dW. Uh and then you just follow the path of information as it flows through the neuron. Right? So changing W causes Z to change. So that's just dZ by dW. Changing Z causes A to change. So that's

1:38 just dA by dZ. Uh and then changing A causes C to

This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.

In this course

Related notes

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Burton Ma.

Want the full transcript?

Save this video in YouTube Collector to get its complete searchable transcript, your own AI summaries, and a library that keeps every video you collect in one place.