Save this video free

UUtah CS 6340 NLP | Fall 2026 | Word embeddings (Part 1)

UofU Data Science · 1:16:13 · Watch on YouTube

Part of University of Utah CS 6340. See all 19 lectures and search them

These notes were made with YouTube Collector. Paste any YouTube link and get the same: a summary, chapters and a searchable transcript.

Overview

Word2Vec’s Skip-gram with Negative Sampling (SGNS) learns dense word vectors from corpus co-occurrence, using the distributional hypothesis that words appearing in similar contexts tend to have related meanings. The lecture builds the method from positive and sampled negative word pairs through its sigmoid-dot-product objective and gradient descent, then explains embedding lookup, practical tools, and the limitation that one static vector cannot represent a word’s different senses.

Key takeaways

Chapters

0:00 Practicum Logistics and the Linguistic Limits of Bag-of-Words
0:48 Vector Semantics Encodes Similarity and Relatedness
6:40 Dense 300-Dimensional Embeddings Versus Sparse Word Counts
10:55 The Distributional Hypothesis and the Artemia Example
13:27 Representation Learning, Embeddings, and Corpus Distributions
19:36 Word2Vec and the Skip-Gram Classification Idea
24:04 Context Windows, Dot Products, and the Sigmoid Probability
33:00 Separate Target and Context Embeddings
35:31 Negative Sampling Creates Diverse Training Pairs
42:06 Training Embeddings with a Positive and Negative Objective
47:25 Deriving the SGNS Negative Log-Likelihood Loss
57:56 Gradient Descent Updates the Word Vectors
1:02:33 Static Embeddings Collapse Polysemous Word Senses
1:08:11 FastText, GloVe, and Gensim for Practical Embeddings
1:11:00 Embedding Matrices, Token Lookup, and Dimension Choices

Keep these chapters and the full searchable transcript in your own library.

Transcript excerpt

0:01 So yeah, practicum is happening next Wednesday in a week. Um, you have a practice assignment to prepare for it. I will share just one message kind of collecting all the thoughts I mentioned around uh practicum today or tomorrow. Um, yeah, there's nothing else for you to do than do the practice assignment. Ask us questions uh if there are any along the way. Uh, next Monday is a Labor Day, so we won't have a class. So we meet uh next Wednesday uh when the practicum uh is today we have a lot to do a little bit uh ambitious plan on my end so I will jump right into it unless there are some pressing questions. Okay let's let's get into it then. Um

0:51 remember last time we were chatting about how uh we have a piece of text. It can be for example a prompt we give to an LLM. That piece of text is then tokenized namely split into a list of tokens in the vocabulary. And then we talked about how now we need some kind of numerical representation that can be fed into a neural network. So neural network can process it. And these numerical representations ideally would already carry information about um you know useful information about meaning of these words and their relation to other words. So just to go over some of those linguistic aspects we were kind of mentioning in Russia at the at the end of the uh last lecture those are word senses. Um here is um just a screenshot

1:40 of the uh victionary item for the word basin. Um one term I want you to know is lema. Lemma refers to the canonical or dictionary form of a word. And we have many uh word forms. These are you know

This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.

In this course

Related notes

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, UofU Data Science.

Want the full transcript?

Save this video in YouTube Collector to get its complete searchable transcript, your own AI summaries, and a library that keeps every video you collect in one place.