More information about this course can be found at this website and all my code can be found at this repo.
Course Intro
Learning Goals
- Focus on foundations of Deep Learning (DL) for NLP (word vectors, RNNs, Transformers, LLMs).
- Four assignments (which can be found at my GitHub repo) and a final project.
Human Language and Intelligence
Language is the primary factor that distinguishes intelligent species like humans from other animals.
The social context of language is important—early NLP models often only capture the denotation (or dictionary definition) of words, but not their contextual meaning and relations. Nowadays, people come up with new meanings for words and their relations every day.
Word Vectors
Word vectors (also known as word embeddings or neural word representations) are a way to represent words as vectors in a high-dimensional space.
J. R. Firth
You shall know a word by the company it keeps.
The similarity between these two words is measured by the cosine similarity between their vectors.
“embedding”: we can think of these words as being embedded in a high-dimensional space.