CS 188: Natural Language Processing — Fall 2022
Prof. Nanyun (Violet) Peng

Announcements | Course Information | Schedule

Announcements

Course Information

Course objectives: Welcome! This course is designed to introduce you to some of the problems and solutions of NLP, and their relation to machine learning, statistics, linguistics, and social sciences. You need to know how to program and use common data structures.
It might also be nice—though it's not required—to have some previous familiarity with linear algebra and probabilities.
At the end you should agree (I hope!) that language is subtle and interesting, feel some ownership over some of NLP's techniques, and be able to understand research papers in the field.

Lectures: M/W 12:00 - 1:50pm
Location: 4760 Boelter Hall.
Prof: Nanyun (Violet) Peng Email: [email protected]
TAs: Zi-Yi Dou Email: [email protected]
Office hrs: Prof: Mon. 11:00am at Eng VI 397A; or zoom: link
TA: Wed. 11:00am Eng VI 389; or zoom: link
TA sessions: Friday 12:00 - 1:50pm, KAPLAN 169
Discussion site: Piazza https://piazza.com/class/l8av3ac3t0e36a
... public questions, discussion, announcements
Web page: https://vnpeng.net/cs188_win22.html
Textbook: Jurafsky & Martin, 3rd ed. (recommended)
Manning & Schütze (recommended)
Policies: Grading: homework 35%, project 15%, midterm 20%, final 25%, participation 5%
Honesty: UCLA Student Conduct Code

Schedule

Warning: The schedule below may change. Links to future lectures and assignments are just placeholders and will not be available until shortly before or after the actual lecture.


Week Monday Wednesday Friday (TA sessions) Suggested Reading
9/26 Introduction
  • Why is NLP hard? What's important?
  • Levels of language
  • NLP applications
 Project description out
Text classification and lexical semantics
  • Text classification
  • Naive Bayes classifier
  • Logistic Regression
  • Review of linear algebra and calculus
  • Intro: J&M chapter 1
  • Chomsky hierarchy: J&M 16
  • Prob/Bayes: M&S 2
10/3 Assignment 1 release
Lexical semantics
  • Semantic phenomena and representations
  • WordNet
  • Thesaurus-based semantic similarity
 Distributional semantics
  • Word-Document Matrix
  • LSA
  • Semantic Similarity
  • Word Vectors
  • Data preparation and ML practice
  • Overview of ML system components
  • Language models: J&M 3
10/10 N-gram language models
  • How to model language?
  • What's wrong with n-grams?
  • What do language models model?
 Smoothing n-grams
  • Add-one or add-λ smoothing
  • Cross-validation
  • Smoothing with backoff
  • Intro to google cloud computing
  • Intro to colab
  • Smoothing: J&M 3; Rosenfeld (2000)
10/17 Assignment 1 due
Log-linear models and neural language models
  • Log-linear models
  • Neural network basics (recap)
  • Feedforward neural language Models
 Assignment 2 release
RNN language models
  • Recurrent neural networks (RNNs)
  • Long short-term memory networks (LSTMs)
 Assignment 1 answer keys release
  • Deep learning workshop
  • Intro to PyTorch
  • Neural language models: J&M 7
  • OpenAI blog post GPT-2 (with paper)
10/24 Project midterm report due
Transformers and Masked Language Models
  • Long-short term memory networks (LSTMs)
  • The transformer model
  • Masked languge models -- BERT
 Masked Language Models (cont.)
  • Neural network basics
  • PyTorch tutorial
  • Transformer paper ; BERT paper
10/31 Midterm exam
(12:00-1:50pm in class)
Return assignment 1 gradings
 Syntax
  • Part-of-speech tagging
  • NP Chunking
  • Shallow Parsing
 Assignment 2 due
Return project feedbacks
  • More Pytorch tutorial
  • Intro to Huggingface
  • The Viterbi Algorithm: J&M 8
  • Hidden Markov Models: J&M Appendix A;
11/7 Sequence tagging models
  • POS-tagging leftovers
  • Hidden Markov Models (HMMs)
  • The Viterbi Algorithm
 Sequence tagging models (cont.)
  • The Viterbi Algorithm leftovers
  • Maximum Entropy Markov Models (MEMMs)
 Return midterm gradings
  • Deep dive of Huggingface tools
  • John Lafferty's paper on CRF
11/14 Assignment 3 release
Named Entity Recognition
  • MEMM leftovers
  • Intro to NER
  • Nested NERs
 Probabilistic parsing
  • What is parsing?
  • Why is it useful?
  • Brute-force algorithm
  • CKY algorithms
  • PCFG parsing
  • Review midterm exam
  • Attributes: J&M 12
  • Parsing: J&M 13
11/21 Dependency Parser
  • Dependency grammar
  • Dependency trees
 Dependency Parser (Cont.)
  • Shift-reduce parser
  • Final exam recitation
  • CCG: Steedman & Baldridge; more
  • TAG/TSG: Van Noord, Guo, Zhang 1/2/3
  • Prob. parsing: J&M 14
11/28 Assignment 3 due
[Guest Lecture]		 [Guest Lecture] Project final report due
  • Final exam recitation
  • MT: J&M 25, M&S 13, statmt.org; tutorial (2003), workbook (1999), introductory essay (1997), technical paper (1993); tutorial (2006) focusing on more recent developments (slides, 3-hour video part 1, part 2)