Foundations of Reinforcement Learning
COMPSCI 2824
Subject & Catalog Number
Course Information
Description
Modern Artificial Intelligence (AI) systems often need the ability to make sequential decisions in an unknown, uncertain, possibly hostile environment, by actively interacting with the environment to collect relevant data. This graduate-level course focuses on the theoretical and algorithmic foundations of Reinforcement Learning (RL). The four main themes of the course are: (1) fundamentals (MDPs, computation, statistics, generalization); (2) provably efficient exploration (and high-dimensional RL); (3) policy optimization (e.g., policy gradient methods); and (4) trending topics (control, offline RL, partial observability, RL for foundation models).
Course Notes
Offered every other year.
Recommended Prep
A strong grasp of the following areas is required: Machine Learning: Equivalent to CS 1810 or higher. Probability and Statistics: Comfort with random variables, expectation, concentration inequalities, etc. Optimization: Understanding of convexity, gradient-based methods, etc. Linear Algebra: Familiarity with vector spaces, eigenvalues, and decompositions.
Available for Harvard Cross Registration
