Computing at Scale

COMPSCI 2420 Subject & Catalog Number

Description

This course focuses on efficient AI computations aimed at reducing the cost of AI model training and inference. Students will learn systematic methods for implementing parallel and distributed computations for computer vision and language models such as CNNs and Transformers across multiple computing cores or nodes. They will also learn techniques for co-designing machine learning models, data curation methods, computing algorithms, and system architectures.

Techniques to be studied include systolic arrays, low-bitwidth arithmetic, model pruning, quantization, distillation, low-rank fine-tuning, dynamic selection of submodels (e.g., experts) based on input, speculative decoding, synthetic data generation with stable diffusion, data and model security, scheduling for efficient memory access, reasoning with reinforcement learning, and test-time computing for reasoning.

As a part of programming assignments, students will utilize large language models to generate code that can leverage AI accelerating techniques learned in this course.

Upon successful completion of this course, students will be equipped to address the challenging tasks of designing and utilizing energy-efficient, high-performance AI accelerators.

Course Notes

This course was previously numbered CS 242.

Recommended Prep

Recommended prerequisites include: (1) programming experience (Python, MatLab or C/C++ should be fine); (2) basic knowledge in systems and machine organization; (3) familiarity in data structures and algorithms; and (4) maturity in mathematics (e.g., being able to make use of undergraduate linear algebra and statistics). For students with strong interest in the subject matter and related research experience, one of these four requirements may be waived. Labs and extra support will provide preparation in the first weeks of the semester to help students quickly obtain parts of the background necessary to excel in the course.