LLM Study Plan
A comprehensive 14-week curriculum covering large language models from transformers and tokenization to RLHF, RAG, agents, and deployment.
Recommended Study Path
Prerequisites
Foundations Study Plan
- Python Foundations
- NumPy & Data Wrangling
- Mathematical Foundations
Complete the Foundations study plan first →
Foundations
Weeks 1-4
- Ch 1-2: Intro & Tokenization
- Ch 3-4: Transformers & Pretraining
Core concepts: attention, embeddings, architectures
Training & Apps
Weeks 5-10
- Ch 5-6: Fine-tuning & RLHF
- Ch 7-9: Prompts, RAG & Agents
LoRA, alignment, retrieval, tool use
Advanced
Weeks 11-14
- Ch 10-11: Evaluation & Multimodal
- Ch 12-13: Safety & Deployment
Benchmarks, multimodal, safety, optimization
All Chapters
Introduction to LLMs
What are LLMs, history of language models, key architectures, scaling laws, applications, and limitations.
Tokenization & Embeddings
Byte-pair encoding, WordPiece and SentencePiece tokenizers, token vocabularies, word embeddings, and positional encodings.
Transformer Architecture
Self-attention mechanism, multi-head attention, feed-forward networks, layer normalization, residual connections, and full transformer blocks.
Pretraining
Language modeling objectives, causal vs masked language modeling, training data curation, infrastructure, and scaling strategies.
Fine-tuning
Transfer learning for LLMs, full fine-tuning, parameter-efficient methods like LoRA and QLoRA, instruction tuning, and data preparation.
RLHF & Alignment
Reward modeling, PPO for language models, Direct Preference Optimization, Constitutional AI, and preference data collection.
Prompt Engineering
Zero-shot and few-shot prompting, chain-of-thought reasoning, system prompts, and structured output generation.
Retrieval-Augmented Generation
Vector databases, embedding models, chunking strategies, RAG pipelines, and advanced retrieval techniques.
LLM Agents & Tools
Tool use patterns, ReAct framework, multi-agent systems, memory architectures, and agent frameworks.
Evaluation & Benchmarks
Perplexity, BLEU and ROUGE metrics, human evaluation, popular benchmarks, and red teaming strategies.
Multimodal LLMs
Vision-language models, audio integration, cross-modal attention, CLIP and its variants, and multimodal applications.
Safety & Ethics
Hallucination detection and mitigation, bias in language models, toxicity filtering, guardrails, and responsible AI practices.
Deployment & Optimization
Model quantization, knowledge distillation, serving infrastructure, inference optimization, and cost management.
Timed Tests
Practice Problem Sets
Sharpen your skills with coding challenges and system design problems.
Curriculum designed to take you from LLM fundamentals to advanced deployment and production optimization.