P
I
XEL
BANK
v9.1.0
Learn
Practice
Research
Explore
About
Pricing
Paper Club
Sign In
Sign Up
P
I
XEL
BANK
v9.1.0
Menu
Learn
Practice
Research
Saved Items
Explore
About
Pricing
Paper Club
Sign In
Loading...
LLM 2: Training & Alignment — all 15 problems
Fine-tuning, LoRA, reward modeling, and preference optimization.
Fine-Tuning Mechanics
LoRA Weight Decomposition
Easy
LoRA Forward Pass
Easy
Gradient Accumulation
Medium
LR Scheduler
Medium
Catastrophic Forgetting Detector
Medium
Preference Optimization
KL Divergence Penalty
Easy
DPO Loss
Medium
PPO Clipped Objective
Medium
Implicit Reward from DPO
Medium
RLHF Reward Shaping
Hard
Reward Modeling
Bradley-Terry Preference Probability
Easy
Reward Model Loss
Easy
Pairwise Ranking Accuracy
Medium
Reward Normalization
Medium
Best-of-N Sampling
Hard