Write your first GPU kernels in Triton. You'll warm up with a masked vector-add kernel, then build a fused, numerically stable row-wise softmax that reads each row once, keeps it in registers and writes it once. You'll check it against torch.softmax on awkward shapes and extreme logits, then benchmark it with triton.testing.do_bench against torch.softmax and a naive multi-kernel PyTorch version, plotting effective GB/s. Nothing to download.
Triton Warm-up: Vector Add
The Softmax Kernel
Correctness & Stability
Benchmarking
Premium Project
This guided, GPU-powered project is available to Premium members. Upgrade to train models end-to-end in an interactive notebook.