Triton Seeded Fused Dropout

hardPROAdvanced KernelsTriton Programming

Problem Statement

Implement inverted dropout in a single kernel using Triton's counter-based RNG: drop each element with probability p, scale survivors by 1/(1-p). Use a runtime seed so the result is reproducible.

Background

tl.rand(seed, offsets) produces deterministic uniform values in [0, 1) per offset. Keep elements where rand > p. Because the RNG is seeded, two launches with the same seed must be identical, and survivors must be exactly x/(1-p).

Your Task

Implement dropout_kernel and run(n=4096, p=0.5, seed=123) that returns True when (a) two same-seed launches match and (b) surviving values equal x/(1-p).

How it is tested

Your solution must define a top-level function run(...) that allocates inputs on the GPU, launches your Triton kernel, and returns a boolean from torch.allclose(triton_out, torch_reference, ...). The grader prints run(...); the expected output is True.

Example

Input

n = 4096, p = 0.5, seed = 123

Output

True

Reasoning

The input values are n = 4096, p = 0.5, and seed = 123, which are used to allocate inputs on the GPU and launch the dropout_kernel.
The dropout_kernel uses Triton's counter-based RNG to generate uniform values in [0, 1) per offset, and drops each element with probability p = 0.5, scaling survivors by $1/(1-p) = 1/(1-0.5) = 2$ .
The kernel is launched twice with the same seed, ensuring that the results are identical due to the deterministic nature of the RNG.
The final output is True because the two same-seed launches match and the surviving values equal x/(1-p), as verified by torch.allclose(triton_out, torch_reference, ...).

Constraints

Use tl.rand(seed, offsets) for per-element randomness
keep = rand > p; out = where(keep, x/(1-p), 0)
Same seed must give identical output (verified in run)

Learn this in

Advanced Kernels & OptimizationGPU Basics with Triton

More from Triton Programming

Triton Single-Query Attention Kernelhard Triton Row Cumulative Sumhard Triton Stable LogSumExp Kernelhard Triton Autotuned Vector Addmedium Triton Matrix-Vector Product (GEMV)medium

Triton Seeded Fused Dropout

Problem Statement

Background

Your Task

How it is tested

Example

Constraints

Learn this in

More from Triton Programming

Background Knowledge

Introduction to Dropout

Understanding Triton's Counter-Based RNG

Importance of Reproducibility

Algorithm/Approach

Continue the full explanation

Editor locked