QLoRA Memory Footprint Estimate
Problem Statement
QLoRA fine-tunes a 4-bit-quantized base model while training only LoRA adapters in higher precision. Estimate the bytes needed to store the base weights plus the trainable adapters.
Background
For a model with base_params frozen weights quantized to base_bits bits each, the base storage is base_params * base_bits / 8 bytes. The LoRA adapters add adapter_params trainable weights stored at adapter_bits bits each: adapter_params * adapter_bits / 8 bytes. Total is their sum (as an integer number of bytes, floored).
This is the calculation behind "fine-tune a 7B model on a single 24 GB GPU": 4-bit base + tiny bf16 adapters.
Your Task
Implement:
def qlora_bytes(base_params, adapter_params, base_bits=4, adapter_bits=16):
Return a dict with "base_bytes", "adapter_bytes", "total_bytes" (all ints, floored).
Input Format
- base_params, adapter_params (int).
- base_bits, adapter_bits (int).
Output Format
- A dict of three ints.
Sample
print(qlora_bytes(7000000000, 40000000))
Output:
{'base_bytes': 3500000000, 'adapter_bytes': 80000000, 'total_bytes': 3580000000}
Example:
print(qlora_bytes(7000000000, 40000000))
{'base_bytes': 3500000000, 'adapter_bytes': 80000000, 'total_bytes': 3580000000}- Calculate the storage for the frozen base model by multiplying the parameter count by the bit-width and converting to bytes: 7,000,000,000×4/8=3,500,000,000 bytes.
- Calculate the storage for the trainable LoRA adapters using their specific parameter count and higher precision: 40,000,000×16/8=80,000,000 bytes.
- Sum the individual byte counts to determine the total memory footprint required for fine-tuning: 3,500,000,000+80,000,000=3,580,000,000 bytes.
- The final output is
{'base_bytes': 3500000000, 'adapter_bytes': 80000000, 'total_bytes': 3580000000}
Constraints:
- All inputs are non-negative ints.
- Bytes = params * bits / 8, floored to an int.
total_bytes = base_bytes + adapter_bytes.
1. Background Knowledge
QLoRA (Quantized Low-Rank Adaptation) is a memory-efficient fine-tuning technique that decouples the storage precision of the frozen base model from the precision of the trainable parameters. The base model weights are quantized to a low bit-width (commonly 4-bit or 8-bit) and kept frozen during training, while small LoRA adapters (low-rank matrices injected into attention and feed-forward layers) are trained in higher precision (typically 16-bit or 32-bit). This separation is what allows a 7B-parameter model to be fine-tuned on a single 24 GB GPU: the bulk of the memory is consumed by the quantized base, and the adapters add only a small overhead.
The core arithmetic is a bit-to-byte conversion. If a parameter is stored at b bits, it occupies b/8 bytes. For n such parameters, the total storage is nâ‹…b/8 bytes. Because real hardware allocates whole bytes, any fractional remainder is discarded (floored). This is distinct from rounding up; the problem explicitly asks for the floor of the exact rational value.
A useful mental model: a 7B model at 4-bit needs roughly 7×109×4/8=3.5×109 bytes, i.e., about 3.5 GB. Adding 40M adapter parameters at 16-bit contributes 4×107×16/8=8×107 bytes, i.e., 80 MB. The total is dominated by the base, which is the entire point of QLoRA.
2. Algorithm Approach
This is a direct computation problem with no search, sorting, or iteration over data structures. The pattern is:
- Compute each component independently using integer arithmetic.
- Floor each component to an integer number of bytes.
- Sum the floored components to get the total.
The key subtlety is where to apply the floor. The problem states that each of base_bytes, adapter_bytes, and total_bytes is floored. The natural reading is that base_bytes and adapter_bytes are each floored individually, and total_bytes is the sum of those two already-floored integers. This is equivalent to flooring the sum only if the fractional parts of the two components sum to less than 1, but in general you should floor each component first and then add, to match the spec exactly.
3. Step-by-Step Strategy
Continue the full explanation
You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.
Editor locked
The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.