QLoRA Memory Footprint Estimate

mediumPROEfficient Adaptation & Hallucination MetricsVision-Language ModelsVLM 1: Vision-Language Models

Problem Statement

QLoRA fine-tunes a 4-bit-quantized base model while training only LoRA adapters in higher precision. Estimate the bytes needed to store the base weights plus the trainable adapters.

Background

For a model with base_params frozen weights quantized to base_bits bits each, the base storage is base_params * base_bits / 8 bytes. The LoRA adapters add adapter_params trainable weights stored at adapter_bits bits each: adapter_params * adapter_bits / 8 bytes. Total is their sum (as an integer number of bytes, floored).

This is the calculation behind "fine-tune a 7B model on a single 24 GB GPU": 4-bit base + tiny bf16 adapters.

Your Task

Implement:

def qlora_bytes(base_params, adapter_params, base_bits=4, adapter_bits=16):

Return a dict with "base_bytes", "adapter_bytes", "total_bytes" (all ints, floored).

Input Format

base_params, adapter_params (int).
base_bits, adapter_bits (int).

Output Format

A dict of three ints.

Sample

print(qlora_bytes(7000000000, 40000000))

Output:

{'base_bytes': 3500000000, 'adapter_bytes': 80000000, 'total_bytes': 3580000000}

Example

Input

print(qlora_bytes(7000000000, 40000000))

Output

{'base_bytes': 3500000000, 'adapter_bytes': 80000000, 'total_bytes': 3580000000}

Reasoning

Calculate the storage for the frozen base model by multiplying the parameter count by the bit-width and converting to bytes: $7,000,000,000 \times 4 / 8 = 3,500,000,000$ bytes.
Calculate the storage for the trainable LoRA adapters using their specific parameter count and higher precision: $40,000,000 \times 16 / 8 = 80,000,000$ bytes.
Sum the individual byte counts to determine the total memory footprint required for fine-tuning: $3,500,000,000 + 80,000,000 = 3,580,000,000$ bytes.
The final output is {'base_bytes': 3500000000, 'adapter_bytes': 80000000, 'total_bytes': 3580000000}

Constraints

All inputs are non-negative ints.
Bytes = params * bits / 8, floored to an int.
total_bytes = base_bytes + adapter_bytes.

Learn this in

Deployment & OptimizationLLM Study Plan Fine-tuningLLM Study Plan Parameter-Efficient Fine-TuningVLM Study Plan

More from VLM 1: Vision-Language Models

Visual Token Pruning by Attention Masshard CHAIR-i and CHAIR-s Hallucination Ratesmedium Merge a LoRA Update into the Base Weightmedium POPE Precision, Recall, and F1medium POPE Object Hallucination Accuracyeasy

Concept primerPREVIEW

1. Background Knowledge

QLoRA (Quantized Low-Rank Adaptation) is a memory-efficient fine-tuning technique that decouples the storage precision of the frozen base model from the precision of the trainable parameters. The base model weights are quantized to a low bit-width (commonly 4-bit or 8-bit) and kept frozen during training, while small LoRA adapters (low-rank matrices injected into attention and feed-forward layers) are trained in higher precision (typically 16-bit or 32-bit). This separation is what allows a 7B-parameter model to be fine-tuned on a single 24 GB GPU: the bulk of the memory is consumed by the quantized base, and the adapters add only a small overhead.

The core arithmetic is a bit-to-byte conversion. If a parameter is stored at $b$ bits, it occupies $b / 8$ bytes. For $n$ such parameters, the total storage is $n \cdot b / 8$ bytes. Because real hardware allocates whole bytes, any fractional remainder is discarded (floored). This is distinct from rounding up; the problem explicitly asks for the floor of the exact rational value.

A useful mental model: a 7B model at 4-bit needs roughly $7 \times 10^9 \times 4 / 8 = 3.5 \times 10^9$ bytes, i.e., about 3.5 GB. Adding 40M adapter parameters at 16-bit contributes $4 \times 10^7 \times 16 / 8 = 8 \times 10^7$ bytes, i.e., 80 MB. The total is dominated by the base, which is the entire point of QLoRA.

2. Algorithm Approach

This is a direct computation problem with no search, sorting, or iteration over data structures. The pattern is:

Compute each component independently using integer arithmetic.
Floor each component to an integer number of bytes.
Sum the floored components to get the total.

The key subtlety is where to apply the floor. The problem states that each of base_bytes, adapter_bytes, and total_bytes is floored. The natural reading is that base_bytes and adapter_bytes are each floored individually, and total_bytes is the sum of those two already-floored integers. This is equivalent to flooring the sum only if the fractional parts of the two components sum to less than 1, but in general you should floor each component first and then add, to match the spec exactly.

3. Step-by-Step Strategy

PRO

Continue the full explanation

You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.

PRO

Editor locked

The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.

solution.py

Python 3.13

Test results