What changes between LoRA and QLoRA in trainable weights, memory and quality?
#Show answer
LoRA keeps the pretrained weight matrix frozen and learns a low-rank update through small trainable adapter matrices. QLoRA builds on this idea by storing the frozen base model in quantized form, commonly 4-bit, while computing gradients for the adapters. Quantization reduces base-model weight residency; it does not remove the need for activations, optimizer state, temporary buffers and sufficient memory for the chosen sequence length.
The original QLoRA research demonstrated fine-tuning a 65B model on a single 48GB GPU under a particular setup, not that every 70B model fits any single GPU. Select rank, precision, batch size and context based on measured peak memory and a held-out quality set.
Trade-offs
QLoRA saves VRAM but brings quantization-related quality and throughput considerations.
Common mistake
Interpreting a research benchmark as a universal hardware guarantee.
Follow-up
Why might QLoRA still run out of memory when the quantized base weights fit?