GuidesUnderstanding AI

LoRA: adapting an AI model without retraining all of it

By the Banzai Labs teamUpdated on 3 min read

Frozen model+A×BA few million weightsOnly two small matrices added to the model are trained
The model stays frozen; only two small matrices added to its layers are trained.

The principle

  1. 1Freeze the modelIts billions of weights do not move.
  2. 2Add two small matricesNext to some of the layers.
  3. 3Train only thoseA fraction of the parameters.

Full or LoRA

Two ways to adapt a model
Full retraining
  • Every weight changed
  • Lots of memory and compute
  • A full copy of the model per version
LoRA
  • A small part trained
  • Possible on a single graphics card
  • One small file per adaptation

Good uses

Case Why LoRA
A brand’s tone and style Behavior changes, not knowledge
A very precise output format Fewer instructions to repeat
Classification into your categories A narrow, well-defined task
Several clients, one model One adapter loaded per client

Before you start

You need quality examples, often a few hundred to a few thousand, and a test set to check the gain. A good prompt with examples is often enough: try that first.

The choice: RAG or fine-tuning. The models: open source model or API and quantization.

Frequently asked questions

Does LoRA teach the model new knowledge?
Not much. It mainly adapts a style, a format or a specific task. For knowledge that changes (catalog, documentation), RAG remains the better fit.
What is QLoRA?
LoRA applied to a model loaded in quantized form (4 bits). It lets you adapt a fairly large model on a single graphics card.
Can LoRA be used on any model?
On open-weight models, yes. For models available only through an API, the provider decides whether it offers adaptation, and in what form.

Read next

All guides