Checkpoint vs. LoRA: what is the difference?
A model checkpoint records model state at a point in time; in image-generation communities, “checkpoint” often means a loadable set of full model weights. A LoRA stores a much smaller learned update that is applied to a compatible base model. The checkpoint may work on its own in the target pipeline; the LoRA does not contain that base.
“Checkpoint” has two related meanings
During training, a checkpoint is a saved state from which work can be evaluated or resumed. It may include model weights, optimizer state, scheduler state, random-number state and training metadata. An inference checkpoint is often a slimmer export containing the weights and configuration required to load a model for use.
In Stable Diffusion communities, “checkpoint” is also shorthand for a main model file selected in a UI. That does not guarantee every dependency is embedded. A pipeline can still require a VAE, text encoder, tokenizer or configuration supplied by the application or accompanying package. Always define “complete” relative to the loader you intend to recover.
A LoRA is a learned update, not a second full model
Low-Rank Adaptation keeps the pretrained weights frozen and learns a low-rank update for selected layers. At inference time, a framework can apply that update dynamically to the base or fuse it into a derived copy. The adapter is small because it stores the update—not because it secretly compresses every base weight.
This dependency is the central checkpoint-versus-LoRA distinction. A portrait-style LoRA trained for one Stable Diffusion family cannot be assumed to work with another architecture. A language-model adapter similarly depends on the base model and target modules described by its configuration.
Practical comparison
| Question | Checkpoint | LoRA |
|---|---|---|
| What it stores | Broad model state; sometimes additional training state | Low-rank adapter weights and configuration |
| Can it run alone? | Often loadable as the main model in its intended pipeline | No; it needs a compatible base and loader |
| Typical use | Base model, full fine-tune, milestone or resume point | Efficient specialisation, style, subject or task adaptation |
| Storage | Usually much larger because it represents far more state | Usually much smaller because only selected updates are learned |
| Recovery question | Do I have the full package or training state I need? | Do I have the exact compatible base, adapter config and weights? |
Does merging a LoRA make it a checkpoint?
Fusing or merging an adapter into a base produces derived weights. A tool may save those weights in a file commonly called a checkpoint, but that output is not the original base and it does not necessarily preserve a clean, reversible separation of base and adapter. Floating-point operations, dtype conversion and export choices can all affect the result.
Keep the unfused adapter and exact base when you want to change strength, combine adapters or reproduce the merge. Keep the merged artifact as well when that exact deployment file is operationally important. Do not delete the sources merely because one merged file loaded successfully once.
What to keep during fine-tuning
For an active run, intermediate checkpoints are useful for evaluation and recovery from interruption. Once the run is complete, keep the milestones that answer a real question: the chosen final adapter, one or more defensible comparison checkpoints, configuration, training code or command, dataset reference and evaluation record. The guide to LoRA training checkpoints treats this as a reproducibility decision rather than a blanket “keep everything.”
A final adapter is not a resumable training state. Exported LoRA weights may be sufficient for inference while omitting optimizer and scheduler state required to continue training exactly.
Archive families, not filenames in isolation
A useful LoRA family includes enough context to identify and obtain the compatible base. When the base is rare, private or locally modified, retain it alongside the adapter. A checkpoint family may include companion components that the runtime supplied silently during the first load.
Tensor Archive helps retain and verify selected local tensors as a related family. It does not guess compatibility from a filename, reconstruct missing training state or turn a LoRA into a standalone base model.
Continue with the artifact you have
If the adapter extension is the confusing part, read what a LoRA file contains. If you are choosing a training method, use LoRA vs. QLoRA. If storage pressure is driving the question, see how to free disk space without changing LoRA weights.
Sources
- LoRA: Low-Rank Adaptation of Large Language Models — the original method and frozen-base/low-rank-update design.
- Hugging Face PEFT: LoRA — current adapter configuration and target-module implementation.
- Hugging Face PEFT checkpoint format — adapter weights, configuration and base-model dependency.
- Diffusers: load adapters — loading, unloading and fusing LoRA adapters with diffusion pipelines.