Fine-Tuning Foundation Models: LoRA & QLoRA

Fine-Tuning Foundation Models: LoRA & QLoRA

Foundation models such as LLaMA, Mistral, and Mistral-based biomedical variants arrive pre-trained on massive corpora, but they often lack the specialized vocabulary and reasoning patterns needed for pharmacy and biotech tasks — interpreting clinical trial endpoints, reasoning about drug–drug interactions, or summarizing regulatory documents. Fine-tuning adapts these models to your domain. The problem is that full fine-tuning requires updating every weight in the model, which for a 7-billion-parameter model means storing optimizer states and gradients that demand tens of gigabytes of GPU memory per training run — far beyond what a typical university lab or startup budget provides. LoRA (Low-Rank Adaptation) solves this by freezing the original model weights entirely and injecting small, trainable low-rank matrices into selected layers (usually the attention projections). Instead of updating a weight matrix W of dimension d × d , LoRA learns two much smaller matrices A ( d × r ) and B ( r × d ), where the rank r is typically 8–64. The effective update is W + BA , and because r is tiny, the number of trainable parameters drops by 99% or more. For a pharmacy use case, this means you can specialize a general-purpose LLM on drug monographs and FDA advisory committee transcripts using a single consumer GPU, and the resulting adapter files are small enough (a few megabytes) to version-control and swap per task. QLoRA (Quantized LoRA) pushes this further by quantizing the frozen base model to 4-bit precision using the NF4 (NormalFloat4) data type, while keeping the LoRA adapters in 16-bit. The quantized weights are only used for the forward pass; gradients flow through them into the adapters. QLoRA typically reduces memory requirements by roughly 4× compared to standard LoRA, making it possible to fine-tune a 13B-parameter model on a single 24 GB GPU — realistic hardware for a biotech informatics team. Empirically, QLoRA matches the quality of 16-bit LoRA for most downstream tasks, which is why it has become the default recipe for domain adaptation in regulated, resource-conscious industries like pharma. The two code examples below use the Hugging Face peft (Parameter-Efficient Fine-Tuning) and bitsandbytes libraries. The first shows standard LoRA on a biomedical question-answering dataset; the second shows the QLoRA variant with 4-bit quantization. Note how little code changes between them — this is intentional, and it means you can treat quantization as a configuration switch rather than a rewrite. # Example 1: Standard LoRA fine-tuning for a drug-interaction QA task from peft import LoraConfig, get_peft_model, TaskType from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, Trainer from datasets import load_dataset base_model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-v0.1") tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-v0.1") # LoRA configuration: only ~0.2% of parameters become trainable lora_config = LoraConfig( task_type=TaskType.CAUSAL_LM, r=16, # rank of the update matrices lora_alpha=32, # scaling factor (commonly 2x rank) lora_dropout=0.05, target_modules=["q_proj", "v_proj"], # attention layers to adapt ) model = get_peft_model(base_model, lora_config) model.print_trainable_parameters() # e.g., "trainable params: 13.6M || all params: 7.1B || trainable%: 0.19" dataset = load_dataset("pubmed_qa", "pqa_labeled") def format_example(batch): prompts = [ f"Question: {q}\nContext: {c[0]}\nAnswer:" for q, c in zip(batch["question"], batch["context"]) ] return tokenizer(prompts, truncation=True, max_length=1024, padding="max_length") tokenized = dataset.map(format_example, batched=True) training_args = TrainingArguments( output_dir="./lora-pharma-qa", per_device_train_batch_size=4, gradient_accumulation_steps=4, learning_rate=2e-4, # LoRA tolerates higher LR than full fine-tuning num_train_epochs=3, bf16=True, logging_steps=10, save_strategy="epoch", ) trainer = Trainer(model=model, args=training_args, train_dataset=tokenized["train"]) trainer.train() model.save_pretrained("./lora-pharma-qa/adapter") # only a few MB on disk # Example 2: QLoRA — same task, base model loaded in 4-bit NF4 import torch from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training from transformers import AutoModelForCausalLM, BitsAndBytesConfig # 4-bit quantization of the frozen base model bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", # NormalFloat4, optimal for weights bnb_4bit_compute_dtype=torch.bfloat16, # compute stays in bf16 bnb_4bit_use_double_quant=True, # quantize the quantization constants ) model = AutoModelForCausalLM.from_pretrained( "mistralai/Mistral-7B-v0.1", quantization_config=bnb_config, device_map="auto", # offload across available GPUs/CPU ) # K-bit models need small preparations: cast norms, enable gradient checkpointing model = prepare_model_for_kbit_training(model) lora_config = LoraConfig( task_type="CAUSAL_LM", r=16, lora_alpha=32, lora_dropout=0.05, target_modules=["q_proj", "k_proj", "v_proj", "o_proj"], # often wider for QLoRA ) model = get_peft_model(model, lora_config) model.print_trainable_parameters() # Base model now occupies ~4 GB instead of ~14 GB; fits comfortably on one GPU. # TrainingArguments and Trainer calls are identical to Example 1 from here on. # At inference time, load the base model in 4-bit and merge or attach the adapter: # peft_model = PeftModel.from_pretrained(base_model, "./lora-pharma-qa/adapter") A few practical points deserve emphasis. First, choose r and lora_alpha based on task complexity: low ranks work well for style adaptation, while complex clinical reasoning may benefit from r = 32–64 and targeting more projection layers. Second, always evaluate against a held-out set of domain-specific questions — a model fine-tuned on PubMedQA may gain biomedical fluency but silently degrade on instruction-following, so keep a general-purpose benchmark in your evaluation suite. Third, in regulated settings, document the adapter version, base model hash, and training data snapshot together; LoRA's small artifact size makes this reproducibility workflow genuinely manageable. LoRA freezes the base model and trains small low-rank matrices, cutting trainable parameters by ~99% while matching full fine-tuning quality on most domain tasks. QLoRA combines 4-bit NF4 quantization of the frozen base model with 16-bit LoRA adapters, reducing memory roughly 4× so a 7–13B model fits on a single consumer GPU. Typical starting hyperparameters: rank r = 16, lora_alpha = 32, dropout 0.05, learning rate ~2e-4. Adapters are megabyte-scale artifacts — easy to version, swap per task (e.g., one adapter for drug-interaction QA, another for protocol summarization), and merge into the base model at deployment. Always evaluate fine-tuned models on both domain-specific and general benchmarks to detect catastrophic forgetting before shipping. Practice exercise: Using the QLoRA template in Example 2, fine-tune Mistral-7B on the BioASQ yes/no question subset (or any biomedical QA dataset of at least 1,000 examples available to you). Run two experiments: one with r=8 targeting only q_proj and v_proj , and one with r=64 targeting all four attention projections. Compare the adapters on accuracy, GPU memory used during training, and adapter file size, then write a short paragraph recommending which configuration you would deploy for a hospital pharmacy chatbot and why.