LoRA fine-tuning of LLaMA and Gemma

• Fine-tune for consistent format, style or domain behavior, or to make a small model do a narrow task cheaply.
• Use RAG for knowledge that changes or must be cited.
• State the baseline you beat: the prompted base model versus the tuned model on the same eval set.

“Fine-tuning gives better accuracy” with no baseline.

• Freeze the base weights and learn a low-rank update ΔW = B·A of rank r for chosen layers.
• Alpha scales the update (effective scale alpha/r); r is typically 8 to 64.
• Target modules: attention projections (q, v) or all linear layers.
• QLoRA: LoRA on a 4-bit quantized base (NF4), so large models fit on one GPU.

Mixing up LoRA with quantization.

• Size, source, cleaning and de-duplication, and the train, validation and test split.
• A held-out domain test set, with metrics such as exact match, F1 or rubric-based LLM judging.
• A check that general ability didn’t regress.

Evaluating on data that overlaps the training set.

• Merged weights versus serving adapters dynamically (vLLM multi-LoRA).
• Where it ran (SageMaker, EKS or Vertex), quantization for serving, and the cost compared with an API model.

Not knowing how the model reached production.

• Overfitting, forgetting, chat-template mismatches or label noise. Pick the one you actually hit and explain how you fixed it.

Saying it all went smoothly.