Fine-tuning a small model for structured extraction
Fine-tuned Llama 3.2 1B with QLoRA on a laptop CPU to pull effort, timeline and cost out of business requirements documents as validated JSON.
- Result
- ~75%
exact-match extraction after fine-tuning, up from single digits for the base model - Role
- Designed the data, training and evaluation pipeline
- Domain
- Independent project, open source
- Period
- Nov 2025
Problem
Estimating a project starts with reading a business requirements document and pulling out effort, timeline and cost. It's the same structured-extraction task the estimation platform solves with large models. I wanted to know how far a small, private, cheap model could go.
Approach
Fine-tune the smallest useful model with parameter-efficient methods on hardware anyone has, generate the training data synthetically so the labels are known, and hold the output to a schema.
What I built
- A synthetic data generator that uses Claude to write 1,000 varied requirements documents across industries, plus 200 augmented variations, with known ground-truth labels.
- Instruction-tuning data with Pydantic schemas, split 80/10/10.
- QLoRA fine-tuning of Llama 3.2 1B (8-bit quantization, LoRA rank 8, gradient accumulation and checkpointing) that trains on a laptop CPU, or in one to two hours on a free Colab GPU.
- An evaluation comparing base and fine-tuned models on valid-JSON rate, exact match, field accuracy, MAE, RMSE and R², with error analysis.
- A production inference module with validation and error handling, and a Gradio demo.
Outcome
- Valid JSON went from about 30% of outputs to about 95%, and exact-match extraction to about 75%.
- Only 0.5–2% of parameters were trained, and the adapter is tens of megabytes.
- Source: github.com/farhan1301/llm-brd-finetuning
Capabilities
Stack
- Python
- Llama 3.2 1B
- QLoRA
- Hugging Face Transformers
- PEFT
- Pydantic
- Claude API
- Gradio
Skills shown
- Python
- Agent evaluation and LLM-as-judge
From the log
- 19 Nov 2025
Fine-tuned Llama 3.2 1B for structured extraction on a laptopTests when a small, private, cheap model can replace a frontier model for one extraction task. Valid JSON rose from about 30% to 95% and exact match to about 75%, measured against the base model.