# Fine-tuning a small model for structured extraction

- Period: Nov 2025
- Domain: Independent project, open source
- Role: Designed the data, training and evaluation pipeline
- Key result: ~75%, exact-match extraction after fine-tuning, up from single digits for the base model
- Capabilities: Applied ML and research; AI evaluation and quality
- Stack: Python, Llama 3.2 1B, QLoRA, Hugging Face Transformers, PEFT, Pydantic, Claude API, Gradio
- URL: https://farhanbaluch.com/work/small-model-finetuning

Fine-tuned Llama 3.2 1B with QLoRA on a laptop CPU to pull effort, timeline and cost out of business requirements documents as validated JSON.

**AI angle.** Not every extraction task needs a frontier model. A 1B model fine-tuned on synthetic data, with Pydantic validating every output, can run cheaply and privately. Knowing when that beats prompting a large model, and proving it with an evaluation against the base model, is a design decision I now make on client work.

**What this shows:**
- I can fine-tune and evaluate models myself, not only call hosted APIs.
- I generate synthetic training data with an LLM and keep the ground truth under control.
- I measure the gain against a baseline, with field-level error metrics, not just a demo.

### Problem

Estimating a project starts with reading a business requirements document and pulling out effort, timeline and cost. It's the same structured-extraction task the [estimation platform](/work/estimation-platform) solves with large models. I wanted to know how far a small, private, cheap model could go.

### Approach

Fine-tune the smallest useful model with parameter-efficient methods on hardware anyone has, generate the training data synthetically so the labels are known, and hold the output to a schema.

### What I built

- A synthetic data generator that uses Claude to write 1,000 varied requirements documents across industries, plus 200 augmented variations, with known ground-truth labels.
- Instruction-tuning data with Pydantic schemas, split 80/10/10.
- QLoRA fine-tuning of Llama 3.2 1B (8-bit quantization, LoRA rank 8, gradient accumulation and checkpointing) that trains on a laptop CPU, or in one to two hours on a free Colab GPU.
- An evaluation comparing base and fine-tuned models on valid-JSON rate, exact match, field accuracy, MAE, RMSE and R², with error analysis.
- A production inference module with validation and error handling, and a Gradio demo.

### Outcome

- Valid JSON went from about 30% of outputs to about 95%, and exact-match extraction to about 75%.
- Only 0.5–2% of parameters were trained, and the adapter is tens of megabytes.
- Source: [github.com/farhan1301/llm-brd-finetuning](https://github.com/farhan1301/llm-brd-finetuning)