All projects

Fine-tuning a small model for structured extraction

Fine-tuned Llama 3.2 1B with QLoRA on a laptop CPU to pull effort, timeline and cost out of business requirements documents as validated JSON.

Result
~75%
exact-match extraction after fine-tuning, up from single digits for the base model
Role
Designed the data, training and evaluation pipeline
Domain
Independent project, open source
Period
Nov 2025

Problem

Estimating a project starts with reading a business requirements document and pulling out effort, timeline and cost. It's the same structured-extraction task the estimation platform solves with large models. I wanted to know how far a small, private, cheap model could go.

Approach

Fine-tune the smallest useful model with parameter-efficient methods on hardware anyone has, generate the training data synthetically so the labels are known, and hold the output to a schema.

What I built

  • A synthetic data generator that uses Claude to write 1,000 varied requirements documents across industries, plus 200 augmented variations, with known ground-truth labels.
  • Instruction-tuning data with Pydantic schemas, split 80/10/10.
  • QLoRA fine-tuning of Llama 3.2 1B (8-bit quantization, LoRA rank 8, gradient accumulation and checkpointing) that trains on a laptop CPU, or in one to two hours on a free Colab GPU.
  • An evaluation comparing base and fine-tuned models on valid-JSON rate, exact match, field accuracy, MAE, RMSE and R², with error analysis.
  • A production inference module with validation and error handling, and a Gradio demo.

Outcome

  • Valid JSON went from about 30% of outputs to about 95%, and exact-match extraction to about 75%.
  • Only 0.5–2% of parameters were trained, and the adapter is tens of megabytes.
  • Source: github.com/farhan1301/llm-brd-finetuning

Capabilities

Stack

  • Python
  • Llama 3.2 1B
  • QLoRA
  • Hugging Face Transformers
  • PEFT
  • Pydantic
  • Claude API
  • Gradio

Skills shown

  • Python
  • Agent evaluation and LLM-as-judge

From the log

  • 19 Nov 2025
    Fine-tuned Llama 3.2 1B for structured extraction on a laptopTests when a small, private, cheap model can replace a frontier model for one extraction task. Valid JSON rose from about 30% to 95% and exact match to about 75%, measured against the base model.

Everything on this site in one file. Copy it and paste it into your agent.

Loading…