# Market research agents for retail banking

- Period: 2025
- Domain: Enterprise bank
- Role: Designed and built the crawler and validation pipeline
- Key result: 99.3%, extraction accuracy across 20+ bank websites
- Stack: Crawl4AI, Playwright, OpenAI, OpenAI Vision, Python, Azure
- URL: https://farhanbaluch.com/work/market-research-agents

An LLM crawler reads product and rate pages across 20+ bank websites, extracts structured data, and checks its own work against screenshots before anything reaches an analyst.

### Problem

Analysts compared competitors' products and rates by visiting every bank's website by hand. A full pass took about two weeks, and the sites changed layout often enough that scrapers kept breaking.

### Approach

Let a language model read pages the way an analyst does, instead of hard-coding selectors, and never trust a single extraction: validate every value against what the page actually shows.

### What I built

- A crawler (Crawl4AI and Playwright) that navigates each bank's product pages and hands rendered content to an LLM extraction step with a fixed output schema.
- A validation pass that screenshots the source page and asks a vision model to confirm each extracted value, flagging disagreements for review.
- Normalized output for analysts, deployed on Azure.

### Outcome

- 99.3% extraction accuracy across 20+ bank websites.
- A market analysis that took two weeks now runs in under an hour.