Custom Copilot agents for test-automation teams
Custom agents and skills that generate UI, API and contract tests in each team's own conventions, rolled out with hands-on sessions and a searchable catalog of 27 agents.
- Result
- 27
agents and skills catalogued across seven SDLC phases - Role
- Wrote the agents and skills, ran the enablement, published the catalog
- Domain
- Enterprise test-automation teams (Quality engineering, Developer platforms and tooling)
- Period
- Mar 2026 – Sep 2026
Problem
Test-automation teams had access to AI assistants, but the tests they produced ignored each team's frameworks, page objects and naming conventions. Engineers rewrote most of the output, and adoption stayed shallow.
Approach
Put each team's conventions into the agent, not into every prompt. Start with the highest-volume work (UI test generation), move down the test pyramid, and make the agents easy to find so teams reuse them instead of writing their own.
What I built
- Custom agents and skills that generate Playwright tests in each team's own conventions, with hands-on sessions so teams actually used them.
- Agent workflows for Pact contract tests and API suites, so service changes are caught before end-to-end runs.
- A rollout plan for code-review skills, rewritten after finding that skills only load from a repository's default branch.
- A searchable catalog of 27 agents and skills, each mapped to one of seven SDLC phases, published in a docs portal.
Outcome
- Engineers look for an existing agent before writing a new one, and the programme can see which SDLC phases still have no agent support.
- These agents became the packages that the distribution platform versions and installs.
Capabilities
Stack
- GitHub Copilot agents
- Agent skills
- Playwright
- Pact
- Docs portal
Skills shown
- Copilot custom agents and skills
- Playwright
- Risk-based test strategy
- Developer experience tooling
- AI governance and standards
From the log
- 22 Sep 2026
Launched a searchable catalog of 27 agentsTurns a pile of agents into a toolkit. Reuse rises, duplicates fall, and the programme can see which SDLC phases still have no agent support. - 13 Aug 2026
Changed the rollout plan for code-review skillsKnowing how the agent platform actually loads skills is the difference between a rollout that works and teams deciding AI review "doesn't do anything". - 20 Apr 2026
Rolled out custom Copilot agents to test-automation teamsGeneric assistants write generic tests. Encoding each team's frameworks and conventions into agents is what made the output mergeable, and the sessions are what made people use it. - 23 Mar 2026
Generated contract and API tests with agentsMoves AI test generation down the test pyramid, where tests are faster and cheaper, instead of only producing more slow UI tests.