TravelHub QA workflow — AI-assisted testing

This is how we define, automate, run and continuously gate tests on TravelHub. It is the same loop for API, UI, backend and binding scenarios:

scenario card  ->  Claude generates the test  ->  run-tests.ps1 runs + logs + evidence  ->  gates CI/CD
     (you)              (AI, you review)                   (one runner, anywhere)

The scenario card is the single source of truth: a human reviews it, Claude generates the test from it, and the evidence report traces each result back to it. No card, no test.

1. Define a scenario

Copy scenario-template.yml to qa/scenarios/<ID>.yml and fill it in. Pick the layer — it decides the tool and where the test lands:

Layer	Proves	Lands in
`backend`	logic & data (tenant scoping, seat accounting)	`backend/*/_test.go`
`api`	HTTP contract: status, schema, errors	Go handler tests / Playwright request
`ui`	a user journey in the browser	`e2e/tests/*.spec.ts`
`binding`	the UI↔API / API↔DB seam (payload sent + response rendered)	`e2e/tests/*.spec.ts` (with API capture)

Worked examples to copy from: BOOK-012 (binding), API-005 (api), SRCH-003 (ui).

The card's evidence: list is a contract: the generated test must capture exactly those screenshots and API calls, and the report must show them.

2. Automate it with Claude Code

The quick way — the qa tool. It assembles a grounded generation prompt (your card + the app model + the conventions) so Claude never guesses selectors, routes, or endpoints:

npm run qa:index                       # build the app-model grounding index (do once / after UI changes)
node scripts/qa-tool.mjs list          # list scenario cards
node scripts/qa-tool.mjs prompt BOOK-012   # print the grounded prompt — paste into Claude Code
node scripts/qa-tool.mjs run    BOOK-012   # run the card's tests + open the evidence report

npm run qa:index reads the codebase and writes app-model.md / app-model.json: the routes, page objects, API endpoints, areas and accounts that actually exist. prompt embeds that plus the card and e2e/CLAUDE.md into one ready-to-paste ask, so the generated test is correct by construction. run wraps run-tests.ps1 for the card's layer and opens the evidence.

Generation stays human-in-the-loop: the tool prepares the grounded ask and runs the result; you (with Claude Code) write and review the test.

Prefer a screen? npm run qa:ui opens a small local web UI (localhost:9500) over the same commands: pick a card, copy its grounded prompt, hit Run tests, and see the evidence report inline. Same engine, nicer front door.

Or by hand: use the template in generate-test.prompt.md with your card id. Claude reads the card and the conventions in e2e/CLAUDE.md and generates the spec using our page objects and fixtures.

Then you review — the diff and the evidence:

the test uses page objects, not raw selectors;
every then has an assertion on observable state;
the evidence report shows each screenshot and API request/response the card listed.

The report is the proof of work: the AI claims it tested the scenario; the captured screenshots and API traffic prove it. Review the evidence, not just the code. Full reviewer checklist is in the prompt file.

Tip: npx playwright codegen http://localhost:4200 records a rough draft by clicking; then ask Claude to refactor it into page-object style. You never write selectors by hand.

3. Run it — one ps1, logged, CI-ready

run-tests.ps1 runs any layer, logs everything, and emits machine-readable results. The same script runs locally, in a git hook, and in a pipeline.

.\run-tests.ps1 -Layer all                         # everything
.\run-tests.ps1 -Layer unit                        # backend + web, no browser (fast)
.\run-tests.ps1 -Layer ui -Grep "oversell" -Open   # one scenario + open the report
.\run-tests.ps1 -Layer binding                      # UI<->API seam with traffic capture

What you get per run, under .testlogs/<timestamp>/:

a log per layer (backend.log, ui.log, …) — full output, tee'd live;
junit.xml — the result format Azure DevOps / Gitea publish;
summary.json — pass/fail per layer;
the evidence report (e2e/evidence-report/) — screenshots + the request, payload and response of every API call, with credentials redacted.

It also re-seeds the demo data if it has gone stale (the seeded flights are dated relative to now and expire after ~3 weeks), so you never chase a failure that is really just old data.

Exit code is 0 only if every layer passed — that is what CI gates on.

4. Run continuously, before CI and CD

Five cadences, fastest to slowest — the same run-tests.ps1 drives them all:

When	What	Enforced?
while writing	watch mode (`ng test`, `playwright --ui`)	no
pre-commit (git hook)	secret scan	local
pre-push (git hook)	`run-tests.ps1 -Layer unit`	local
pre-merge (CI)	full suite + evidence artifact	blocks merge
nightly (scheduled)	full suite	alerts on drift

Install the local hooks once after cloning:

.\scripts\install-hooks.ps1

pre-commit blocks committed secrets (it is what would have caught the dev DB password leak). Fast, no toolchain needed.
pre-push runs the unit layers so a logic break never leaves your machine.

"Before CI" is those two hooks (shift-left). "Before CD" is the pre-merge gate plus a smoke run against staging before promote. The nightly run matters: the stale-data failure above would surface at 2 a.m. instead of in front of a person.

CI / CD wiring

run-tests.ps1 is the contract; the pipeline just calls it and publishes its outputs (junit.xml + the evidence report). Azure DevOps integration is planned — when we add it, the pipeline will run this same script and use PublishTestResults@2 on junit.xml plus PublishBuildArtifacts on the evidence report. Nothing in steps 1–3 changes.

The 15-minute demo

Walk one card end to end:

Open BOOK-012 — the scenario.
Prompt Claude Code with the template → watch it generate the Playwright spec using the page objects.
.\run-tests.ps1 -Layer binding -Grep "oversell" -Open → logs stream, the evidence report opens showing the 409 and both bookings' request/response.
(Later) show the same ps1 running in the pipeline, blocking a bad merge.

That one loop is the whole system.

6.8 KiB Raw Blame History Unescape Escape