TS_Proto/qa/README.md

TravelHub QA workflow — AI-assisted testing

This is how we define, automate, run and continuously gate tests on TravelHub. It is the same loop for API, UI, backend and binding scenarios:

scenario card  ->  Claude generates the test  ->  run-tests.ps1 runs + logs + evidence  ->  gates CI/CD
     (you)              (AI, you review)                   (one runner, anywhere)

The scenario card is the single source of truth: a human reviews it, Claude generates the test from it, and the evidence report traces each result back to it. No card, no test.

---

1. Define a scenario

Copy scenario-template.yml to qa/scenarios/<ID>.yml and fill it in. Pick the layer — it decides the tool and where the test lands:

Layer	Proves	Lands in
backend	logic & data (tenant scoping, seat accounting)	backend/**/*_test.go
api	HTTP contract: status, schema, errors	Go handler tests / Playwright request
ui	a user journey in the browser	e2e/tests/*.spec.ts
binding	the UI↔API / API↔DB seam (payload sent + response rendered)	e2e/tests/*.spec.ts (with API capture)

Worked examples to copy from: BOOK-012 (binding), API-005 (api), SRCH-003 (ui).

The card's evidence: list is a contract: the generated test must capture exactly those screenshots and API calls, and the report must show them.

2. Automate it with Claude Code

From the repo root, use the prompt in generate-test.prompt.md with your card id. Claude reads the card and the conventions in e2e/CLAUDE.md and generates the spec using our page objects and fixtures.

Then you review — the diff and the evidence:

• the test uses page objects, not raw selectors;
• every then has an assertion on observable state;
• the evidence report shows each screenshot and API request/response the card listed.

The report is the proof of work: the AI claims it tested the scenario; the captured screenshots and API traffic prove it. Review the evidence, not just the code. Full reviewer checklist is in the prompt file.

Tip: npx playwright codegen http://localhost:4200 records a rough draft by clicking; then ask Claude to refactor it into page-object style. You never write selectors by hand.

3. Run it — one ps1, logged, CI-ready

run-tests.ps1 runs any layer, logs everything, and emits machine-readable results. The same script runs locally, in a git hook, and in a pipeline.

.\run-tests.ps1 -Layer all                         # everything
.\run-tests.ps1 -Layer unit                        # backend + web, no browser (fast)
.\run-tests.ps1 -Layer ui -Grep "oversell" -Open   # one scenario + open the report
.\run-tests.ps1 -Layer binding                      # UI<->API seam with traffic capture

What you get per run, under .testlogs/<timestamp>/:

• a log per layer (backend.log, ui.log, …) — full output, tee'd live;
• junit.xml — the result format Azure DevOps / Gitea publish;
• summary.json — pass/fail per layer;
• the evidence report (e2e/evidence-report/) — screenshots + the request, payload and response of every API call, with credentials redacted.

It also re-seeds the demo data if it has gone stale (the seeded flights are dated relative to now and expire after ~3 weeks), so you never chase a failure that is really just old data.

Exit code is 0 only if every layer passed — that is what CI gates on.

4. Run continuously, before CI and CD

Five cadences, fastest to slowest — the same run-tests.ps1 drives them all:

When	What	Enforced?
while writing	watch mode (ng test, playwright --ui)	no
pre-commit (git hook)	secret scan	local
pre-push (git hook)	run-tests.ps1 -Layer unit	local
pre-merge (CI)	full suite + evidence artifact	blocks merge
nightly (scheduled)	full suite	alerts on drift

Install the local hooks once after cloning:

.\scripts\install-hooks.ps1

• pre-commit blocks committed secrets (it is what would have caught the dev DB password leak). Fast, no toolchain needed.
• pre-push runs the unit layers so a logic break never leaves your machine.

"Before CI" is those two hooks (shift-left). "Before CD" is the pre-merge gate plus a smoke run against staging before promote. The nightly run matters: the stale-data failure above would surface at 2 a.m. instead of in front of a person.

CI / CD wiring

run-tests.ps1 is the contract; the pipeline just calls it and publishes its outputs (junit.xml + the evidence report). Azure DevOps integration is planned — when we add it, the pipeline will run this same script and use PublishTestResults@2 on junit.xml plus PublishBuildArtifacts on the evidence report. Nothing in steps 1–3 changes.

---

The 15-minute demo

Walk one card end to end:

1. Open BOOK-012 — the scenario.
2. Prompt Claude Code with the template → watch it generate the Playwright spec using the page objects.
3. .\run-tests.ps1 -Layer binding -Grep "oversell" -Open → logs stream, the evidence report opens showing the 409 and both bookings' request/response.
4. (Later) show the same ps1 running in the pipeline, blocking a bad merge.

That one loop is the whole system.