# TravelHub QA workflow — AI-assisted testing

This is how we define, automate, run and continuously gate tests on TravelHub.
It is the same loop for API, UI, backend and binding scenarios:

```
scenario card  ->  Claude generates the test  ->  run-tests.ps1 runs + logs + evidence  ->  gates CI/CD
     (you)              (AI, you review)                   (one runner, anywhere)
```

The scenario card is the single source of truth: a human reviews it, Claude
generates the test from it, and the evidence report traces each result back to
it. **No card, no test.**

---

## 1. Define a scenario

Copy [`scenario-template.yml`](scenario-template.yml) to
`qa/scenarios/<ID>.yml` and fill it in. Pick the layer — it decides the tool and
where the test lands:

| Layer | Proves | Lands in |
| ----- | ------ | -------- |
| `backend` | logic & data (tenant scoping, seat accounting) | `backend/**/*_test.go` |
| `api` | HTTP contract: status, schema, errors | Go handler tests / Playwright request |
| `ui` | a user journey in the browser | `e2e/tests/*.spec.ts` |
| `binding` | the UI↔API / API↔DB seam (payload sent + response rendered) | `e2e/tests/*.spec.ts` (with API capture) |

Worked examples to copy from:
[BOOK-012](scenarios/BOOK-012-no-oversell.yml) (binding),
[API-005](scenarios/API-005-cross-tenant-booking.yml) (api),
[SRCH-003](scenarios/SRCH-003-sort-by-fare.yml) (ui).

The card's `evidence:` list is a contract: the generated test must capture
exactly those screenshots and API calls, and the report must show them.

## 2. Automate it with Claude Code

**The quick way — the `qa` tool.** It assembles a *grounded* generation prompt
(your card + the app model + the conventions) so Claude never guesses selectors,
routes, or endpoints:

```bash
npm run qa:index                       # build the app-model grounding index (do once / after UI changes)
node scripts/qa-tool.mjs list          # list scenario cards
node scripts/qa-tool.mjs prompt BOOK-012   # print the grounded prompt — paste into Claude Code
node scripts/qa-tool.mjs run    BOOK-012   # run the card's tests + open the evidence report
```

`npm run qa:index` reads the codebase and writes
[`app-model.md`](app-model.md) / `app-model.json`: the routes, page objects,
API endpoints, areas and accounts that actually exist. `prompt` embeds that plus
the card and [`e2e/CLAUDE.md`](../e2e/CLAUDE.md) into one ready-to-paste ask, so
the generated test is correct by construction. `run` wraps `run-tests.ps1` for
the card's layer and opens the evidence.

Generation stays human-in-the-loop: the tool prepares the grounded ask and runs
the result; you (with Claude Code) write and review the test.

**Prefer a screen?** `npm run qa:ui` opens a small local web UI (localhost:9500)
over the same commands: pick a card, copy its grounded prompt, hit **Run tests**,
and see the evidence report inline. Same engine, nicer front door.

**Or by hand:** use the template in
[`generate-test.prompt.md`](generate-test.prompt.md) with your card id. Claude
reads the card and the conventions in [`e2e/CLAUDE.md`](../e2e/CLAUDE.md) and
generates the spec using our page objects and fixtures.

Then **you review** — the diff *and* the evidence:

- the test uses page objects, not raw selectors;
- every `then` has an assertion on observable state;
- the evidence report shows each screenshot and API request/response the card
  listed.

The report is the proof of work: the AI *claims* it tested the scenario; the
captured screenshots and API traffic *prove* it. Review the evidence, not just
the code. Full reviewer checklist is in the prompt file.

> Tip: `npx playwright codegen http://localhost:4200` records a rough draft by
> clicking; then ask Claude to refactor it into page-object style. You never
> write selectors by hand.

## 3. Run it — one ps1, logged, CI-ready

[`run-tests.ps1`](../run-tests.ps1) runs any layer, logs everything, and emits
machine-readable results. The **same script runs locally, in a git hook, and in
a pipeline**.

```powershell
.\run-tests.ps1 -Layer all                         # everything
.\run-tests.ps1 -Layer unit                        # backend + web, no browser (fast)
.\run-tests.ps1 -Layer ui -Grep "oversell" -Open   # one scenario + open the report
.\run-tests.ps1 -Layer binding                      # UI<->API seam with traffic capture
```

What you get per run, under `.testlogs/<timestamp>/`:

- a log per layer (`backend.log`, `ui.log`, …) — full output, tee'd live;
- `junit.xml` — the result format Azure DevOps / Gitea publish;
- `summary.json` — pass/fail per layer;
- the **evidence report** (`e2e/evidence-report/`) — screenshots + the request,
  payload and response of every API call, with credentials redacted.

It also **re-seeds the demo data if it has gone stale** (the seeded flights are
dated relative to now and expire after ~3 weeks), so you never chase a failure
that is really just old data.

Exit code is 0 only if every layer passed — that is what CI gates on.

## 4. Run continuously, before CI and CD

Five cadences, fastest to slowest — the same `run-tests.ps1` drives them all:

| When | What | Enforced? |
| ---- | ---- | --------- |
| while writing | watch mode (`ng test`, `playwright --ui`) | no |
| **pre-commit** (git hook) | secret scan | local |
| **pre-push** (git hook) | `run-tests.ps1 -Layer unit` | local |
| **pre-merge** (CI) | full suite + evidence artifact | **blocks merge** |
| nightly (scheduled) | full suite | alerts on drift |

Install the local hooks once after cloning:

```powershell
.\scripts\install-hooks.ps1
```

- **pre-commit** blocks committed secrets (it is what would have caught the dev
  DB password leak). Fast, no toolchain needed.
- **pre-push** runs the unit layers so a logic break never leaves your machine.

"Before CI" is those two hooks (shift-left). "Before CD" is the pre-merge gate
plus a smoke run against staging before promote. The **nightly** run matters:
the stale-data failure above would surface at 2 a.m. instead of in front of a
person.

### CI / CD wiring

`run-tests.ps1` is the contract; the pipeline just calls it and publishes its
outputs (`junit.xml` + the evidence report). **Azure DevOps integration is
planned** — when we add it, the pipeline will run this same script and use
`PublishTestResults@2` on `junit.xml` plus `PublishBuildArtifacts` on the
evidence report. Nothing in steps 1–3 changes.

---

## The 15-minute demo

Walk one card end to end:

1. Open [BOOK-012](scenarios/BOOK-012-no-oversell.yml) — the scenario.
2. Prompt Claude Code with [the template](generate-test.prompt.md) → watch it
   generate the Playwright spec using the page objects.
3. `.\run-tests.ps1 -Layer binding -Grep "oversell" -Open` → logs stream, the
   evidence report opens showing the 409 and both bookings' request/response.
4. (Later) show the same ps1 running in the pipeline, blocking a bad merge.

That one loop is the whole system.