Add AI-assisted QA test workflow: scenario cards, runner, hooks, API capture

A complete loop for defining, generating, running and gating tests:

- qa/: scenario-card template + examples (API/UI/binding), the Claude
  generation prompt, and a QA runbook tying it together
- e2e/CLAUDE.md: conventions that keep AI-generated tests consistent
- run-tests.ps1: one runner for every layer (backend/api/web/ui/binding),
  logs per run under .testlogs/, JUnit output, evidence report, and a
  re-seed guard against stale demo data
- .githooks/ + scripts/install-hooks.ps1: pre-commit secret scan and
  pre-push unit tests (shift-left before CI)
- evidence report now captures each API call's request, payload and
  response (Authorization redacted); JUnit reporter added for CI
- scripts/db.mjs: db:check freshness guard

Azure DevOps pipeline deferred; the runner already emits JUnit + the
evidence report for any CI to publish.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

e2e/CLAUDE.md

@@ -0,0 +1,72 @@
+# Conventions for generating Playwright tests in this project
+
+Read this before generating or editing any test under `e2e/`. These rules keep
+AI-generated tests consistent, reviewable, and trustworthy. A generated test
+that breaks them should be rejected in review.
+
+## Where things go
+
+- **Specs** live in `e2e/tests/<area>.spec.ts`, grouped by the areas defined in
+  `e2e/reporters/report-model.ts` (auth, search, booking, tenant-isolation,
+  dashboard, simulation, responsive). Add a scenario to the spec whose area it
+  belongs to; do not create a new spec file per scenario.
+- **Page objects** live in `e2e/pages/*.page.ts`. One class per screen,
+  exposing intent (`search.searchRoute('JFK','LHR')`), never raw selectors in
+  the test.
+- **Fixtures** live in `e2e/fixtures/`. Import `test` and `expect` from
+  `../fixtures/test`, never from `@playwright/test` directly — the project
+  fixtures add evidence capture, the console-error guard, and API-traffic
+  capture automatically.
+- **Accounts** come from `e2e/fixtures/accounts.ts`. Never hardcode an email or
+  password in a spec.
+
+## Hard rules
+
+1. **No raw selectors in specs.** If a selector is missing, add a method or
+   locator to the relevant page object and use that. Prefer role- and
+   text-based locators; avoid brittle CSS/nth-child chains.
+2. **No hardcoded waits.** Never `waitForTimeout`. Wait on state: `await
+   expect(locator).toBeVisible()`, `waitForURL`, the page object's `settled()`.
+3. **Never weaken an assertion to make a test pass.** If the app is wrong, the
+   test should fail. Report the discrepancy; do not loosen the check.
+4. **Never hardcode or print secrets.** Credentials come from `accounts.ts`.
+   The API-capture fixture already redacts the `Authorization` header — do not
+   defeat it.
+5. **Capture evidence at each `then`.** Use `evidence.step('<what>', { ...data })`
+   so the report shows a screenshot and the figures the test observed. Use
+   `evidence.note(...)` for data compared across pages.
+6. **Leave the data as you found it.** A test that creates a booking must cancel
+   it (or the suite's cleanup must). Use `uniqueSurname()` so rows are findable
+   and removable.
+7. **Assert on behaviour the user can observe**, not on implementation details.
+   `expect(seatsAfter).toBe(seatsBefore - 2)` survives a refactor; asserting an
+   internal call does not.
+
+## Shape of a generated test
+
+```ts
+import { expect, test, uniqueSurname } from '../fixtures/test';
+import { SKYLINE_AGENT } from '../fixtures/accounts';
+
+test.describe('Booking and cancelling', () => {
+  test('the last seat cannot be oversold', async ({ signInAs, search, checkout, evidence, page }) => {
+    // given
+    await signInAs(SKYLINE_AGENT);
+    // when ... (drive through page objects)
+    // then ...
+    await expect(/* observable state */).toBeTruthy();
+    await evidence.step('Seat refused as oversold', { status: 409 });
+  });
+});
+```
+
+## After generating
+
+Run it and read the evidence report before committing:
+
+```
+..\run-tests.ps1 -Layer ui -Grep "<part of the title>" -Open
+```
+
+The report must show the screenshots and API request/response the scenario card
+listed under `evidence:`. If they are missing, the test is not done.

e2e/fixtures/http-capture.ts

@@ -0,0 +1,110 @@
+/**
+ * Captures the API traffic a test produced, for the evidence report.
+ *
+ * A screenshot shows what the user saw; this shows what crossed the wire -- the
+ * request (method, path, payload) and the response (status, body) for every
+ * `/api/**` call. Together they make the report a complete record of the
+ * scenario.
+ *
+ * Security: the Authorization header carries the Basic-auth credential (a
+ * base64 of email:password), so it is redacted before anything is stored. The
+ * report is meant to be shared; a credential must never ride along in it.
+ */
+import type { Response } from '@playwright/test';
+
+export interface ApiCall {
+  method: string;
+  path: string;
+  status: number;
+  statusText: string;
+  durationMs: number;
+  requestHeaders: Record<string, string>;
+  requestBody?: string;
+  responseBody?: string;
+  truncated?: boolean;
+}
+
+// Headers that must never appear in a shareable report.
+const SENSITIVE = new Set(['authorization', 'cookie', 'set-cookie', 'proxy-authorization']);
+
+// Response bodies are capped so a large flight list cannot bloat the report.
+const MAX_BODY = 12000;
+
+function redactHeaders(headers: Record<string, string>): Record<string, string> {
+  const out: Record<string, string> = {};
+  for (const [key, value] of Object.entries(headers)) {
+    out[key] = SENSITIVE.has(key.toLowerCase()) ? '[redacted]' : value;
+  }
+  return out;
+}
+
+/** Pretty-prints JSON and caps length, reporting whether it was truncated. */
+function presentBody(raw: string, contentType: string): { body: string; truncated: boolean } {
+  let text = raw;
+  if (/json/i.test(contentType)) {
+    try {
+      text = JSON.stringify(JSON.parse(raw), null, 2);
+    } catch {
+      // Leave non-parseable bodies as-is.
+    }
+  }
+  if (text.length > MAX_BODY) {
+    return { body: text.slice(0, MAX_BODY) + '\n… (truncated)', truncated: true };
+  }
+  return { body: text, truncated: false };
+}
+
+/**
+ * Turns one response into an ApiCall record, or null if it is not an API call.
+ * The body is read here, while the response is still live, and returned via the
+ * promise the caller awaits at test teardown.
+ */
+export function captureApiCall(response: Response): Promise<ApiCall | null> {
+  const request = response.request();
+
+  let url: URL;
+  try {
+    url = new URL(request.url());
+  } catch {
+    return Promise.resolve(null);
+  }
+  if (!url.pathname.startsWith('/api/')) {
+    return Promise.resolve(null);
+  }
+
+  return (async () => {
+    try {
+      const contentType = response.headers()['content-type'] ?? '';
+      let responseBody: string | undefined;
+      let truncated = false;
+
+      if (/json|text|xml|form/i.test(contentType)) {
+        const raw = await response.text().catch(() => '');
+        if (raw) {
+          const shown = presentBody(raw, contentType);
+          responseBody = shown.body;
+          truncated = shown.truncated;
+        }
+      }
+
+      const timing = request.timing();
+      const durationMs = timing.responseEnd > 0 ? Math.round(timing.responseEnd) : 0;
+
+      return {
+        method: request.method(),
+        path: url.pathname + url.search,
+        status: response.status(),
+        statusText: response.statusText(),
+        durationMs,
+        requestHeaders: redactHeaders(request.headers()),
+        requestBody: request.postData() ?? undefined,
+        responseBody,
+        truncated: truncated || undefined,
+      };
+    } catch {
+      // A response that cannot be read (e.g. the page navigated away) is
+      // skipped rather than failing the test it was only observing.
+      return null;
+    }
+  })();
+}

e2e/fixtures/test.ts

@@ -2,6 +2,7 @@ import { expect, test as base } from '@playwright/test';
 
 import { Account } from './accounts';
 import { Evidence } from './evidence';
+import { ApiCall, captureApiCall } from './http-capture';
 import { BookingDetailPage, BookingsPage } from '../pages/bookings.page';
 import { CheckoutPage } from '../pages/checkout.page';
 import { DashboardPage } from '../pages/dashboard.page';
@@ -26,6 +27,9 @@ interface Fixtures {
 
   /** Console and uncaught errors seen during the test; asserted empty at the end. */
   consoleErrors: string[];
+
+  /** Records every /api/** request and response for the run report. */
+  apiTraffic: void;
 }
 
 export const test = base.extend<Fixtures>({
@@ -75,6 +79,28 @@ export const test = base.extend<Fixtures>({
     },
     { auto: true },
   ],
+
+  // Records the request/response of every API call the test made, attached as
+  // evidence-http for the reporter. Auto, so no test has to opt in.
+  apiTraffic: [
+    async ({ page }, use, testInfo) => {
+      const pending: Array<Promise<ApiCall | null>> = [];
+      page.on('response', (response) => {
+        pending.push(captureApiCall(response));
+      });
+
+      await use();
+
+      const calls = (await Promise.all(pending)).filter((c): c is ApiCall => c !== null);
+      if (calls.length > 0) {
+        await testInfo.attach('evidence-http', {
+          body: Buffer.from(JSON.stringify(calls, null, 2), 'utf8'),
+          contentType: 'application/json',
+        });
+      }
+    },
+    { auto: true },
+  ],
 });
 
 export { expect } from '@playwright/test';

e2e/playwright.config.ts

@@ -22,9 +22,22 @@ export default defineConfig({
 
   // The evidence reporter turns the screenshots and data captured by passing
   // tests into a readable run report; the html one stays for debugging failures.
+  // junit.xml is the machine-readable result any CI (Azure DevOps, Gitea) can
+  // ingest; the evidence reporter is the human record; html is for debugging.
   reporter: process.env.CI
-    ? [['github'], ['html', { open: 'never' }], ['./reporters/evidence.ts'], ['list']]
-    : [['html', { open: 'never' }], ['./reporters/evidence.ts'], ['list']],
+    ? [
+        ['github'],
+        ['junit', { outputFile: 'results/junit.xml' }],
+        ['html', { open: 'never' }],
+        ['./reporters/evidence.ts'],
+        ['list'],
+      ]
+    : [
+        ['junit', { outputFile: 'results/junit.xml' }],
+        ['html', { open: 'never' }],
+        ['./reporters/evidence.ts'],
+        ['list'],
+      ],
 
   use: {
     baseURL: WEB,

e2e/reporters/evidence.ts

@@ -21,6 +21,7 @@ import type { FullConfig, FullResult, Reporter, Suite, TestCase, TestResult } fr
 import { DATA_PREFIX, SHOT_PREFIX } from '../fixtures/evidence';
 import { renderHtml } from './render-html';
 import {
+  ApiCall,
   RunMeta,
   Scenario,
   Step,
@@ -69,6 +70,7 @@ export default class EvidenceReporter implements Reporter {
     const suite = path.slice(2, -1).join(' › ');
 
     const steps = this.collectSteps(test, result);
+    const apiCalls = this.collectApiCalls(result);
 
     this.scenarios.push({
       project,
@@ -79,10 +81,28 @@ export default class EvidenceReporter implements Reporter {
       expected: result.status === test.expectedStatus,
       durationMs: result.duration,
       steps,
+      apiCalls,
       error: result.error?.message?.split('\n').slice(0, 4).join('\n'),
     });
   }
 
+  /** Reads the evidence-http attachment the apiTraffic fixture produced. */
+  private collectApiCalls(result: TestResult): ApiCall[] {
+    const att = result.attachments.find((a) => a.name === 'evidence-http');
+    if (!att) {
+      return [];
+    }
+    const raw = att.body ? att.body.toString('utf8') : att.path ? safeRead(att.path) : '';
+    if (!raw) {
+      return [];
+    }
+    try {
+      return JSON.parse(raw) as ApiCall[];
+    } catch {
+      return [];
+    }
+  }
+
   /** Pairs `evidence:NN:name` screenshots with `evidence-data:NN:name` blobs. */
   private collectSteps(test: TestCase, result: TestResult): Step[] {
     const bySeq = new Map<string, Step>();
@@ -180,6 +200,7 @@ export default class EvidenceReporter implements Reporter {
         (n, s) => n + s.steps.filter((step) => step.shot).length,
         0,
       ),
+      apiCalls: this.scenarios.reduce((n, s) => n + s.apiCalls.length, 0),
     };
 
     mkdirSync(OUT_DIR, { recursive: true });
@@ -206,7 +227,7 @@ export default class EvidenceReporter implements Reporter {
         'Evidence report — what passed, with data and screenshots:',
         `  ${url}`,
         `  ${summary.passed}/${summary.total} passed · ${summary.screenshots} screenshots · ` +
-          `${summary.narrated} scenarios with recorded data`,
+          `${summary.apiCalls} API calls · ${summary.narrated} scenarios with recorded data`,
         '',
         '  Open it:  npm run report',
         '  Markdown: ' + pathToFileURL(join(OUT_DIR, 'report.md')).href,
@@ -254,7 +275,7 @@ function renderMarkdown(summary: Summary, scenarios: Scenario[]): string {
       lines.push(`### ${mark} ${heading}`, '');
       lines.push(`*${scenario.suite ? scenario.suite + ' · ' : ''}${scenario.project} · ${friendly(scenario.durationMs)}*`, '');
 
-      if (scenario.steps.length === 0) {
+      if (scenario.steps.length === 0 && scenario.apiCalls.length === 0) {
         lines.push('_No evidence captured._', '');
         continue;
       }
@@ -272,10 +293,32 @@ function renderMarkdown(summary: Summary, scenarios: Scenario[]): string {
           lines.push(`![${step.name}](${step.shot})`, '');
         }
       }
+
+      if (scenario.apiCalls.length > 0) {
+        lines.push(`**API calls (${scenario.apiCalls.length})**`, '');
+        for (const call of scenario.apiCalls) {
+          lines.push(`- \`${call.method} ${call.path}\` → **${call.status}** (${call.durationMs} ms)`);
+          if (call.requestBody) {
+            lines.push('', '  Request:', '', '  ```json', indent(call.requestBody), '  ```');
+          }
+          if (call.responseBody) {
+            lines.push('', '  Response:', '', '  ```json', indent(call.responseBody), '  ```');
+          }
+        }
+        lines.push('');
+      }
     }
   }
 
   return lines.join('\n');
 }
 
+/** Indents a block two spaces so it nests under a Markdown list item. */
+function indent(text: string): string {
+  return text
+    .split('\n')
+    .map((line) => '  ' + line)
+    .join('\n');
+}
+
 

e2e/reporters/render-html.ts

@@ -6,6 +6,7 @@
  * ("2.3.1 failed"), and a print stylesheet so it can be saved as a PDF.
  */
 import {
+  ApiCall,
   Area,
   RunMeta,
   Scenario,
@@ -108,6 +109,7 @@ ${STYLES}
       <div><dt>Scenarios executed</dt><dd>${summary.total}</dd></div>
       <div><dt>Duration</dt><dd>${friendly(summary.durationMs)}</dd></div>
       <div><dt>Screenshots captured</dt><dd>${summary.screenshots}</dd></div>
+      <div><dt>API calls captured</dt><dd>${summary.apiCalls}</dd></div>
       <div><dt>Scenarios with recorded data</dt><dd>${summary.narrated}</dd></div>
     </dl>
   </div>
@@ -207,10 +209,11 @@ function renderArea(entry: AreaBlock): string {
 
 function renderScenario(scenario: Scenario, ref: string): string {
   const hasFacts = scenario.steps.some((s) => s.facts);
-  const steps =
-    scenario.steps.length === 0
-      ? '<p class="no-evidence">No screenshot was captured for this scenario.</p>'
-      : scenario.steps.map(renderStep).join('');
+  const hasAnything = scenario.steps.length > 0 || scenario.apiCalls.length > 0;
+  const steps = scenario.steps.map(renderStep).join('');
+  const emptyNote = hasAnything
+    ? ''
+    : '<p class="no-evidence">No evidence was captured for this scenario.</p>';
 
   return `
   <details class="scenario" data-facts="${hasFacts ? '1' : '0'}"${hasFacts ? ' open' : ''}>
@@ -222,6 +225,7 @@ function renderScenario(scenario: Scenario, ref: string): string {
       </span>
       <span class="right">
         <span class="hide-narrow">${escapeHtml(scenario.project)}</span>
+        <span class="hide-narrow">${scenario.apiCalls.length} API</span>
         <span>${friendly(scenario.durationMs)}</span>
         <span class="tag ${statusClass(scenario.status)}">${statusWord(scenario.status)}</span>
       </span>
@@ -229,10 +233,56 @@ function renderScenario(scenario: Scenario, ref: string): string {
     <div class="body">
       ${scenario.error ? `<pre class="error">${escapeHtml(scenario.error)}</pre>` : ''}
       ${steps}
+      ${emptyNote}
+      ${renderApiCalls(scenario.apiCalls)}
     </div>
   </details>`;
 }
 
+/** The API traffic block: collapsed by default so it never dominates. */
+function renderApiCalls(calls: ApiCall[]): string {
+  if (calls.length === 0) {
+    return '';
+  }
+
+  const rows = calls.map(renderApiCall).join('');
+
+  return `
+    <details class="api">
+      <summary>API calls &middot; ${calls.length} request${calls.length === 1 ? '' : 's'}</summary>
+      <div class="api-list">${rows}</div>
+    </details>`;
+}
+
+function renderApiCall(call: ApiCall): string {
+  const cls = call.status >= 500 ? 'c5' : call.status >= 400 ? 'c4' : call.status >= 300 ? 'c3' : 'c2';
+
+  const headerRows = Object.entries(call.requestHeaders)
+    .map(([k, v]) => `<tr><td>${escapeHtml(k)}</td><td>${escapeHtml(v)}</td></tr>`)
+    .join('');
+
+  const request = call.requestBody
+    ? `<div class="part"><span class="part-label">Request payload</span><pre>${escapeHtml(call.requestBody)}</pre></div>`
+    : '';
+
+  const response = call.responseBody
+    ? `<div class="part"><span class="part-label">Response body${call.truncated ? ' (truncated)' : ''}</span><pre>${escapeHtml(call.responseBody)}</pre></div>`
+    : '';
+
+  return `
+    <div class="call">
+      <div class="call-head">
+        <span class="method m-${escapeHtml(call.method.toLowerCase())}">${escapeHtml(call.method)}</span>
+        <span class="call-path">${escapeHtml(call.path)}</span>
+        <span class="status ${cls}">${call.status} ${escapeHtml(call.statusText)}</span>
+        <span class="call-time">${call.durationMs} ms</span>
+      </div>
+      <details class="headers"><summary>Request headers</summary><table>${headerRows}</table></details>
+      ${request}
+      ${response}
+    </div>`;
+}
+
 function renderStep(step: Step): string {
   const facts = step.facts
     ? `<table class="facts">${Object.entries(step.facts)
@@ -372,6 +422,41 @@ const STYLES = `
          border-radius:6px;font-size:.83rem;white-space:pre-wrap;
          font-family:ui-monospace,monospace}
 
+  /* --- API traffic --- */
+  details.api{border-top:1px solid var(--line);margin-top:6px}
+  details.api > summary{list-style:none;cursor:pointer;padding:12px 0 10px;font-size:.78rem;
+       font-weight:650;letter-spacing:.04em;text-transform:uppercase;color:var(--muted)}
+  details.api > summary::-webkit-details-marker{display:none}
+  details.api > summary::before{content:'\\25B8 ';color:var(--faint)}
+  details.api[open] > summary::before{content:'\\25BE '}
+  .api-list{display:flex;flex-direction:column;gap:10px;padding-bottom:6px}
+  .call{border:1px solid var(--line);border-radius:7px;overflow:hidden;background:#fff}
+  .call-head{display:flex;align-items:center;gap:10px;padding:8px 11px;background:#f8fafc;
+       flex-wrap:wrap;font-size:.82rem}
+  .method{font-weight:700;font-size:.7rem;letter-spacing:.04em;padding:2px 7px;border-radius:4px;
+       background:#e5e7eb;color:#374151}
+  .method.m-get{background:#e0edff;color:#0b5cd6}
+  .method.m-post{background:#e6f4f1;color:#067a6e}
+  .method.m-patch,.method.m-put{background:#fef3c7;color:#b45309}
+  .method.m-delete{background:#fdeaee;color:#b3243f}
+  .call-path{font-family:ui-monospace,Menlo,monospace;font-size:.8rem;color:var(--ink);
+       word-break:break-all;flex:1;min-width:120px}
+  .status{font-weight:700;font-size:.76rem;font-variant-numeric:tabular-nums}
+  .status.c2{color:var(--ok)} .status.c3{color:var(--med)}
+  .status.c4{color:var(--fail)} .status.c5{color:#fff;background:var(--fail);padding:1px 6px;border-radius:4px}
+  .call-time{color:var(--faint);font-size:.76rem;font-variant-numeric:tabular-nums}
+  .part{padding:8px 11px;border-top:1px solid var(--line)}
+  .part-label{display:block;font-size:.68rem;letter-spacing:.06em;text-transform:uppercase;
+       color:var(--faint);font-weight:650;margin-bottom:5px}
+  .call pre{background:#0f172a;color:#e2e8f0;padding:10px 12px;border-radius:6px;overflow-x:auto;
+       font:12px/1.5 ui-monospace,Menlo,monospace;margin:0;max-height:340px}
+  details.headers{padding:8px 11px;border-top:1px solid var(--line)}
+  details.headers > summary{cursor:pointer;font-size:.72rem;color:var(--muted)}
+  details.headers table{border-collapse:collapse;font-size:.76rem;margin-top:6px}
+  details.headers td{border:1px solid var(--line);padding:3px 8px;
+       font-family:ui-monospace,Menlo,monospace;word-break:break-all}
+  details.headers td:first-child{color:var(--muted);background:#f8fafc}
+
   .toolbar{display:flex;flex-wrap:wrap;gap:8px;align-items:center;margin-bottom:20px}
   .toolbar button{border:1px solid var(--rule);background:#fff;border-radius:6px;
                   padding:6px 13px;font:inherit;font-size:.85rem;cursor:pointer;color:var(--body)}
@@ -468,7 +553,9 @@ const SCRIPT = `
   // first and put it back the way the reader had it afterwards.
   let restore = [];
   window.addEventListener('beforeprint', () => {
-    restore = [...document.querySelectorAll('.scenario')].map((d) => [d, d.open]);
+    // Open every details element (scenarios and the API/header blocks inside),
+    // since a collapsed one prints only its summary line.
+    restore = [...document.querySelectorAll('details')].map((d) => [d, d.open]);
     for (const [d] of restore) {
       d.open = true;
     }

e2e/reporters/report-model.ts

@@ -15,6 +15,19 @@ export interface Step {
   facts?: Record<string, unknown>;
 }
 
+/** One API request/response pair observed during a scenario. */
+export interface ApiCall {
+  method: string;
+  path: string;
+  status: number;
+  statusText: string;
+  durationMs: number;
+  requestHeaders: Record<string, string>;
+  requestBody?: string;
+  responseBody?: string;
+  truncated?: boolean;
+}
+
 /** One test, as the report talks about it. */
 export interface Scenario {
   project: string;
@@ -25,6 +38,7 @@ export interface Scenario {
   expected: boolean;
   durationMs: number;
   steps: Step[];
+  apiCalls: ApiCall[];
   error?: string;
 }
 
@@ -45,6 +59,7 @@ export interface Summary {
   skipped: number;
   narrated: number;
   screenshots: number;
+  apiCalls: number;
 }
 
 // --- areas -----------------------------------------------------------------