Add AI-assisted QA test workflow: scenario cards, runner, hooks, API capture

A complete loop for defining, generating, running and gating tests:

- qa/: scenario-card template + examples (API/UI/binding), the Claude
  generation prompt, and a QA runbook tying it together
- e2e/CLAUDE.md: conventions that keep AI-generated tests consistent
- run-tests.ps1: one runner for every layer (backend/api/web/ui/binding),
  logs per run under .testlogs/, JUnit output, evidence report, and a
  re-seed guard against stale demo data
- .githooks/ + scripts/install-hooks.ps1: pre-commit secret scan and
  pre-push unit tests (shift-left before CI)
- evidence report now captures each API call's request, payload and
  response (Authorization redacted); JUnit reporter added for CI
- scripts/db.mjs: db:check freshness guard

Azure DevOps pipeline deferred; the runner already emits JUnit + the
evidence report for any CI to publish.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
nilesh.p
2026-10-10 13:04:46 +05:30
co-authored by Claude Opus 4.8
parent df9e534e42
commit 45b5b34680
20 changed files with 1054 additions and 11 deletions
+45
View File
@@ -0,0 +1,45 @@
#!/bin/sh
# Fast, local shift-left gate: block obvious secrets from being committed.
# Runs in seconds, needs no toolchain. Enable with: scripts/install-hooks.ps1
#
# Bypass in a genuine emergency with: git commit --no-verify (discouraged).
staged=$(git diff --cached --name-only --diff-filter=ACM)
[ -z "$staged" ] && exit 0
fail=0
report() { echo " BLOCKED: $1"; fail=1; }
for f in $staged; do
# Skip the hook scripts themselves: they legitimately contain the patterns
# (and literals) they search for, so scanning them would self-block.
case "$f" in
.githooks/*) continue;;
esac
# Never commit a real env file (only .env.example is allowed).
case "$f" in
*.env|.env) report "$f looks like a secrets file (.env is git-ignored for a reason)";;
esac
# Scan staged content for credential patterns.
[ -f "$f" ] || continue
content=$(git show ":$f" 2>/dev/null) || continue
echo "$content" | grep -Eq 'GITEA_TOKEN[[:space:]]*=[[:space:]]*[A-Za-z0-9]' && \
report "$f contains a GITEA_TOKEN value"
echo "$content" | grep -Eq 'PGPASSWORD[[:space:]]*=[[:space:]]*[^ \"$]' && \
report "$f hardcodes PGPASSWORD"
echo "$content" | grep -q 'BEGIN .*PRIVATE KEY' && \
report "$f contains a private key"
# The known dev DB password must never re-enter the tree (see VAPT FINDING-8).
echo "$content" | grep -q 'Did10n@c' && \
report "$f contains the real dev DB password"
done
if [ "$fail" -ne 0 ]; then
echo ""
echo "Commit blocked: remove the secret(s) above, or use .env (git-ignored)."
exit 1
fi
exit 0
+20
View File
@@ -0,0 +1,20 @@
#!/bin/sh
# Pre-push gate: run the fast test layers (Go + Angular unit) before code leaves
# the machine. The browser e2e suite is left for CI to keep push quick.
#
# Bypass in a genuine emergency with: git push --no-verify (discouraged).
echo "pre-push: running unit layers (backend + web) ..."
# Delegate to the one runner so local and CI behave identically. The ps1
# refreshes PATH itself, so this works even in a terminal opened before the
# toolchain was installed.
powershell.exe -NoProfile -ExecutionPolicy Bypass -File ./run-tests.ps1 -Layer unit -NoReport
code=$?
if [ "$code" -ne 0 ]; then
echo ""
echo "pre-push blocked: unit tests failed. Fix them, or push --no-verify to override."
exit 1
fi
exit 0
+2
View File
@@ -38,3 +38,5 @@ e2e/**/*.pdf
# --- app.ps1 runtime (pids, logs) ------------------------------------------- # --- app.ps1 runtime (pids, logs) -------------------------------------------
.run/ .run/
.testlogs/
e2e/results/
+12 -2
View File
@@ -290,8 +290,18 @@ invisibility, and the dashboard before and after a sale. Every other test still
contributes its end-of-test screenshot automatically (`screenshot: 'on'`), so contributes its end-of-test screenshot automatically (`screenshot: 'on'`), so
the report covers the whole run; the toolbar filters to the narrated ones. the report covers the whole run; the toolbar filters to the narrated ones.
A typical full run produces **96 screenshots across 82 scenarios**, about 14 MB. **API traffic.** Every scenario also records the request/response of every
The directory is git-ignored and CI keeps it as an artifact for 30 days. `/api/**` call it made — method, path, request headers, payload, status, and
response body — shown as a collapsible "API calls" block under the scenario.
This is automatic (an auto fixture in `fixtures/http-capture.ts`); no test has
to opt in. The **`Authorization` header is redacted** before anything is stored
— it carries the Basic-auth credential, and the report is meant to be shared.
Response bodies are capped (12 KB) so a large flight list cannot bloat the
report.
A typical full run produces **96 screenshots and ~390 captured API calls across
82 scenarios**, about 15 MB of HTML. The directory is git-ignored and CI keeps
it as an artifact for 30 days.
### Simulated-condition tests ### Simulated-condition tests
+72
View File
@@ -0,0 +1,72 @@
# Conventions for generating Playwright tests in this project
Read this before generating or editing any test under `e2e/`. These rules keep
AI-generated tests consistent, reviewable, and trustworthy. A generated test
that breaks them should be rejected in review.
## Where things go
- **Specs** live in `e2e/tests/<area>.spec.ts`, grouped by the areas defined in
`e2e/reporters/report-model.ts` (auth, search, booking, tenant-isolation,
dashboard, simulation, responsive). Add a scenario to the spec whose area it
belongs to; do not create a new spec file per scenario.
- **Page objects** live in `e2e/pages/*.page.ts`. One class per screen,
exposing intent (`search.searchRoute('JFK','LHR')`), never raw selectors in
the test.
- **Fixtures** live in `e2e/fixtures/`. Import `test` and `expect` from
`../fixtures/test`, never from `@playwright/test` directly — the project
fixtures add evidence capture, the console-error guard, and API-traffic
capture automatically.
- **Accounts** come from `e2e/fixtures/accounts.ts`. Never hardcode an email or
password in a spec.
## Hard rules
1. **No raw selectors in specs.** If a selector is missing, add a method or
locator to the relevant page object and use that. Prefer role- and
text-based locators; avoid brittle CSS/nth-child chains.
2. **No hardcoded waits.** Never `waitForTimeout`. Wait on state: `await
expect(locator).toBeVisible()`, `waitForURL`, the page object's `settled()`.
3. **Never weaken an assertion to make a test pass.** If the app is wrong, the
test should fail. Report the discrepancy; do not loosen the check.
4. **Never hardcode or print secrets.** Credentials come from `accounts.ts`.
The API-capture fixture already redacts the `Authorization` header — do not
defeat it.
5. **Capture evidence at each `then`.** Use `evidence.step('<what>', { ...data })`
so the report shows a screenshot and the figures the test observed. Use
`evidence.note(...)` for data compared across pages.
6. **Leave the data as you found it.** A test that creates a booking must cancel
it (or the suite's cleanup must). Use `uniqueSurname()` so rows are findable
and removable.
7. **Assert on behaviour the user can observe**, not on implementation details.
`expect(seatsAfter).toBe(seatsBefore - 2)` survives a refactor; asserting an
internal call does not.
## Shape of a generated test
```ts
import { expect, test, uniqueSurname } from '../fixtures/test';
import { SKYLINE_AGENT } from '../fixtures/accounts';
test.describe('Booking and cancelling', () => {
test('the last seat cannot be oversold', async ({ signInAs, search, checkout, evidence, page }) => {
// given
await signInAs(SKYLINE_AGENT);
// when ... (drive through page objects)
// then ...
await expect(/* observable state */).toBeTruthy();
await evidence.step('Seat refused as oversold', { status: 409 });
});
});
```
## After generating
Run it and read the evidence report before committing:
```
..\run-tests.ps1 -Layer ui -Grep "<part of the title>" -Open
```
The report must show the screenshots and API request/response the scenario card
listed under `evidence:`. If they are missing, the test is not done.
+110
View File
@@ -0,0 +1,110 @@
/**
* Captures the API traffic a test produced, for the evidence report.
*
* A screenshot shows what the user saw; this shows what crossed the wire -- the
* request (method, path, payload) and the response (status, body) for every
* `/api/**` call. Together they make the report a complete record of the
* scenario.
*
* Security: the Authorization header carries the Basic-auth credential (a
* base64 of email:password), so it is redacted before anything is stored. The
* report is meant to be shared; a credential must never ride along in it.
*/
import type { Response } from '@playwright/test';
export interface ApiCall {
method: string;
path: string;
status: number;
statusText: string;
durationMs: number;
requestHeaders: Record<string, string>;
requestBody?: string;
responseBody?: string;
truncated?: boolean;
}
// Headers that must never appear in a shareable report.
const SENSITIVE = new Set(['authorization', 'cookie', 'set-cookie', 'proxy-authorization']);
// Response bodies are capped so a large flight list cannot bloat the report.
const MAX_BODY = 12000;
function redactHeaders(headers: Record<string, string>): Record<string, string> {
const out: Record<string, string> = {};
for (const [key, value] of Object.entries(headers)) {
out[key] = SENSITIVE.has(key.toLowerCase()) ? '[redacted]' : value;
}
return out;
}
/** Pretty-prints JSON and caps length, reporting whether it was truncated. */
function presentBody(raw: string, contentType: string): { body: string; truncated: boolean } {
let text = raw;
if (/json/i.test(contentType)) {
try {
text = JSON.stringify(JSON.parse(raw), null, 2);
} catch {
// Leave non-parseable bodies as-is.
}
}
if (text.length > MAX_BODY) {
return { body: text.slice(0, MAX_BODY) + '\n… (truncated)', truncated: true };
}
return { body: text, truncated: false };
}
/**
* Turns one response into an ApiCall record, or null if it is not an API call.
* The body is read here, while the response is still live, and returned via the
* promise the caller awaits at test teardown.
*/
export function captureApiCall(response: Response): Promise<ApiCall | null> {
const request = response.request();
let url: URL;
try {
url = new URL(request.url());
} catch {
return Promise.resolve(null);
}
if (!url.pathname.startsWith('/api/')) {
return Promise.resolve(null);
}
return (async () => {
try {
const contentType = response.headers()['content-type'] ?? '';
let responseBody: string | undefined;
let truncated = false;
if (/json|text|xml|form/i.test(contentType)) {
const raw = await response.text().catch(() => '');
if (raw) {
const shown = presentBody(raw, contentType);
responseBody = shown.body;
truncated = shown.truncated;
}
}
const timing = request.timing();
const durationMs = timing.responseEnd > 0 ? Math.round(timing.responseEnd) : 0;
return {
method: request.method(),
path: url.pathname + url.search,
status: response.status(),
statusText: response.statusText(),
durationMs,
requestHeaders: redactHeaders(request.headers()),
requestBody: request.postData() ?? undefined,
responseBody,
truncated: truncated || undefined,
};
} catch {
// A response that cannot be read (e.g. the page navigated away) is
// skipped rather than failing the test it was only observing.
return null;
}
})();
}
+26
View File
@@ -2,6 +2,7 @@ import { expect, test as base } from '@playwright/test';
import { Account } from './accounts'; import { Account } from './accounts';
import { Evidence } from './evidence'; import { Evidence } from './evidence';
import { ApiCall, captureApiCall } from './http-capture';
import { BookingDetailPage, BookingsPage } from '../pages/bookings.page'; import { BookingDetailPage, BookingsPage } from '../pages/bookings.page';
import { CheckoutPage } from '../pages/checkout.page'; import { CheckoutPage } from '../pages/checkout.page';
import { DashboardPage } from '../pages/dashboard.page'; import { DashboardPage } from '../pages/dashboard.page';
@@ -26,6 +27,9 @@ interface Fixtures {
/** Console and uncaught errors seen during the test; asserted empty at the end. */ /** Console and uncaught errors seen during the test; asserted empty at the end. */
consoleErrors: string[]; consoleErrors: string[];
/** Records every /api/** request and response for the run report. */
apiTraffic: void;
} }
export const test = base.extend<Fixtures>({ export const test = base.extend<Fixtures>({
@@ -75,6 +79,28 @@ export const test = base.extend<Fixtures>({
}, },
{ auto: true }, { auto: true },
], ],
// Records the request/response of every API call the test made, attached as
// evidence-http for the reporter. Auto, so no test has to opt in.
apiTraffic: [
async ({ page }, use, testInfo) => {
const pending: Array<Promise<ApiCall | null>> = [];
page.on('response', (response) => {
pending.push(captureApiCall(response));
});
await use();
const calls = (await Promise.all(pending)).filter((c): c is ApiCall => c !== null);
if (calls.length > 0) {
await testInfo.attach('evidence-http', {
body: Buffer.from(JSON.stringify(calls, null, 2), 'utf8'),
contentType: 'application/json',
});
}
},
{ auto: true },
],
}); });
export { expect } from '@playwright/test'; export { expect } from '@playwright/test';
+15 -2
View File
@@ -22,9 +22,22 @@ export default defineConfig({
// The evidence reporter turns the screenshots and data captured by passing // The evidence reporter turns the screenshots and data captured by passing
// tests into a readable run report; the html one stays for debugging failures. // tests into a readable run report; the html one stays for debugging failures.
// junit.xml is the machine-readable result any CI (Azure DevOps, Gitea) can
// ingest; the evidence reporter is the human record; html is for debugging.
reporter: process.env.CI reporter: process.env.CI
? [['github'], ['html', { open: 'never' }], ['./reporters/evidence.ts'], ['list']] ? [
: [['html', { open: 'never' }], ['./reporters/evidence.ts'], ['list']], ['github'],
['junit', { outputFile: 'results/junit.xml' }],
['html', { open: 'never' }],
['./reporters/evidence.ts'],
['list'],
]
: [
['junit', { outputFile: 'results/junit.xml' }],
['html', { open: 'never' }],
['./reporters/evidence.ts'],
['list'],
],
use: { use: {
baseURL: WEB, baseURL: WEB,
+45 -2
View File
@@ -21,6 +21,7 @@ import type { FullConfig, FullResult, Reporter, Suite, TestCase, TestResult } fr
import { DATA_PREFIX, SHOT_PREFIX } from '../fixtures/evidence'; import { DATA_PREFIX, SHOT_PREFIX } from '../fixtures/evidence';
import { renderHtml } from './render-html'; import { renderHtml } from './render-html';
import { import {
ApiCall,
RunMeta, RunMeta,
Scenario, Scenario,
Step, Step,
@@ -69,6 +70,7 @@ export default class EvidenceReporter implements Reporter {
const suite = path.slice(2, -1).join(' › '); const suite = path.slice(2, -1).join(' › ');
const steps = this.collectSteps(test, result); const steps = this.collectSteps(test, result);
const apiCalls = this.collectApiCalls(result);
this.scenarios.push({ this.scenarios.push({
project, project,
@@ -79,10 +81,28 @@ export default class EvidenceReporter implements Reporter {
expected: result.status === test.expectedStatus, expected: result.status === test.expectedStatus,
durationMs: result.duration, durationMs: result.duration,
steps, steps,
apiCalls,
error: result.error?.message?.split('\n').slice(0, 4).join('\n'), error: result.error?.message?.split('\n').slice(0, 4).join('\n'),
}); });
} }
/** Reads the evidence-http attachment the apiTraffic fixture produced. */
private collectApiCalls(result: TestResult): ApiCall[] {
const att = result.attachments.find((a) => a.name === 'evidence-http');
if (!att) {
return [];
}
const raw = att.body ? att.body.toString('utf8') : att.path ? safeRead(att.path) : '';
if (!raw) {
return [];
}
try {
return JSON.parse(raw) as ApiCall[];
} catch {
return [];
}
}
/** Pairs `evidence:NN:name` screenshots with `evidence-data:NN:name` blobs. */ /** Pairs `evidence:NN:name` screenshots with `evidence-data:NN:name` blobs. */
private collectSteps(test: TestCase, result: TestResult): Step[] { private collectSteps(test: TestCase, result: TestResult): Step[] {
const bySeq = new Map<string, Step>(); const bySeq = new Map<string, Step>();
@@ -180,6 +200,7 @@ export default class EvidenceReporter implements Reporter {
(n, s) => n + s.steps.filter((step) => step.shot).length, (n, s) => n + s.steps.filter((step) => step.shot).length,
0, 0,
), ),
apiCalls: this.scenarios.reduce((n, s) => n + s.apiCalls.length, 0),
}; };
mkdirSync(OUT_DIR, { recursive: true }); mkdirSync(OUT_DIR, { recursive: true });
@@ -206,7 +227,7 @@ export default class EvidenceReporter implements Reporter {
'Evidence report — what passed, with data and screenshots:', 'Evidence report — what passed, with data and screenshots:',
` ${url}`, ` ${url}`,
` ${summary.passed}/${summary.total} passed · ${summary.screenshots} screenshots · ` + ` ${summary.passed}/${summary.total} passed · ${summary.screenshots} screenshots · ` +
`${summary.narrated} scenarios with recorded data`, `${summary.apiCalls} API calls · ${summary.narrated} scenarios with recorded data`,
'', '',
' Open it: npm run report', ' Open it: npm run report',
' Markdown: ' + pathToFileURL(join(OUT_DIR, 'report.md')).href, ' Markdown: ' + pathToFileURL(join(OUT_DIR, 'report.md')).href,
@@ -254,7 +275,7 @@ function renderMarkdown(summary: Summary, scenarios: Scenario[]): string {
lines.push(`### ${mark} ${heading}`, ''); lines.push(`### ${mark} ${heading}`, '');
lines.push(`*${scenario.suite ? scenario.suite + ' · ' : ''}${scenario.project} · ${friendly(scenario.durationMs)}*`, ''); lines.push(`*${scenario.suite ? scenario.suite + ' · ' : ''}${scenario.project} · ${friendly(scenario.durationMs)}*`, '');
if (scenario.steps.length === 0) { if (scenario.steps.length === 0 && scenario.apiCalls.length === 0) {
lines.push('_No evidence captured._', ''); lines.push('_No evidence captured._', '');
continue; continue;
} }
@@ -272,10 +293,32 @@ function renderMarkdown(summary: Summary, scenarios: Scenario[]): string {
lines.push(`![${step.name}](${step.shot})`, ''); lines.push(`![${step.name}](${step.shot})`, '');
} }
} }
if (scenario.apiCalls.length > 0) {
lines.push(`**API calls (${scenario.apiCalls.length})**`, '');
for (const call of scenario.apiCalls) {
lines.push(`- \`${call.method} ${call.path}\` → **${call.status}** (${call.durationMs} ms)`);
if (call.requestBody) {
lines.push('', ' Request:', '', ' ```json', indent(call.requestBody), ' ```');
}
if (call.responseBody) {
lines.push('', ' Response:', '', ' ```json', indent(call.responseBody), ' ```');
}
}
lines.push('');
}
} }
} }
return lines.join('\n'); return lines.join('\n');
} }
/** Indents a block two spaces so it nests under a Markdown list item. */
function indent(text: string): string {
return text
.split('\n')
.map((line) => ' ' + line)
.join('\n');
}
+92 -5
View File
@@ -6,6 +6,7 @@
* ("2.3.1 failed"), and a print stylesheet so it can be saved as a PDF. * ("2.3.1 failed"), and a print stylesheet so it can be saved as a PDF.
*/ */
import { import {
ApiCall,
Area, Area,
RunMeta, RunMeta,
Scenario, Scenario,
@@ -108,6 +109,7 @@ ${STYLES}
<div><dt>Scenarios executed</dt><dd>${summary.total}</dd></div> <div><dt>Scenarios executed</dt><dd>${summary.total}</dd></div>
<div><dt>Duration</dt><dd>${friendly(summary.durationMs)}</dd></div> <div><dt>Duration</dt><dd>${friendly(summary.durationMs)}</dd></div>
<div><dt>Screenshots captured</dt><dd>${summary.screenshots}</dd></div> <div><dt>Screenshots captured</dt><dd>${summary.screenshots}</dd></div>
<div><dt>API calls captured</dt><dd>${summary.apiCalls}</dd></div>
<div><dt>Scenarios with recorded data</dt><dd>${summary.narrated}</dd></div> <div><dt>Scenarios with recorded data</dt><dd>${summary.narrated}</dd></div>
</dl> </dl>
</div> </div>
@@ -207,10 +209,11 @@ function renderArea(entry: AreaBlock): string {
function renderScenario(scenario: Scenario, ref: string): string { function renderScenario(scenario: Scenario, ref: string): string {
const hasFacts = scenario.steps.some((s) => s.facts); const hasFacts = scenario.steps.some((s) => s.facts);
const steps = const hasAnything = scenario.steps.length > 0 || scenario.apiCalls.length > 0;
scenario.steps.length === 0 const steps = scenario.steps.map(renderStep).join('');
? '<p class="no-evidence">No screenshot was captured for this scenario.</p>' const emptyNote = hasAnything
: scenario.steps.map(renderStep).join(''); ? ''
: '<p class="no-evidence">No evidence was captured for this scenario.</p>';
return ` return `
<details class="scenario" data-facts="${hasFacts ? '1' : '0'}"${hasFacts ? ' open' : ''}> <details class="scenario" data-facts="${hasFacts ? '1' : '0'}"${hasFacts ? ' open' : ''}>
@@ -222,6 +225,7 @@ function renderScenario(scenario: Scenario, ref: string): string {
</span> </span>
<span class="right"> <span class="right">
<span class="hide-narrow">${escapeHtml(scenario.project)}</span> <span class="hide-narrow">${escapeHtml(scenario.project)}</span>
<span class="hide-narrow">${scenario.apiCalls.length} API</span>
<span>${friendly(scenario.durationMs)}</span> <span>${friendly(scenario.durationMs)}</span>
<span class="tag ${statusClass(scenario.status)}">${statusWord(scenario.status)}</span> <span class="tag ${statusClass(scenario.status)}">${statusWord(scenario.status)}</span>
</span> </span>
@@ -229,10 +233,56 @@ function renderScenario(scenario: Scenario, ref: string): string {
<div class="body"> <div class="body">
${scenario.error ? `<pre class="error">${escapeHtml(scenario.error)}</pre>` : ''} ${scenario.error ? `<pre class="error">${escapeHtml(scenario.error)}</pre>` : ''}
${steps} ${steps}
${emptyNote}
${renderApiCalls(scenario.apiCalls)}
</div> </div>
</details>`; </details>`;
} }
/** The API traffic block: collapsed by default so it never dominates. */
function renderApiCalls(calls: ApiCall[]): string {
if (calls.length === 0) {
return '';
}
const rows = calls.map(renderApiCall).join('');
return `
<details class="api">
<summary>API calls &middot; ${calls.length} request${calls.length === 1 ? '' : 's'}</summary>
<div class="api-list">${rows}</div>
</details>`;
}
function renderApiCall(call: ApiCall): string {
const cls = call.status >= 500 ? 'c5' : call.status >= 400 ? 'c4' : call.status >= 300 ? 'c3' : 'c2';
const headerRows = Object.entries(call.requestHeaders)
.map(([k, v]) => `<tr><td>${escapeHtml(k)}</td><td>${escapeHtml(v)}</td></tr>`)
.join('');
const request = call.requestBody
? `<div class="part"><span class="part-label">Request payload</span><pre>${escapeHtml(call.requestBody)}</pre></div>`
: '';
const response = call.responseBody
? `<div class="part"><span class="part-label">Response body${call.truncated ? ' (truncated)' : ''}</span><pre>${escapeHtml(call.responseBody)}</pre></div>`
: '';
return `
<div class="call">
<div class="call-head">
<span class="method m-${escapeHtml(call.method.toLowerCase())}">${escapeHtml(call.method)}</span>
<span class="call-path">${escapeHtml(call.path)}</span>
<span class="status ${cls}">${call.status} ${escapeHtml(call.statusText)}</span>
<span class="call-time">${call.durationMs} ms</span>
</div>
<details class="headers"><summary>Request headers</summary><table>${headerRows}</table></details>
${request}
${response}
</div>`;
}
function renderStep(step: Step): string { function renderStep(step: Step): string {
const facts = step.facts const facts = step.facts
? `<table class="facts">${Object.entries(step.facts) ? `<table class="facts">${Object.entries(step.facts)
@@ -372,6 +422,41 @@ const STYLES = `
border-radius:6px;font-size:.83rem;white-space:pre-wrap; border-radius:6px;font-size:.83rem;white-space:pre-wrap;
font-family:ui-monospace,monospace} font-family:ui-monospace,monospace}
/* --- API traffic --- */
details.api{border-top:1px solid var(--line);margin-top:6px}
details.api > summary{list-style:none;cursor:pointer;padding:12px 0 10px;font-size:.78rem;
font-weight:650;letter-spacing:.04em;text-transform:uppercase;color:var(--muted)}
details.api > summary::-webkit-details-marker{display:none}
details.api > summary::before{content:'\\25B8 ';color:var(--faint)}
details.api[open] > summary::before{content:'\\25BE '}
.api-list{display:flex;flex-direction:column;gap:10px;padding-bottom:6px}
.call{border:1px solid var(--line);border-radius:7px;overflow:hidden;background:#fff}
.call-head{display:flex;align-items:center;gap:10px;padding:8px 11px;background:#f8fafc;
flex-wrap:wrap;font-size:.82rem}
.method{font-weight:700;font-size:.7rem;letter-spacing:.04em;padding:2px 7px;border-radius:4px;
background:#e5e7eb;color:#374151}
.method.m-get{background:#e0edff;color:#0b5cd6}
.method.m-post{background:#e6f4f1;color:#067a6e}
.method.m-patch,.method.m-put{background:#fef3c7;color:#b45309}
.method.m-delete{background:#fdeaee;color:#b3243f}
.call-path{font-family:ui-monospace,Menlo,monospace;font-size:.8rem;color:var(--ink);
word-break:break-all;flex:1;min-width:120px}
.status{font-weight:700;font-size:.76rem;font-variant-numeric:tabular-nums}
.status.c2{color:var(--ok)} .status.c3{color:var(--med)}
.status.c4{color:var(--fail)} .status.c5{color:#fff;background:var(--fail);padding:1px 6px;border-radius:4px}
.call-time{color:var(--faint);font-size:.76rem;font-variant-numeric:tabular-nums}
.part{padding:8px 11px;border-top:1px solid var(--line)}
.part-label{display:block;font-size:.68rem;letter-spacing:.06em;text-transform:uppercase;
color:var(--faint);font-weight:650;margin-bottom:5px}
.call pre{background:#0f172a;color:#e2e8f0;padding:10px 12px;border-radius:6px;overflow-x:auto;
font:12px/1.5 ui-monospace,Menlo,monospace;margin:0;max-height:340px}
details.headers{padding:8px 11px;border-top:1px solid var(--line)}
details.headers > summary{cursor:pointer;font-size:.72rem;color:var(--muted)}
details.headers table{border-collapse:collapse;font-size:.76rem;margin-top:6px}
details.headers td{border:1px solid var(--line);padding:3px 8px;
font-family:ui-monospace,Menlo,monospace;word-break:break-all}
details.headers td:first-child{color:var(--muted);background:#f8fafc}
.toolbar{display:flex;flex-wrap:wrap;gap:8px;align-items:center;margin-bottom:20px} .toolbar{display:flex;flex-wrap:wrap;gap:8px;align-items:center;margin-bottom:20px}
.toolbar button{border:1px solid var(--rule);background:#fff;border-radius:6px; .toolbar button{border:1px solid var(--rule);background:#fff;border-radius:6px;
padding:6px 13px;font:inherit;font-size:.85rem;cursor:pointer;color:var(--body)} padding:6px 13px;font:inherit;font-size:.85rem;cursor:pointer;color:var(--body)}
@@ -468,7 +553,9 @@ const SCRIPT = `
// first and put it back the way the reader had it afterwards. // first and put it back the way the reader had it afterwards.
let restore = []; let restore = [];
window.addEventListener('beforeprint', () => { window.addEventListener('beforeprint', () => {
restore = [...document.querySelectorAll('.scenario')].map((d) => [d, d.open]); // Open every details element (scenarios and the API/header blocks inside),
// since a collapsed one prints only its summary line.
restore = [...document.querySelectorAll('details')].map((d) => [d, d.open]);
for (const [d] of restore) { for (const [d] of restore) {
d.open = true; d.open = true;
} }
+15
View File
@@ -15,6 +15,19 @@ export interface Step {
facts?: Record<string, unknown>; facts?: Record<string, unknown>;
} }
/** One API request/response pair observed during a scenario. */
export interface ApiCall {
method: string;
path: string;
status: number;
statusText: string;
durationMs: number;
requestHeaders: Record<string, string>;
requestBody?: string;
responseBody?: string;
truncated?: boolean;
}
/** One test, as the report talks about it. */ /** One test, as the report talks about it. */
export interface Scenario { export interface Scenario {
project: string; project: string;
@@ -25,6 +38,7 @@ export interface Scenario {
expected: boolean; expected: boolean;
durationMs: number; durationMs: number;
steps: Step[]; steps: Step[];
apiCalls: ApiCall[];
error?: string; error?: string;
} }
@@ -45,6 +59,7 @@ export interface Summary {
skipped: number; skipped: number;
narrated: number; narrated: number;
screenshots: number; screenshots: number;
apiCalls: number;
} }
// --- areas ----------------------------------------------------------------- // --- areas -----------------------------------------------------------------
+135
View File
@@ -0,0 +1,135 @@
# TravelHub QA workflow — AI-assisted testing
This is how we define, automate, run and continuously gate tests on TravelHub.
It is the same loop for API, UI, backend and binding scenarios:
```
scenario card -> Claude generates the test -> run-tests.ps1 runs + logs + evidence -> gates CI/CD
(you) (AI, you review) (one runner, anywhere)
```
The scenario card is the single source of truth: a human reviews it, Claude
generates the test from it, and the evidence report traces each result back to
it. **No card, no test.**
---
## 1. Define a scenario
Copy [`scenario-template.yml`](scenario-template.yml) to
`qa/scenarios/<ID>.yml` and fill it in. Pick the layer — it decides the tool and
where the test lands:
| Layer | Proves | Lands in |
| ----- | ------ | -------- |
| `backend` | logic & data (tenant scoping, seat accounting) | `backend/**/*_test.go` |
| `api` | HTTP contract: status, schema, errors | Go handler tests / Playwright request |
| `ui` | a user journey in the browser | `e2e/tests/*.spec.ts` |
| `binding` | the UI↔API / API↔DB seam (payload sent + response rendered) | `e2e/tests/*.spec.ts` (with API capture) |
Worked examples to copy from:
[BOOK-012](scenarios/BOOK-012-no-oversell.yml) (binding),
[API-005](scenarios/API-005-cross-tenant-booking.yml) (api),
[SRCH-003](scenarios/SRCH-003-sort-by-fare.yml) (ui).
The card's `evidence:` list is a contract: the generated test must capture
exactly those screenshots and API calls, and the report must show them.
## 2. Automate it with Claude Code
From the repo root, use the prompt in
[`generate-test.prompt.md`](generate-test.prompt.md) with your card id. Claude
reads the card and the conventions in [`e2e/CLAUDE.md`](../e2e/CLAUDE.md) and
generates the spec using our page objects and fixtures.
Then **you review** — the diff *and* the evidence:
- the test uses page objects, not raw selectors;
- every `then` has an assertion on observable state;
- the evidence report shows each screenshot and API request/response the card
listed.
The report is the proof of work: the AI *claims* it tested the scenario; the
captured screenshots and API traffic *prove* it. Review the evidence, not just
the code. Full reviewer checklist is in the prompt file.
> Tip: `npx playwright codegen http://localhost:4200` records a rough draft by
> clicking; then ask Claude to refactor it into page-object style. You never
> write selectors by hand.
## 3. Run it — one ps1, logged, CI-ready
[`run-tests.ps1`](../run-tests.ps1) runs any layer, logs everything, and emits
machine-readable results. The **same script runs locally, in a git hook, and in
a pipeline**.
```powershell
.\run-tests.ps1 -Layer all # everything
.\run-tests.ps1 -Layer unit # backend + web, no browser (fast)
.\run-tests.ps1 -Layer ui -Grep "oversell" -Open # one scenario + open the report
.\run-tests.ps1 -Layer binding # UI<->API seam with traffic capture
```
What you get per run, under `.testlogs/<timestamp>/`:
- a log per layer (`backend.log`, `ui.log`, …) — full output, tee'd live;
- `junit.xml` — the result format Azure DevOps / Gitea publish;
- `summary.json` — pass/fail per layer;
- the **evidence report** (`e2e/evidence-report/`) — screenshots + the request,
payload and response of every API call, with credentials redacted.
It also **re-seeds the demo data if it has gone stale** (the seeded flights are
dated relative to now and expire after ~3 weeks), so you never chase a failure
that is really just old data.
Exit code is 0 only if every layer passed — that is what CI gates on.
## 4. Run continuously, before CI and CD
Five cadences, fastest to slowest — the same `run-tests.ps1` drives them all:
| When | What | Enforced? |
| ---- | ---- | --------- |
| while writing | watch mode (`ng test`, `playwright --ui`) | no |
| **pre-commit** (git hook) | secret scan | local |
| **pre-push** (git hook) | `run-tests.ps1 -Layer unit` | local |
| **pre-merge** (CI) | full suite + evidence artifact | **blocks merge** |
| nightly (scheduled) | full suite | alerts on drift |
Install the local hooks once after cloning:
```powershell
.\scripts\install-hooks.ps1
```
- **pre-commit** blocks committed secrets (it is what would have caught the dev
DB password leak). Fast, no toolchain needed.
- **pre-push** runs the unit layers so a logic break never leaves your machine.
"Before CI" is those two hooks (shift-left). "Before CD" is the pre-merge gate
plus a smoke run against staging before promote. The **nightly** run matters:
the stale-data failure above would surface at 2 a.m. instead of in front of a
person.
### CI / CD wiring
`run-tests.ps1` is the contract; the pipeline just calls it and publishes its
outputs (`junit.xml` + the evidence report). **Azure DevOps integration is
planned** — when we add it, the pipeline will run this same script and use
`PublishTestResults@2` on `junit.xml` plus `PublishBuildArtifacts` on the
evidence report. Nothing in steps 1–3 changes.
---
## The 15-minute demo
Walk one card end to end:
1. Open [BOOK-012](scenarios/BOOK-012-no-oversell.yml) — the scenario.
2. Prompt Claude Code with [the template](generate-test.prompt.md) → watch it
generate the Playwright spec using the page objects.
3. `.\run-tests.ps1 -Layer binding -Grep "oversell" -Open` → logs stream, the
evidence report opens showing the 409 and both bookings' request/response.
4. (Later) show the same ps1 running in the pipeline, blocking a bad merge.
That one loop is the whole system.
+45
View File
@@ -0,0 +1,45 @@
# Prompt template — generate a Playwright test from a scenario card
Paste this into Claude Code (from the repo root), filling in the card id.
Claude has the repo in context, so it will read the card and the conventions
itself.
---
Generate the automated test for scenario card `qa/scenarios/<ID>.yml`.
Follow `e2e/CLAUDE.md` exactly. In particular:
- Put the test in the spec file for the card's `area`; add to the existing
`test.describe` block, do not create a new spec.
- Drive the UI only through the page objects in `e2e/pages/`. If a needed action
or locator is missing, add a method to the right page object and use it.
- Import `test`/`expect` from `../fixtures/test`; take accounts from
`e2e/fixtures/accounts.ts`.
- For every clause under the card's `then:` and every item under `evidence:`,
add an `evidence.step(name, { ...observed data })` capturing a screenshot and
the figures the card asks for.
- No raw selectors in the spec, no `waitForTimeout`, no hardcoded secrets.
- If the test creates data (a booking), clean it up at the end.
When done:
1. Show me the diff (the spec, and any page-object change).
2. Run it: `.\run-tests.ps1 -Layer <layer> -Grep "<title fragment>" -Open`.
3. Confirm the evidence report shows each screenshot and the API
request/response listed in the card's `evidence:` section.
Do not loosen any assertion to make it pass. If the application behaves
differently from the card, stop and tell me what differs — that is a finding,
not something to code around.
---
## Reviewer checklist (the human reading the diff)
- [ ] Test lives in the right area spec; uses page objects, no raw selectors.
- [ ] Every `then` has a matching assertion on observable state.
- [ ] Evidence steps cover everything the card listed.
- [ ] No `waitForTimeout`, no hardcoded credentials, no weakened assertions.
- [ ] It actually ran green, and the evidence report proves the scenario.
- [ ] Data created by the test is cleaned up.
+49
View File
@@ -0,0 +1,49 @@
# TravelHub test scenario card — TEMPLATE
#
# One card = one scenario. Author this BEFORE any test code exists: it is the
# thing a human reviews, the thing Claude Code generates the test from, and the
# thing the evidence report traces each result back to.
#
# Rule: no card, no test. Copy this file to qa/scenarios/<ID>.yml and fill it in.
id: AREA-000 # stable id, e.g. BOOK-012, SRCH-003, AUTH-001
title: One sentence, present tense, describing the behaviour under test
# Which layer this scenario lives at. Picks the tool and where the test lands:
# backend Go logic/data (repository, middleware, config) -> backend/**/*_test.go
# api HTTP contract (status, schema, errors) -> Go handler tests / Playwright request
# ui a user journey through the browser -> e2e/tests/*.spec.ts
# binding the UI<->API or API<->DB seam (payload + render) -> e2e/tests/*.spec.ts (with API capture)
layer: ui
area: Booking and cancelling # must match an area in e2e/reporters/report-model.ts
risk: medium # low | medium | high -> drives how hard CI gates it
owner: qa # qa | dev — who maintains this card
# The world the test assumes before it starts.
preconditions:
- signed in as agent@skyline.test / skyline
- at least one bookable flight on JFK -> LHR
# Given / When / Then. Keep each a single clear clause.
steps:
given: the agent is on the flight search screen
when: they search JFK -> LHR and select the cheapest flight
then: the checkout screen shows that flight at the same fare
# What the evidence report must contain to prove this ran. The generated test
# should capture exactly these with evidence.step(...).
evidence:
- screenshot of the checkout screen
- the GET /api/flights request and response
- the fare carried from search into checkout (recorded data)
# Inputs the test uses. Keep data here, not hardcoded in the spec.
data:
origin: JFK
destination: LHR
passengers: 1
# Optional: what must NOT happen (negative assertions).
must_not:
- no booking is created (this scenario stops at checkout)
@@ -0,0 +1,27 @@
id: API-005
title: One agency cannot read another agency's booking by id
layer: api
area: Keeping agencies separate
risk: high
owner: qa
preconditions:
- a confirmed booking exists under the Globetrotter tenant
- valid credentials for the Skyline tenant
steps:
given: Skyline holds a valid booking id that belongs to Globetrotter
when: Skyline requests GET /api/bookings/{that-id}
then: the API responds 404 Not Found (not 403), leaking nothing about the record
evidence:
- the GET /api/bookings/{id} request (X-Tenant skyline) and the 404 response
- the same id fetched successfully as Globetrotter, for contrast (recorded data)
data:
owning_tenant: globetrotter
requesting_tenant: skyline
must_not:
- the response body must not contain the other tenant's passenger or contact data
- the status must not be 200 or 403 (404 keeps existence hidden)
+28
View File
@@ -0,0 +1,28 @@
id: BOOK-012
title: The last seat cannot be oversold to two agents at once
layer: binding
area: Booking and cancelling
risk: high
owner: qa
preconditions:
- signed in as agent@skyline.test / skyline
- a flight with exactly 1 seat remaining
steps:
given: a flight showing "1 left"
when: two bookings for that flight are confirmed at the same moment
then: one succeeds (201 with a PNR), the other is refused (409), and seats reach 0
evidence:
- the POST /api/bookings request and response for BOTH attempts
- the seat count before and after (recorded data)
- screenshot of the 409 "just sold" message on the losing attempt
data:
route: JFK -> LHR
passengers: 1
must_not:
- the flight must never show negative seats
- both bookings must never both succeed
+25
View File
@@ -0,0 +1,25 @@
id: SRCH-003
title: Flight results sort by lowest fare
layer: ui
area: Searching for flights
risk: low
owner: qa
preconditions:
- signed in as agent@skyline.test / skyline
- more than one flight on JFK -> LHR
steps:
given: the agent has searched JFK -> LHR
when: they choose "Lowest fare" from the sort control
then: the fares shown are in non-decreasing order down the page
evidence:
- screenshot of the sorted results
- the ordered list of fares read off the page (recorded data)
- the GET /api/flights?sort=price request and response
data:
origin: JFK
destination: LHR
sort: price
+252
View File
@@ -0,0 +1,252 @@
#requires -Version 5.1
<#
.SYNOPSIS
Run the TravelHub test suite by layer, log everything, and emit CI-ready
results and an evidence report. The same script runs locally, in a git
hook, and in a pipeline.
.DESCRIPTION
One runner for every test layer so there is a single source of truth for
"how tests run here". It:
- refreshes PATH from the registry (works in stale terminals)
- guards e2e against stale demo data by re-seeding when inventory is old
- writes a timestamped log per step under .testlogs\<run-id>\
- emits JUnit XML (e2e) that Azure DevOps / Gitea can publish
- regenerates the evidence report (screenshots + API request/response)
- returns a non-zero exit code if any layer fails, so CI can gate on it
Layers map to the scenario taxonomy in qa\:
backend Go unit + integration tests (repository, middleware, config)
api Go HTTP-contract tests (handlers, middleware)
web Angular unit tests
ui Playwright browser journeys (UI + binding, with API capture)
binding alias for ui -- the UI<->API seam, proven by captured traffic
unit backend + web (no browser; fast)
all backend + web + ui
.PARAMETER Layer
Which layer to run. Default: all.
.PARAMETER Grep
Only run tests whose title matches this (passed to `go test -run` and
`playwright -g`).
.PARAMETER Reseed
Force a database re-seed before running, instead of only when stale.
.PARAMETER NoReport
Skip regenerating the evidence report (faster; ui/binding only).
.PARAMETER Open
Open the evidence report in the browser when the run finishes.
.EXAMPLE
.\run-tests.ps1 -Layer all
.EXAMPLE
.\run-tests.ps1 -Layer ui -Grep "oversell" -Open
.EXAMPLE
.\run-tests.ps1 -Layer unit # fast pre-commit gate, no browser
#>
[CmdletBinding()]
param(
[ValidateSet('all', 'unit', 'backend', 'api', 'web', 'ui', 'binding')]
[string]$Layer = 'all',
[string]$Grep,
[switch]$Reseed,
[switch]$NoReport,
[switch]$Open
)
$ErrorActionPreference = 'Stop'
# --- configuration ---------------------------------------------------------
$Root = $PSScriptRoot
$Backend = Join-Path $Root 'backend'
$Frontend = Join-Path $Root 'frontend'
$E2E = Join-Path $Root 'e2e'
$RunId = Get-Date -Format 'yyyyMMdd-HHmmss'
$LogDir = Join-Path $Root ".testlogs\$RunId"
# --- console ---------------------------------------------------------------
function Write-Head($msg) { Write-Host "`n==== $msg ====" -ForegroundColor Cyan }
function Write-Ok($msg) { Write-Host " PASS $msg" -ForegroundColor Green }
function Write-Fail($msg) { Write-Host " FAIL $msg" -ForegroundColor Red }
function Write-Info($msg) { Write-Host " $msg" -ForegroundColor DarkGray }
# --- environment -----------------------------------------------------------
function Sync-Path {
$m = [Environment]::GetEnvironmentVariable('Path', 'Machine')
$u = [Environment]::GetEnvironmentVariable('Path', 'User')
$env:Path = ($m, $u | Where-Object { $_ }) -join ';'
}
function Resolve-Tool($name, [string[]]$fallbacks) {
$cmd = Get-Command $name -ErrorAction SilentlyContinue
if ($cmd) { return $cmd.Source }
foreach ($f in $fallbacks) { if (Test-Path $f) { return $f } }
throw "$name not found on PATH."
}
function Get-Go { Resolve-Tool 'go.exe' @('C:\Program Files\Go\bin\go.exe') }
function Get-Npm { Resolve-Tool 'npm.cmd' @('C:\Program Files\nodejs\npm.cmd') }
function Get-Npx { Resolve-Tool 'npx.cmd' @('C:\Program Files\nodejs\npx.cmd') }
function Get-Node { Resolve-Tool 'node.exe' @('C:\Program Files\nodejs\node.exe') }
# --- step runner -----------------------------------------------------------
$script:Results = @()
# Run one command, tee its output to a log file, and record pass/fail by exit
# code. Never throws -- a failing step is recorded and the run continues so the
# report covers every layer.
function Invoke-Step {
param(
[string]$Name,
[string]$WorkDir,
[string]$Exe,
[string[]]$CmdArgs,
[string]$LogName
)
Write-Head $Name
$log = Join-Path $LogDir $LogName
Write-Info "log: $log"
$sw = [Diagnostics.Stopwatch]::StartNew()
Push-Location $WorkDir
# A native tool writing to stderr (e.g. a webserver startup warning) must
# not be treated as a PowerShell error. Pass/fail is the exit code alone;
# stderr is just logged. Continue preference keeps 2>&1 from throwing.
$prev = $ErrorActionPreference
$ErrorActionPreference = 'Continue'
try {
& $Exe @CmdArgs 2>&1 | Tee-Object -FilePath $log
$code = $LASTEXITCODE
} finally {
$ErrorActionPreference = $prev
Pop-Location
}
$sw.Stop()
$ok = ($code -eq 0)
$script:Results += [pscustomobject]@{
Layer = $Name; Exit = $code; Ok = $ok; Seconds = [math]::Round($sw.Elapsed.TotalSeconds, 1); Log = $log
}
if ($ok) { Write-Ok "$Name ($($sw.Elapsed.TotalSeconds.ToString('0.0'))s)" }
else { Write-Fail "$Name (exit $code)" }
}
# --- preconditions ---------------------------------------------------------
function Ensure-Inventory {
# Only the browser layers need live, future-dated flight data.
$node = Get-Node
$check = Join-Path $Root 'scripts\db.mjs'
if ($Reseed) {
Write-Info 'Re-seeding database (-Reseed).'
& $node (Join-Path $Root 'scripts\db.mjs') reset | Out-Null
return
}
& $node $check check 2>&1 | Out-Null
if ($LASTEXITCODE -ne 0) {
Write-Info 'Demo inventory is stale or missing -- re-seeding before e2e.'
& $node (Join-Path $Root 'scripts\db.mjs') reset | Out-Null
} else {
Write-Info 'Demo inventory is fresh.'
}
}
# --- layers ----------------------------------------------------------------
function Run-Backend {
$go = Get-Go
$cmdArgs = @('test', './...')
if ($Grep) { $cmdArgs += @('-run', $Grep) }
Invoke-Step 'backend' $Backend $go $cmdArgs 'backend.log'
}
function Run-Api {
# The HTTP contract lives in the handler + middleware packages.
$go = Get-Go
$cmdArgs = @('test', './internal/handlers/...', './internal/middleware/...')
if ($Grep) { $cmdArgs += @('-run', $Grep) }
Invoke-Step 'api' $Backend $go $cmdArgs 'api.log'
}
function Run-Web {
$npx = Get-Npx
Invoke-Step 'web' $Frontend $npx @('ng', 'test', '--no-watch') 'web.log'
}
function Run-Ui {
param([string]$Label = 'ui')
Ensure-Inventory
$npx = Get-Npx
$cmdArgs = @('playwright', 'test')
if ($Grep) { $cmdArgs += @('-g', $Grep) }
Invoke-Step $Label $E2E $npx $cmdArgs "$Label.log"
}
# --- evidence + results collection -----------------------------------------
function Collect-Artifacts {
# JUnit (for CI test publishing) and the evidence report both live under e2e.
$junit = Join-Path $E2E 'results\junit.xml'
if (Test-Path $junit) {
Copy-Item $junit (Join-Path $LogDir 'junit.xml') -Force
Write-Info "JUnit results -> $(Join-Path $LogDir 'junit.xml')"
}
$evidence = Join-Path $E2E 'evidence-report\index.html'
if (Test-Path $evidence) {
Write-Info "Evidence report -> $evidence"
if ($Open) { Start-Process $evidence }
}
}
# --- dispatch --------------------------------------------------------------
Sync-Path
New-Item -ItemType Directory -Force -Path $LogDir | Out-Null
Write-Host "TravelHub test run $RunId (layer: $Layer)" -ForegroundColor White
switch ($Layer) {
'backend' { Run-Backend }
'api' { Run-Api }
'web' { Run-Web }
'ui' { Run-Ui 'ui' }
'binding' { Run-Ui 'binding' }
'unit' { Run-Backend; Run-Web }
'all' { Run-Backend; Run-Web; Run-Ui 'ui' }
}
if (-not $NoReport) { Collect-Artifacts }
# --- summary ---------------------------------------------------------------
Write-Head 'Summary'
$script:Results | Format-Table Layer, @{N='Result';E={ if ($_.Ok) { 'PASS' } else { "FAIL ($($_.Exit))" } }}, Seconds -AutoSize | Out-String | Write-Host
# Write a small machine-readable summary alongside the logs.
$summary = [pscustomobject]@{
runId = $RunId
layer = $Layer
when = (Get-Date).ToString('o')
results = $script:Results
passed = @($script:Results | Where-Object Ok).Count
failed = @($script:Results | Where-Object { -not $_.Ok }).Count
}
$summary | ConvertTo-Json -Depth 4 | Out-File (Join-Path $LogDir 'summary.json') -Encoding utf8
Write-Info "Logs: $LogDir"
$failed = @($script:Results | Where-Object { -not $_.Ok }).Count
if ($failed -gt 0) {
Write-Fail "$failed layer(s) failed."
exit 1
}
Write-Ok 'All layers passed.'
exit 0
+22
View File
@@ -173,6 +173,28 @@ const commands = {
createDatabase(CONFIG.testDatabase); createDatabase(CONFIG.testDatabase);
console.log(`✓ ${CONFIG.testDatabase} ready for go test ./...`); console.log(`✓ ${CONFIG.testDatabase} ready for go test ./...`);
}, },
// Freshness guard for the e2e suite. The seed generates flights relative to
// now(), so after ~3 weeks all seeded departures are in the past and the
// browser tests fail on empty search results. Exit non-zero when stale so a
// runner can re-seed before running.
check() {
if (!databaseExists(CONFIG.database)) {
console.error(`✗ ${CONFIG.database} does not exist`);
process.exit(2);
}
const out = run(
['-tAc', 'SELECT count(*) FROM flights WHERE departure_time > now()'],
{ database: CONFIG.database, capture: true },
);
const future = Number((out || '').trim());
if (Number.isFinite(future) && future > 0) {
console.log(`✓ inventory fresh: ${future} future flights`);
process.exit(0);
}
console.error('✗ inventory stale: no future flights — re-seed needed (npm run db:reset)');
process.exit(1);
},
}; };
const command = process.argv[2]; const command = process.argv[2];
+17
View File
@@ -0,0 +1,17 @@
#requires -Version 5.1
# Point git at the versioned hooks in .githooks so every developer gets the
# same pre-commit / pre-push gates. Run once after cloning.
$ErrorActionPreference = 'Stop'
$root = Split-Path $PSScriptRoot -Parent
Push-Location $root
try {
git config core.hooksPath .githooks
# Git for Windows runs hooks via its bundled bash; the files are sh scripts.
Write-Host 'Git hooks installed (core.hooksPath -> .githooks):' -ForegroundColor Green
Write-Host ' pre-commit : blocks committed secrets'
Write-Host ' pre-push : runs backend + web unit tests'
Write-Host ''
Write-Host 'Bypass in an emergency with --no-verify (discouraged).'
} finally {
Pop-Location
}