Add AI-assisted QA test workflow: scenario cards, runner, hooks, API capture

A complete loop for defining, generating, running and gating tests:

- qa/: scenario-card template + examples (API/UI/binding), the Claude
  generation prompt, and a QA runbook tying it together
- e2e/CLAUDE.md: conventions that keep AI-generated tests consistent
- run-tests.ps1: one runner for every layer (backend/api/web/ui/binding),
  logs per run under .testlogs/, JUnit output, evidence report, and a
  re-seed guard against stale demo data
- .githooks/ + scripts/install-hooks.ps1: pre-commit secret scan and
  pre-push unit tests (shift-left before CI)
- evidence report now captures each API call's request, payload and
  response (Authorization redacted); JUnit reporter added for CI
- scripts/db.mjs: db:check freshness guard

Azure DevOps pipeline deferred; the runner already emits JUnit + the
evidence report for any CI to publish.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
nilesh.p
2026-10-10 13:04:46 +05:30
co-authored by Claude Opus 4.8
parent df9e534e42
commit 45b5b34680
20 changed files with 1054 additions and 11 deletions
+49
View File
@@ -0,0 +1,49 @@
# TravelHub test scenario card — TEMPLATE
#
# One card = one scenario. Author this BEFORE any test code exists: it is the
# thing a human reviews, the thing Claude Code generates the test from, and the
# thing the evidence report traces each result back to.
#
# Rule: no card, no test. Copy this file to qa/scenarios/<ID>.yml and fill it in.
id: AREA-000 # stable id, e.g. BOOK-012, SRCH-003, AUTH-001
title: One sentence, present tense, describing the behaviour under test
# Which layer this scenario lives at. Picks the tool and where the test lands:
# backend Go logic/data (repository, middleware, config) -> backend/**/*_test.go
# api HTTP contract (status, schema, errors) -> Go handler tests / Playwright request
# ui a user journey through the browser -> e2e/tests/*.spec.ts
# binding the UI<->API or API<->DB seam (payload + render) -> e2e/tests/*.spec.ts (with API capture)
layer: ui
area: Booking and cancelling # must match an area in e2e/reporters/report-model.ts
risk: medium # low | medium | high -> drives how hard CI gates it
owner: qa # qa | dev — who maintains this card
# The world the test assumes before it starts.
preconditions:
- signed in as agent@skyline.test / skyline
- at least one bookable flight on JFK -> LHR
# Given / When / Then. Keep each a single clear clause.
steps:
given: the agent is on the flight search screen
when: they search JFK -> LHR and select the cheapest flight
then: the checkout screen shows that flight at the same fare
# What the evidence report must contain to prove this ran. The generated test
# should capture exactly these with evidence.step(...).
evidence:
- screenshot of the checkout screen
- the GET /api/flights request and response
- the fare carried from search into checkout (recorded data)
# Inputs the test uses. Keep data here, not hardcoded in the spec.
data:
origin: JFK
destination: LHR
passengers: 1
# Optional: what must NOT happen (negative assertions).
must_not:
- no booking is created (this scenario stops at checkout)