Add AI-assisted QA test workflow: scenario cards, runner, hooks, API capture

A complete loop for defining, generating, running and gating tests:

- qa/: scenario-card template + examples (API/UI/binding), the Claude
  generation prompt, and a QA runbook tying it together
- e2e/CLAUDE.md: conventions that keep AI-generated tests consistent
- run-tests.ps1: one runner for every layer (backend/api/web/ui/binding),
  logs per run under .testlogs/, JUnit output, evidence report, and a
  re-seed guard against stale demo data
- .githooks/ + scripts/install-hooks.ps1: pre-commit secret scan and
  pre-push unit tests (shift-left before CI)
- evidence report now captures each API call's request, payload and
  response (Authorization redacted); JUnit reporter added for CI
- scripts/db.mjs: db:check freshness guard

Azure DevOps pipeline deferred; the runner already emits JUnit + the
evidence report for any CI to publish.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
nilesh.p
2026-10-10 13:04:46 +05:30
co-authored by Claude Opus 4.8
parent df9e534e42
commit 45b5b34680
20 changed files with 1054 additions and 11 deletions
@@ -0,0 +1,27 @@
id: API-005
title: One agency cannot read another agency's booking by id
layer: api
area: Keeping agencies separate
risk: high
owner: qa
preconditions:
- a confirmed booking exists under the Globetrotter tenant
- valid credentials for the Skyline tenant
steps:
given: Skyline holds a valid booking id that belongs to Globetrotter
when: Skyline requests GET /api/bookings/{that-id}
then: the API responds 404 Not Found (not 403), leaking nothing about the record
evidence:
- the GET /api/bookings/{id} request (X-Tenant skyline) and the 404 response
- the same id fetched successfully as Globetrotter, for contrast (recorded data)
data:
owning_tenant: globetrotter
requesting_tenant: skyline
must_not:
- the response body must not contain the other tenant's passenger or contact data
- the status must not be 200 or 403 (404 keeps existence hidden)
+28
View File
@@ -0,0 +1,28 @@
id: BOOK-012
title: The last seat cannot be oversold to two agents at once
layer: binding
area: Booking and cancelling
risk: high
owner: qa
preconditions:
- signed in as agent@skyline.test / skyline
- a flight with exactly 1 seat remaining
steps:
given: a flight showing "1 left"
when: two bookings for that flight are confirmed at the same moment
then: one succeeds (201 with a PNR), the other is refused (409), and seats reach 0
evidence:
- the POST /api/bookings request and response for BOTH attempts
- the seat count before and after (recorded data)
- screenshot of the 409 "just sold" message on the losing attempt
data:
route: JFK -> LHR
passengers: 1
must_not:
- the flight must never show negative seats
- both bookings must never both succeed
+25
View File
@@ -0,0 +1,25 @@
id: SRCH-003
title: Flight results sort by lowest fare
layer: ui
area: Searching for flights
risk: low
owner: qa
preconditions:
- signed in as agent@skyline.test / skyline
- more than one flight on JFK -> LHR
steps:
given: the agent has searched JFK -> LHR
when: they choose "Lowest fare" from the sort control
then: the fares shown are in non-decreasing order down the page
evidence:
- screenshot of the sorted results
- the ordered list of fares read off the page (recorded data)
- the GET /api/flights?sort=price request and response
data:
origin: JFK
destination: LHR
sort: price