nilesh.pandClaude Opus 4.8
45b5b34680
A complete loop for defining, generating, running and gating tests: - qa/: scenario-card template + examples (API/UI/binding), the Claude generation prompt, and a QA runbook tying it together - e2e/CLAUDE.md: conventions that keep AI-generated tests consistent - run-tests.ps1: one runner for every layer (backend/api/web/ui/binding), logs per run under .testlogs/, JUnit output, evidence report, and a re-seed guard against stale demo data - .githooks/ + scripts/install-hooks.ps1: pre-commit secret scan and pre-push unit tests (shift-left before CI) - evidence report now captures each API call's request, payload and response (Authorization redacted); JUnit reporter added for CI - scripts/db.mjs: db:check freshness guard Azure DevOps pipeline deferred; the runner already emits JUnit + the evidence report for any CI to publish. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
50 lines
2.1 KiB
YAML
50 lines
2.1 KiB
YAML
# TravelHub test scenario card — TEMPLATE
|
|
#
|
|
# One card = one scenario. Author this BEFORE any test code exists: it is the
|
|
# thing a human reviews, the thing Claude Code generates the test from, and the
|
|
# thing the evidence report traces each result back to.
|
|
#
|
|
# Rule: no card, no test. Copy this file to qa/scenarios/<ID>.yml and fill it in.
|
|
|
|
id: AREA-000 # stable id, e.g. BOOK-012, SRCH-003, AUTH-001
|
|
title: One sentence, present tense, describing the behaviour under test
|
|
|
|
# Which layer this scenario lives at. Picks the tool and where the test lands:
|
|
# backend Go logic/data (repository, middleware, config) -> backend/**/*_test.go
|
|
# api HTTP contract (status, schema, errors) -> Go handler tests / Playwright request
|
|
# ui a user journey through the browser -> e2e/tests/*.spec.ts
|
|
# binding the UI<->API or API<->DB seam (payload + render) -> e2e/tests/*.spec.ts (with API capture)
|
|
layer: ui
|
|
|
|
area: Booking and cancelling # must match an area in e2e/reporters/report-model.ts
|
|
risk: medium # low | medium | high -> drives how hard CI gates it
|
|
owner: qa # qa | dev — who maintains this card
|
|
|
|
# The world the test assumes before it starts.
|
|
preconditions:
|
|
- signed in as agent@skyline.test / skyline
|
|
- at least one bookable flight on JFK -> LHR
|
|
|
|
# Given / When / Then. Keep each a single clear clause.
|
|
steps:
|
|
given: the agent is on the flight search screen
|
|
when: they search JFK -> LHR and select the cheapest flight
|
|
then: the checkout screen shows that flight at the same fare
|
|
|
|
# What the evidence report must contain to prove this ran. The generated test
|
|
# should capture exactly these with evidence.step(...).
|
|
evidence:
|
|
- screenshot of the checkout screen
|
|
- the GET /api/flights request and response
|
|
- the fare carried from search into checkout (recorded data)
|
|
|
|
# Inputs the test uses. Keep data here, not hardcoded in the spec.
|
|
data:
|
|
origin: JFK
|
|
destination: LHR
|
|
passengers: 1
|
|
|
|
# Optional: what must NOT happen (negative assertions).
|
|
must_not:
|
|
- no booking is created (this scenario stops at checkout)
|