07 — Quality and Verification

Submodules: accessibility-gate (axe-core v4.11, WCAG 2.2 AA, focus-not-obscured, target-size 24px, accessible-auth)|performance-optimization (CWV, INP 3-phase, budgets)|security-hardening (CSP nonce-based, OWASP 2025, supply-chain #3)|computer-use-automation (native macOS)|chrome-and-browser-workflows (Chrome/Playwright MCP)|completeness-verification (multi-pass AI visual)|visual-inspection-loop (screenshot/critique/fix)|tdd-verification (10-point journey)|testing-matrices (payment/email/form)|adversarial-testing (chaos/stress)|stagehand-ai-testing (AI browser fallback, a11y-tree selectors)|visual-regression (Percy AI+Chromatic+pixelmatch)|contract-testing (Zod vs live APIs)|slop-detection (AI filler scanner)|eval-driven-development (LLM-as-judge)|ui-completeness-sweep (MANDATORY Playwright+GPT-4o, blocks done until >=8/10)|semgrep-codebase-rules (AST-level per-project rules)|e2e-accumulation (append-only parallel chunks)|evidence-collection (R2 video+screenshots).

Verification Pyramid

L5: Post-Deploy (08). L4: Visual (screenshots+AI). L3: E2E (Playwright v1.59+). L2: Integration (API/DB). L1: Static (lint/typecheck). Every code change: L1-L3. Every deploy: L4-L5.

Playwright AI Agents (v1.59.1+)

Planner: explores app, designs test plans from natural language. Generator: creates executable test code. Healer: auto-fixes broken tests. Pattern: static specs for stable tests, AI agents for flaky/new. Run agents only on failed tests in second pass (70% token savings).

Playwright v1.59.1 New APIs

page.screencast({ path }) — video recording with action annotations and real-time frame capture (video receipts for CI). browser.bind() — connect to running browser instances. page.consoleMessages()/page.pageErrors()/page.requests() — snapshot-in-time accessors (no event listeners needed). await using async disposables for auto-cleanup. Trace CLI for agent-driven test analysis.

MCP-Based Testing

Playwright MCP operates on accessibility tree, not screenshots. Returns structured snapshots: role hierarchy, names, states. Target "Role: button, Name: Checkout" — 10x more stable than CSS selectors. AOM-reasoning > DOM-scraping.

Multi-Agent Testing Pattern

Functional agent: happy path clicks. Security agent: XSS probing, auth bypass. Accessibility agent: WCAG 2.2 compliance. Performance agent: CWV measurement. Run all four in parallel per deploy.

Self-Healing Loop

Test fails→read full error→classify (code/test/env/flaky)→fix root cause→re-run→loop max 5. NEVER .skip/.only/.fixme()/comment assertions/increase timeouts. After fix: full suite.

Zero Console Errors

const errors: string[] = [];
page.on('console', msg => { if (msg.type() === 'error') errors.push(msg.text()); });
expect(errors).toEqual([]);

Zero Recommendations Gate

After all tests pass: "How improve?" If ANY recommendation→implement+retest. Done when AI has zero suggestions.

L1: Static

TypeScript (tsc --noEmit), ESLint flat config, Prettier. Fix all. Suppress lines only with comment.

L2: Unit/Integration (Vitest)

100% on new functions. Happy+error+edge+auth paths. Don't test: framework internals, 3rd-party, CSS, simple getters.

L3: E2E (Playwright v1.59+)

Homepage-first. No sleeps (waitFor, toBeVisible, waitForResponse). Stable selectors: data-testid→role→text→Stagehand AI fallback. Parallel-safe, deterministic, production URLs. 6bp: 375/390/768/1024/1280/1920.

L4: Visual

Percy AI Visual Review Agent (3x review reduction, 40% false positive filtering, OCR text-shift elimination). Chromatic for component-level (Storybook+Playwright). pixelmatch for local CI. See visual-inspection-loop.md + completeness-verification.md.

8-Check Quality Gate

#	Check	Pass
1	E2E	0 failures
2	Visual	No breaks 1280+375px, Percy/Chromatic clean
3	Links	All 200
4	SEO	JSON-LD 4+, OG 1200x630, sitemap
5	Performance	LCP<2.5s, INP<200ms, CLS<0.1
6	A11y	WCAG 2.2 AA, axe-core 0, focus-not-obscured, target>=24px
7	Security	CSP nonce-based, OWASP 2025, no secrets, Zod
8	Web Property	Manifest, shortcuts, 4+ JSON-LD, cross-site links

WCAG & ADA Landscape

WCAG 2.2 AA = baseline (9 new SC: focus-not-obscured, target-size-minimum 24px, accessible-auth, consistent-help, redundant-entry, dragging-movements, focus-appearance 2px/3:1). ADA Title II: large entities April 2027, smaller April 2028. WCAG 3.0 working draft (174 requirements, est. 2028-2030, no A/AA/AAA — assertions+scoring). axe-core v4.11.3 covers WCAG 2.0/2.1/2.2 at A/AA/AAA.

GitHub Integration

PR checks: gh pr checks→gh run view --log-failed→diagnose→fix→push→verify.

CWV Targets

LCP <2.5s (4-phase: TTFB→resource delay→load duration→render delay). INP <200ms (3-phase: input delay→processing→presentation). CLS <0.1 (explicit dimensions, aspect-ratio, font-display). 47% of sites pass all three. See performance-optimization.md.

Quick Check (small fixes)

tsc --noEmit, relevant E2E, visual 1280+375, no console errors, deploy+purge+verify 200.

Test Shortcuts

New page: adapted homepage test. New form: 8-point matrix. New API: valid 200, invalid 400, missing 404. Visual: screenshot at 6bp. Auth: unauth 401, wrong 403, valid 200.

Shared values (breakpoints, CSP): CONVENTIONS.md.