Proactive Audit

Overview

Automated health checks for framework artifacts that were modified during the current pipeline. This skill fills the gap between reactive verification (tests, lint) and proactive framework-level validation (wiring, syntax, security patterns).

Core principle: Framework artifact changes require the same rigor as code changes. If a skill was created, verify it is wired. If a hook was modified, verify it compiles. If an agent was changed, verify its tool/skill lists are consistent.

When to Invoke

Invoke this skill as the final pipeline step whenever ANY of the following paths were created, modified, or deleted during the session:

• .claude/hooks/**/*.cjs
• .claude/skills/**/SKILL.md
• .claude/agents/**/*.md
• .claude/workflows/**/*.md
• .claude/schemas/**/*.json
• .claude/templates/**/*
• .claude/CLAUDE.md
• .claude/lib/routing/routing-table.cjs

Invocation:

Skill({ skill: 'proactive-audit' });

Mandatory Skills

Skill	Purpose	When
task-management-protocol	Track audit progress	Always
ripgrep	Fast targeted artifact search	During checks
code-semantic-search	Pattern discovery across artifacts	When investigating
context-compressor	Compress large audit results	When output is large
verification-before-completion	Gate completion on zero CRITICAL	Before marking done
memory-search	Check prior audit patterns	At start

Step 1: Detect Changed Artifacts

Use git diff to identify which framework artifacts changed in this session:

# Primary: git diff against recent commits
git diff --name-only HEAD~5 -- .claude/hooks/ .claude/skills/ .claude/agents/ .claude/workflows/ .claude/schemas/ .claude/templates/ .claude/CLAUDE.md .claude/lib/routing/

# Secondary: check unstaged changes
git diff --name-only -- .claude/hooks/ .claude/skills/ .claude/agents/ .claude/workflows/ .claude/schemas/ .claude/templates/

# Tertiary: check untracked files
git ls-files --others --exclude-standard .claude/hooks/ .claude/skills/ .claude/agents/ .claude/workflows/ .claude/schemas/ .claude/templates/

Combine all three lists into a deduplicated set of changed artifact paths.

Step 2: Apply Check Matrix

For each changed artifact, apply the relevant checks from this matrix:

Hook Files (.claude/hooks/**/*.cjs)

Check ID	Check	Command	Severity
H-01	Syntax validity	node --check <file>	CRITICAL
H-02	SE-02: raw JSON.parse without safeParseJSON	grep -n "JSON.parse(" <file> then verify safeParseJSON import	HIGH
H-03	SE-01: shell injection via shell: true	grep -n "shell:\\s*true" <file>	HIGH
H-04	Hook registered in settings.json	grep "<hook-filename>" .claude/settings.json	MEDIUM
H-05	Exit code correctness	Verify try/catch wrapping, exit 0 on non-critical errors	MEDIUM

H-02 detail: If JSON.parse( is found, check if the file also imports safeParseJSON from .claude/lib/utils/safe-json.cjs. If not, flag as HIGH finding. Exclude test files (*.test.cjs).

Skill Files (.claude/skills/**/SKILL.md)

Check ID	Check	Command	Severity
S-01	Skill appears in skill-catalog.md	grep "<skill-name>" .claude/docs/skill-catalog.md	HIGH
S-02	At least one agent has skill in frontmatter	grep -r "<skill-name>" .claude/agents/ --include="*.md"	MEDIUM
S-03	Skill appears in CLAUDE.md Section 8.5	grep "<skill-name>" .claude/CLAUDE.md	MEDIUM
S-04	SKILL.md has valid frontmatter	Verify name:, description:, version: fields exist	MEDIUM
S-05	Validate skills (if available)	pnpm validate:skills 2>&1	LOW

Agent Files (.claude/agents/**/*.md)

Check ID	Check	Command	Severity
A-01	Agent appears in agent-registry.json	grep "<agent-name>" .claude/context/agent-registry.json	HIGH
A-02	Agent's skills: list references existing skills	For each skill in frontmatter, verify .claude/skills/<skill>/SKILL.md exists	MEDIUM
A-03	Agent's tools: list contains only valid tools	Verify each tool name against known tool list	MEDIUM
A-04	Agent appears in CLAUDE.md routing table	grep "<agent-name>" .claude/CLAUDE.md	MEDIUM

Workflow Files (.claude/workflows/**/*.md)

Check ID	Check	Command	Severity
W-01	Workflow referenced in WORKFLOW_AGENT_MAP.md	grep "<workflow-name>" .claude/docs/@WORKFLOW_AGENT_MAP.md	MEDIUM
W-02	Referenced agents exist	For each agent name in workflow, verify agent file exists	MEDIUM

Schema Files (.claude/schemas/**/*.json)

Check ID	Check	Command	Severity
SC-01	Valid JSON syntax	node -e "JSON.parse(require('fs').readFileSync('<file>', 'utf8'))"	CRITICAL
SC-02	Schema appears in schema-catalog.md	grep "<schema-name>" .claude/context/artifacts/catalogs/schema-catalog.md	MEDIUM

Root Cleanliness Check

ls -1 | grep -cvE '^(\.|node_modules|src|tests|scripts|dist|build|docs|package\.json|package-lock\.json|pnpm-lock\.yaml|tsconfig|eslint|prettier|jest|vitest|README|LICENSE|CHANGELOG|CLAUDE\.md|\.env)'

FAIL if the count is greater than 0.

Known slop patterns (any of these in project root = FAIL):

• *-debug*.txt, *-debug*.log, debug-*.json
• dump-*.cjs, dump-*.js, dump-*.json
• rename_*.cjs, revert_*.cjs, update_*.cjs
• test-out.txt, lint-output.txt, eslint.json, errors.json
• UUID-named files (e.g. a3f2c1b0-*.json)
• new_session_analysis.md or any *.analysis.md not under .claude/context/
• Any .cjs/.js/.mjs not referenced in package.json scripts or tracked project source
• Any .md not named README.md, CLAUDE.md, LICENSE, or CHANGELOG.md

Action when FAIL:

1. Delete the offending files (or move to .claude/context/tmp/ if content may be needed)
2. Log deletion to session-gap-log.jsonl with type: "cleanup"
3. Append a reflection-spawn-request.json entry with trigger: "ai-slop-found" so the root cause is investigated

Reference: .claude/rules/cleanup-always.md

Documentation Staleness Check

After any feature work, verify:

1. CHANGELOG.md — does it have an entry dated within the last session?

   • Check: git log --oneline -5 vs grep "## \[" CHANGELOG.md | head -3
   • FAIL if: feature commits exist but no matching CHANGELOG entry

2. .env.example — does it document all env vars in the codebase?

   • Check: grep -r "process.env\." .claude/skills/ .claude/hooks/ | grep -oP "process\.env\.\K\w+" | sort -u
   • Compare against entries in .env.example
   • FAIL if: env var used in code but not documented in .env.example

3. README.md — are agent/skill counts current?

   • Check counts in README vs jq '.agents | length' .claude/context/agent-registry.json
   • WARN if counts diverge by more than 2

Routing Files (.claude/lib/routing/routing-table.cjs, .claude/CLAUDE.md)

Check ID	Check	Command	Severity
R-01	routing-table.cjs syntax	node --check .claude/lib/routing/routing-table.cjs	CRITICAL
R-02	Validate skills (full)	pnpm validate:skills 2>&1	MEDIUM

Step 3: Generate Report

Write a structured report to .claude/context/reports/ecosystem-audit/proactive-audit-{ISO-date}.md with this format:

<!-- Agent: qa | Task: #N | Session: YYYY-MM-DD -->

# Proactive Audit Report

**Date:** YYYY-MM-DD
**Artifacts Scanned:** N
**Findings:** N CRITICAL, N HIGH, N MEDIUM, N LOW
**Overall:** PASS | FAIL

## Changed Artifacts

- path/to/artifact1 (type: hook)
- path/to/artifact2 (type: skill)

## Findings

### CRITICAL

| ID   | File          | Check  | Detail                 | Remediation      |
| ---- | ------------- | ------ | ---------------------- | ---------------- |
| H-01 | hooks/foo.cjs | Syntax | SyntaxError at line 42 | Fix syntax error |

### HIGH

| ID   | File          | Check | Detail                                      | Remediation                                               |
| ---- | ------------- | ----- | ------------------------------------------- | --------------------------------------------------------- |
| H-02 | hooks/bar.cjs | SE-02 | JSON.parse at line 15 without safeParseJSON | Import safeParseJSON from .claude/lib/utils/safe-json.cjs |

### MEDIUM

(same table format)

### PASS

| ID   | File          | Check  | Result |
| ---- | ------------- | ------ | ------ |
| H-01 | hooks/baz.cjs | Syntax | OK     |

## Summary

- Total checks run: N
- Passed: N
- Failed: N
- Pass rate: N%

Step 4: Return Verdict

After generating the report:

• If ANY CRITICAL findings exist: return FAIL with the report path and list of critical findings
• If ANY HIGH findings exist: return WARN with the report path and list of high findings
• If only MEDIUM/LOW findings: return PASS with the report path and note about medium findings
• If no findings: return PASS with the report path

Iron Laws

1. ALWAYS run every applicable check from the check matrix — skipping "small" changes is how undetected wiring failures accumulate across sessions.
2. NEVER trust task metadata alone for change detection — use git diff as the primary source of changed artifact paths.
3. NEVER report PASS without actually executing each check command — self-attested PASS without evidence violates verification-before-completion.
4. NEVER ignore SE-02 (prototype pollution) findings in hook files — a single compromised hook can corrupt all subsequent tool calls in the pipeline.
5. ALWAYS validate hook syntax with node --check before reporting findings — broken hooks silently block the entire tool pipeline.

Anti-Patterns

Anti-Pattern	Why It Fails	Correct Approach
Skipping checks for "small" changes	Small wiring failures accumulate silently until a pipeline breaks	Run all checks regardless of perceived change size
Trusting task metadata for change detection	Metadata can be incomplete or stale; misses unstaged changes	Use git diff --name-only + git ls-files --others as primary source
Self-attesting PASS without running commands	Unverified PASS masks real failures; violates verification-before-completion	Execute every check command and capture output as evidence
Ignoring SE-02 (prototype pollution) in hooks	One polluted hook corrupts Object.prototype globally across all tool calls	Flag SE-02 as HIGH severity and block pipeline until fixed
Reporting findings without remediation steps	Developers know what broke but not how to fix it	Include specific remediation for every finding with file+line reference

Severity Guide

Severity	Meaning	Action Required
CRITICAL	Framework will break	Fix immediately, block pipeline completion
HIGH	Security risk or invisible artifact	Fix before next session, warn user
MEDIUM	Missing integration, incomplete wiring	Fix in follow-up task
LOW	Best practice violation, cosmetic	Track for future improvement

Integration with Router Step 0.7

The router invokes this skill via Step 0.7 in the Router Output Contract (CLAUDE.md Section 0.1). The router:

1. Detects that framework artifacts were modified during the pipeline
2. Spawns a QA agent with this skill as the final pipeline step
3. Reads the audit report
4. If CRITICAL findings: spawns developer to fix them before claiming completion
5. If HIGH findings: warns user and notes findings in pipeline summary
6. If PASS: proceeds to claim pipeline completion

Related Skills

• verification-before-completion -- General evidence-based completion gates
• checklist-generator -- IEEE 1028 quality checklists
• sharp-edges -- Known hazard patterns (SE-01 through SE-07)

Related References

• .claude/context/plans/proactive-audit-design-2026-02-22.md -- Design document
• .claude/rules/security.md -- SE-01 and SE-02 patterns
• .claude/rules/artifact-integration.md -- Must-have integration requirements