Ecosystem Integrity Scanner

Overview

Automated structural integrity scanner for the entire Agent Studio framework. Catches issues that silently break agent workflows: wrong require() paths that crash at runtime, phantom skills referenced in agent frontmatter that don't exist on disk, stale agent registry counts, archived references still active in production code, UTF-16 encoding that breaks parsers, and bloated agent configs.

Core principle: Framework structural issues are pre-production bugs. A phantom skill reference means an agent will fail silently. A broken require() means a hook crashes at the wrong moment. The scanner enforces zero-tolerance for these.

When to Invoke

Skill({ skill: 'ecosystem-integrity-scanner' });

Invoke proactively after:

• Bulk framework changes (batch skill/agent/hook creation or archiving)
• Refactoring module paths or directory structure
• Running a major audit or cleanup pass
• Suspecting phantom references or broken require() chains
• Before finalizing any CI sanity gate

Mandatory Skills

Invoke before starting this skill:

Skill	Purpose	When
task-management-protocol	Track scan progress	Always
ripgrep	Fast targeted code search	When investigating
code-semantic-search	Concept discovery	When pattern hunting
context-compressor	Compress large audit results	When output is large
verification-before-completion	Gate completion on zero errors	Before marking done
memory-search	Check prior audit patterns	At start

Iron Laws

1. Always run from project root. validate-ecosystem-integrity.cjs resolves all paths relative to process.cwd(). Running from a subdirectory corrupts all path resolution. Never invoke from a non-root directory.

2. Never ignore PHANTOM_REQUIRE errors. A [PHANTOM_REQUIRE] error means a hook or script will throw MODULE_NOT_FOUND at runtime. These always represent real breakage — no false positives once the scanner is properly calibrated.

3. Never mark scan complete while errors > 0. Warnings are advisory; errors are blocking. Report path and severity but do not call the pipeline done until errors reach zero or are explicitly accepted as known exceptions with owner annotation.

4. Always route HIGH errors to specialist agents. Phantom skills → qa agent. Broken require() → developer agent. Archive refs in production → developer. Never attempt to remediate complex structural failures inline without delegation.

5. Always save the audit report before completing. Write the categorized findings to .claude/context/reports/qa/ecosystem-integrity-report-{ISO-date}.md before calling TaskUpdate(completed). Evidence must persist across context resets.

Anti-Patterns

Anti-Pattern	Risk	Correct Approach
Treating [STALE_CATALOG] as informational noise	Registry drifts from reality; agents get wrong counts	Always update agent-registry.json when agent files change
Skipping scan after bulk framework batch ops	Batch ops most commonly create phantom refs	Run scan as the FINAL step of every batch operation
Manually patching require() paths without re-scanning	One fix may mask three new breaks	Always re-run full scan after patching paths
Running scanner against _archive/ subtrees	Archive content has intentionally broken refs; false positives	Scanner skip rules handle this; trust the skip list
Adding to DYNAMIC_SCRIPT_GENERATORS without review	Accidentally suppressing real phantom detections	Only add files that provably generate child scripts

Step 1: Run the Integrity Audit Engine

node scripts/validation/validate-ecosystem-integrity.cjs

The script performs six targeted checks:

1. [PHANTOM_REQUIRE] — Broken require() / import paths that resolve to non-existent files (after trying .js, .cjs, .mjs, .json extensions).
2. [PHANTOM_SKILL] — Skills listed in agent frontmatter skills: arrays that have no corresponding .claude/skills/{name}/ directory.
3. [EMPTY_DIR] — Empty directories in .claude/tools/ or .claude/skills/ that indicate abandoned scaffolding creating registry footprinting issues.
4. [ENCODING] — Files with UTF-16 BOM (0xFEFF/0xFFFE) that break JSON/MD parsers silently.
5. [ARCHIVED_REF] — Production .claude/ code that still references _archive/ or .claude/archive/ paths — active code pointing at dead modules.
6. [STALE_CATALOG] — agent-registry.json entry count differs from actual agent .md file count on disk (drift between manifest and reality).

Step 2: Categorize and Report

Parse outputs into a structured report at .claude/context/reports/qa/ecosystem-integrity-report-{ISO-date}.md:

# Ecosystem Integrity Report

<!-- Agent: qa | Task: #{id} | Session: {date} -->

**Date:** YYYY-MM-DD
**Overall Status:** PASS / FAIL
**Errors:** N | **Warnings:** N

## HIGH — Runtime Blocking (must fix before deployment)

- `[PHANTOM_REQUIRE]` — Module resolution failures that crash hooks/scripts
- `[PHANTOM_SKILL]` — Missing skills that break agent workflows silently

## MEDIUM — Structural Integrity (fix in current sprint)

- `[ARCHIVED_REF]` — Active code pointing at archived/dead modules
- `[STALE_CATALOG]` — Registry count mismatch with actual agent files

## LOW — Housekeeping (fix in next maintenance window)

- `[EMPTY_DIR]` — Ghost directories creating registry noise
- `[ENCODING]` — UTF-16 files that may break parsers

Step 3: Trigger Remediations

Delegate fixes to the correct specialist:

Error Type	Spawn Agent	Task Description
[PHANTOM_REQUIRE]	developer	Fix broken require() path in {file}
[PHANTOM_SKILL]	qa	Remove or create missing skill {name}
[ARCHIVED_REF]	developer	Replace archive reference in {file}
[STALE_CATALOG]	developer	Regenerate agent-registry.json
[EMPTY_DIR]	developer	Remove empty directory {path}
[ENCODING]	developer	Re-encode {file} as UTF-8

After fixes, re-run the scanner and verify errors reach zero before completion.

Memory Protocol (MANDATORY)

Before starting:

cat .claude/context/memory/learnings.md
cat .claude/context/memory/issues.md

Review prior audit patterns and known suppressed false positives.

After completing:

• Audit pattern → .claude/context/memory/learnings.md
• New false-positive suppression rule → .claude/context/memory/decisions.md
• Unresolved error requiring owner → .claude/context/memory/issues.md

Assume interruption: If the audit report isn't saved to disk, it didn't happen.