OpenClaw Testing

Prove the changed contract with the smallest meaningful check, complete required checks, then finish. Broaden or repeat only for changed inputs, failures, or unresolved risks. For behavioral proof, use the boundary guide; use $test-audit when authoring or reviewing tests.

For ordinary local tests, start at docs/reference/test.md#routine-local-order and #core-commands; read docs/ci.md when CI scope or runner behavior matters. Follow the touched subtree's AGENTS.md.

Select The Proof

Trusted development tests, changed checks, typecheck/lint, and builds run locally, including broader suites when the contract warrants them. Remote compute is for isolation, clean installation, packaging, Docker, live services, desktop/platform behavior, or an explicit operator request.

Change or question	Starting point
Runtime defect	Reproduce narrowly; rerun that proof after the repair, plus relevant siblings
Trusted source diff	pnpm changed:lanes --json, pnpm check:changed, focused tests
Public SDK/plugin contract	Changed checks plus representative consumer tests; no automatic all-plugin sweep
Build output, lazy imports, package boundaries	Include pnpm build
Workflow	git diff --check and pnpm check:workflows
Documentation only	Relevant docs/link/format sanity and git diff --check; no runtime tests

For specialized proof, load only the selected route:

• Remote leases and credentials: $crabbox, with the OpenClaw bootstrap binding below.
• Package installation, plugin package trust, Docker/live lane selection or reruns: Package And Docker Proof.
• Release candidates, full-validation dispatch, evidence identity or recovery: $release-openclaw-ci. A narrow green rerun does not itself authorize publication. Do not substitute moving main for the recorded candidate or Tooling SHA.
• Plugin release matrix: $release-openclaw-plugin-testing.
• New or changed Docker lanes: $openclaw-docker-e2e-authoring.
• Channel/UI behavior: the relevant channel proof skill or $control-ui-e2e; mock-Gateway boundary proof is valid when it covers the changed path. State concrete live-proof gaps.

Source And State Boundaries

Untrusted contributor/fork tooling must never execute locally, including its wrapper or config. Use secretless fork CI or sanitized direct AWS under $crabbox; never credential-hydrated Testbox. Credentialed execution requires maintainer approval after review, and never hydrates an untrusted lease.

For untrusted OpenClaw AWS proof, supply the clean trusted main copy of scripts/crabbox-untrusted-bootstrap.sh as Crabbox's <trusted-bootstrap-script>. Bind the fresh lease and --fresh-pr checkout to the reviewed full head SHA. The trusted bootstrap verifies that SHA, the IMDSv2 no-role boundary, and the package-manager pin before installing into an isolated HOME. Keep CRABBOX_ENV_ALLOW=CI, --no-hydrate, no instance role, and no Tailscale. A moved head needs a fresh lease; missing no-role proof or no remote PR means secretless CI. Read the Crabbox untrusted procedure before allocation.

For trusted remote proof, use node scripts/crabbox-wrapper.mjs with the resolved provider; do not silently switch providers or bypass sync/security exclusions. Save and reuse task-owned leases, verify the materialized candidate, keep evidence outside the synced checkout, and stop owned leases at handoff.

Use isolated state and a free port. Never restart, edit, or test against an operator Gateway or real data without explicit per-task approval. Do not kill unrelated processes or reconcile a shared dependency install while jobs use it.

Local Commands

pnpm changed:lanes --json
pnpm check:changed
pnpm test <path-or-filter>
pnpm test:changed

check:changed selects formatting, typecheck, lint, and guard work and may run targeted owner Vitest tests. Inspect its plan with node scripts/check-changed.mjs --dry-run -- <paths...>; it is not the full test suite. test:changed chooses direct tests, mapped/sibling tests, and import dependents; shared harness/config edits can need explicit targets or the broad fallback OPENCLAW_TEST_CHANGED_BROAD=1 pnpm test:changed. pnpm verify runs the full check and then test when that scope is justified.

Use repository wrappers rather than raw Vitest so project routing and setup remain correct. If dependencies are ready in a linked worktree, these bypass pnpm dependency reconciliation:

node scripts/check-changed.mjs
node scripts/run-vitest.mjs <path-or-filter>

Concurrent test/check commands must not share a Vitest filesystem module cache: serialize them, group tests in one invocation, or give each command a distinct OPENCLAW_VITEST_FS_MODULE_CACHE_PATH. Checks can schedule Vitest too. For worker-sensitive failures, OPENCLAW_VITEST_MAX_WORKERS=1 pnpm test <path> provides a focused serial probe; do not make a forced environment the repair.

CI Failures

gh run list --branch <branch> --limit 10 --json databaseId,headSha,status,conclusion,url
gh run view <run-id> --json status,conclusion,headSha,url,jobs
gh run view <run-id> --job <failed-job-id> --log

Bind the diagnosis to the exact SHA and job. Check whether cancellation means a newer same-branch run superseded it. Fetch relevant failed logs once and reuse them; prefer exact run/job state over a stale PR rollup. Separate product, harness, infrastructure, and credential failures before choosing a retry.

For prompt snapshot drift that passes on macOS, reproduce in CI's Linux/Node environment before regenerating; a local pass cannot override failing CI bytes. Fix related failures and rerun the affected proof. Route unrelated failures with evidence rather than broadening this task automatically.

Test failure policy

Treat test failures as defects and make a bounded, best-effort attempt to reproduce them (same shard order first), identify their cause, and fix the owning fixture, shared state, ordering, or product. When a safe fix is established, add a regression and document the cause; cite another owner's fix when applicable. If reasonable investigation cannot establish or complete a safe fix, record the original failure, attempted reproductions, evidence, and remaining uncertainty in the PR, then continue under the normal CI and review gates. The unresolved failure alone must not block landing or trigger an extra approval request. Never claim a passing replay proves a fix. Do not rerun, re-push, or refresh merely to get green, or conceal failures with retries, longer timeouts, weaker assertions, broader mocks, or altered baselines.