69 件 ・ 関連度順
概要と使いどころ
You're a quality engineer who has seen agents that aced benchmarks fail spectacularly in production. You've learned that evaluating LLM agents is fundamentally different from testing traditional software—the same input can produce different outputs, and "correct" often has no single answer.
日本語の概要は準備中です。原文の説明を表示しています。
Reviews one SKILL.md against the format rules and returns a findings table: strict YAML front matter with exactly name and description, name equal to the folder, description as the trigger, the eight-heading skeleton, capability-neutral wording, safety boundaries. Each finding carries severity, rule identifier, location, quoted evidence and replacement text; the verdict READY, REVISE or BLOCKED is an apparent state and the owner decides. Use when the user asks to "review this SKILL.md", "check my skill file before upload", "lint this skill description", "validate the front matter of this skill" or "score this skill folder". Do not use to write an agent's instructions field, use agent-instructions-drafter instead; for injection or leakage exposures use agent-instructions-red-team; for launch tests use agent-evaluation-plan. Drafts for human review; never approves, authorises or signs off.
日本語の概要は準備中です。原文の説明を表示しています。
Reviews the instructions text and skills of a declarative agent, with its stated capabilities, knowledge sources and audience, for prompt-injection exposure, data-leakage paths, over-broad permissions and missing refusals, and returns an attack-surface map, a findings table with severity, quoted evidence and attack path, one pasteable fix per finding, a probe pack the owner can run and a refusal-coverage table. Use when the user asks to "red-team this agent", "check these instructions for prompt injection", "could this agent leak data", "review the permissions on my agent" or "what should this agent refuse". Do not use for checking a SKILL.md against the format rules, use skill-file-reviewer instead; do not use for designing the launch test set, use agent-evaluation-plan instead. Drafts for human review; never approves, authorises or signs off.
日本語の概要は準備中です。原文の説明を表示しています。
Designs a pre-launch evaluation plan for a declarative agent from its instructions text, skills, capabilities, knowledge sources and sample requests: a behaviour contract of testable statements quoted from the text, test prompts by scenario family and tier with setup, expected behaviour, pass and fail criteria and red flags, a red-flag catalogue, a blank scoring sheet, a coverage table and a go or no-go checklist the owner completes and decides. Use when the user asks to "write test cases for my agent", "how do I evaluate this agent before launch", "build a test plan for the agent", "what should I check before publishing this agent" or "give me a go no-go checklist". Do not use for finding injection or leakage exposures in the instructions, use agent-instructions-red-team instead; do not use for reviewing one SKILL.md, use skill-file-reviewer instead. Drafts for human review; never approves, authorises or signs off.
日本語の概要は準備中です。原文の説明を表示しています。
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on re...
日本語の概要は準備中です。原文の説明を表示しています。
Use test-driven development for behavior-changing feature or fix work, and whenever the user mentions TDD, test-first, red-green-refactor, tracer bullets, integration tests, or public-interface behavior tests. Skip for docs-only, path-only rename, formatting-only, or purely mechanical chores unless explicitly requested.
日本語の概要は準備中です。原文の説明を表示しています。
Build command-only adapters for `eval` run mode using strict stdin/stdout JSON contracts.
日本語の概要は準備中です。原文の説明を表示しています。
End-to-end eval-suite orchestration with the `eval` command: run -> grade -> compare -> calibrate, using strict JSON contracts and schema discovery.
日本語の概要は準備中です。原文の説明を表示しています。
Interview the user relentlessly about a plan or design until reaching shared understanding, resolving each branch of the decision tree. Use when user wants to stress-test a plan, get grilled on their design, or mentions "grill me".
日本語の概要は準備中です。原文の説明を表示しています。
Analyze graded `trial_result` JSONL outputs from the eval pipeline, including reliability and custom slicing.
日本語の概要は準備中です。原文の説明を表示しています。
Autonomous AI agent benchmark evaluation register: task completion rates, planning accuracy, tool invocation precision, and cost benchmarks.
日本語の概要は準備中です。原文の説明を表示しています。
Evaluate agent behavior with versioned cases and explicit verifiers. Use when comparing agent or prompt changes, reproducing failures, or running agent regression tests.
日本語の概要は準備中です。原文の説明を表示しています。
Use when summarizing agent evaluations where autonomous, assisted, failed, timed-out, or invalid outcomes must remain distinct and comparable.
日本語の概要は準備中です。原文の説明を表示しています。
Build automated evaluation suites for AI agents using golden datasets, rubrics, and regression gates. Use when shipping agent features, validating prompt changes, or gating deployments on quality.
日本語の概要は準備中です。原文の説明を表示しています。
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.
日本語の概要は準備中です。原文の説明を表示しています。
You're a quality engineer who has seen agents that aced benchmarks fail spectacularly in production. You've learned that evaluating LLM agents is fundamentally different from testing traditional software—the same input can produce different outputs, and "correct" often has no single answer.
日本語の概要は準備中です。原文の説明を表示しています。
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks
日本語の概要は準備中です。原文の説明を表示しています。
You're a quality engineer who has seen agents that aced benchmarks fail spectacularly in production. You've learned that evaluating LLM agents is fundamentally different from testing traditional software—the same input can produce different outputs, and "correct" often has no single answer.
日本語の概要は準備中です。原文の説明を表示しています。
コーディングエージェント（Claude Code、Aider、Codexなど）のカスタムタスクによる直接比較。合格率、コスト、時間、一貫性メトリクスを測定
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks
日本語の概要は準備中です。原文の説明を表示しています。
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.
日本語の概要は準備中です。原文の説明を表示しています。