「eval」の検索結果

Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when: - Implementing self-critique and reflection loops - Building evaluator-optimizer pipelines for quality-critical generation - Creating test-driven code refinement workflows - Designing rubric-based or LLM-as-judge evaluation systems - Adding iterative improvement to agent outputs (code, reports, analysis) - Measuring and improving agent response quality

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate any agent skill against a merged framework — Anthropic's Claude Code best practices plus Matt Pocock's writing-great-skills methodology — across 4 axes (Trigger, Structure, Steering, Pruning). Produces an evidence-cited scorecard (0–100), a weighted overall score, and diagnosed failure modes with prioritized fixes. Use when the user asks to evaluate, rate, or audit a skill ("evaluate this skill", "skill scorecard", "review SKILL.md"), or to compare two skills.

日本語の概要は準備中です。原文の説明を表示しています。

Designs coding-agent observability and evals. Use when measuring traces, replay, checkpoint lineage, tool grading, regression, or cost. Not for general LLM evals (ai-evals).

日本語の概要は準備中です。原文の説明を表示しています。

Runs the evals of a BMad skill through the agent harness the user works in and reports what they show: baseline against the bare model, variant against a stripped or prior version, quality against a rubric, and trigger accuracy of the description with an optional optimization loop. Use when the user asks to evaluate, benchmark, test the triggers of, grade, or compare versions of a skill, or when the bmad-toolsmith skill asks for an eval of a skill it built.

日本語の概要は準備中です。原文の説明を表示しています。

Design a source evaluation protocol using lateral reading and credibility checks for digital information. Use when students need to evaluate websites, online sources, or social media claims.

日本語の概要は準備中です。原文の説明を表示しています。

Design an LLM eval — task schema, scoring rubric, dataset composition, and pass/fail thresholds. Use when asked to "design an LLM eval", "write a scoring rubric", or "how do we measure this model".

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate model performance — check for accuracy drops, data drift, and error patterns. Use when asked about "model accuracy dropped", "evaluate the model", "check for drift", or "model performance".

日本語の概要は準備中です。原文の説明を表示しています。

Foundry hosted agent を Agent 365 Autopilot として発行し、自分のメールアドレス・予定表・権限で働く『デジタルな同僚』を Teams / Microsoft 365 Copilot に公開する。初回 scaffold で AI チームメイト評価Hub（Code Apps）、通常会話を採点する Python EvaluationWorker、回帰テストを同時生成し、publish 時に Foundry Monitor/Evaluations の日次評価も自動構成する。複数 Autopilot は共通 Dataverse Hub を agentkey で安全に共有する。メール応対 / Dataverse 検索 / Web 検索 / 定期実行 / コード実行 / 成果物共有 / Teams プレゼンスの機能ブロックに対応し、自己ホスト経路も互換用途として残す。CI/CD・レビューゲートは alm スキルに委譲する。

Plan, evaluate, and produce paid ad campaigns across platforms (Meta, Google, LinkedIn, TikTok, Microsoft, Pinterest, Snapchat, Amazon, X). Evaluate existing ads, map funnel stages (TOF/MOF/BOF), produce ad variants per platform with policy compliance, output ready-to-upload copy. Use when "ad campaign", "create ads", "run ads for", "paid campaign", "media plan", "launch ads", "Meta campaign", "Google Ads campaign", "multi-platform ads", "evaluate my ads", "how are my ads doing", "audit my ads", "ad performance", "analyze ads", or any request to evaluate existing or create new advertising.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate tabulated subcontractor bids against specs and drawings — scope gap analysis, exclusion risk scoring, award recommendation. Triggers: 'evaluate bids', 'bid evaluation', 'lowest responsible bidder'.

日本語の概要は準備中です。原文の説明を表示しています。

Assess an open-source agent framework for investment readiness by evaluating community health, supersession risk, architecture alignment, and governance sustainability. Produces a four-tier classification (INVEST / EVALUATE-FURTHER / CONTRIBUTE-CAUTIOUSLY / AVOID) to guide resource allocation decisions before committing engineering effort.

日本語の概要は準備中です。原文の説明を表示しています。

执行模型测评（统一入口）。LLM/VLM 走主流程 scripts/run_evaluation.py + Evaluation YAML；model_family=dit 走 DiT 扩展节（references/dit/evaluate_workflow.md，vbench.py 批量推理产出视频）。

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate a local skill with the terum-skills CLI — hygiene, trigger evals and execution cases against a baseline, billed to the user's own Claude Code — after confirming the cost, and show the result as a Markdown board. Use when the user asks to eval, test, score or benchmark a skill, or to see whether an edit made it better.

日本語の概要は準備中です。原文の説明を表示しています。

Use when building, migrating, or debugging Agent Evals on Inngest: scoring AI agent or workflow outcomes, deferred scorers, sessions, traces, step experiments, experiment variant attribution, Insights queries, or production eval loops for prompts, models, tools, providers, and agent behavior. Covers TypeScript SDK v4 scoring beta APIs, `scoreMiddleware`, `step.score`, `inngest.score`, `createScorer`, `defer`, `group.experiment`, `experimentRef`, `meta.sessions`, and when to use durable workflow primitives for outcome-based evaluation.

日本語の概要は準備中です。原文の説明を表示しています。

Run an explicit end-to-end goal loop with frozen Scope/criteria, worker delegation, external verification, evaluator review, and bounded retries. Use when the user asks for 'goal-loop', 'ゴールまで回す', '必ずやり遂げる', 'loop until done', 'until it actually works', or an orchestrator + evaluator loop. Do not use for ordinary planning, casual advice, reviews, or small single-file fixes.

Build and query a vault-local contextual BM25 retrieval index with optional multilingual Nomic cosine reranking; use for retrieve, hybrid retrieval, BM25, rerank, contextual retrieval, chunk search, vault search, semantic search, find relevant passages, or retrieval diagnostics. Derived caches stay under .vault-meta, remote egress requires explicit consent, and unavailable reranking falls back deterministically.

日本語の概要は準備中です。原文の説明を表示しています。

Experimental behavioral evaluator. Drives a running artifact via browser/curl/CLI and records rubric evidence; the runner is not yet a fully implemented automated evaluation system.

日本語の概要は準備中です。原文の説明を表示しています。

Quantitative agent evaluation using 4-metric framework (correctness/step_ratio/tool_call_ratio/latency_ratio) with ideal trajectory annotation and capability-categorical taxonomy. Use when measuring agent efficiency, comparing agent variants, or gating new agents through correctness→efficiency phases. Complements harness-eval (SE benchmarks) and evaluator-optimizer (qualitative rubric).

日本語の概要は準備中です。原文の説明を表示しています。

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

日本語の概要は準備中です。原文の説明を表示しています。

Avalia LLMs em 100+ benchmarks de 18+ harnesses (MMLU, HumanEval, GSM8K, segurança, VLM) com execução multi-backend. Use quando precisar de avaliação escalável em Docker local, HPC Slurm ou plataformas cloud. Plataforma enterprise-grade da NVIDIA com arquitetura container-first para benchmarking reproduzível.

日本語の概要は準備中です。原文の説明を表示しています。

Avalia modelos de geração de código em HumanEval, MBPP, MultiPL-E e 15+ benchmarks com métricas pass@k. Use ao comparar modelos de código, avaliar capacidades de codificação, testar suporte multilíngue ou medir qualidade de geração de código. Padrão da indústria do BigCode Project usado pelos leaderboards do HuggingFace.

日本語の概要は準備中です。原文の説明を表示しています。

Improves Agent Skills via validate → rule-based eval cases → plugin-eval → prompt evals → bounded edits with held-out gates. Use when tuning skill quality, routing, or adopting Chrome/Microsoft T-named quality gates—not for bulk validate-only or SkillOpt automation.

日本語の概要は準備中です。原文の説明を表示しています。

既存skillの品質を実走eval駆動で反復改善したいとき、run-skill-live-trialの受け入れFAILやgoal-proxy乖離を改善に引き継ぐときに、PASS詐欺・context汚染・評価縮退を構造的に防ぐeval帰属改善ループとして起動する。

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

日本語の概要は準備中です。原文の説明を表示しています。