「evaluation」の検索結果

Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarksUse when "agent testing, agent evaluation, benchmark agents, agent reliability, test agent, testing, evaluation, benchmark, agents, reliability, quality" mentioned.

日本語の概要は準備中です。原文の説明を表示しています。

Agent Evaluation workflow skill. Use this skill when the user needs Testing and benchmarking LLM agents including behavioral testing, and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.

日本語の概要は準備中です。原文の説明を表示しています。

Design a source evaluation protocol using lateral reading and credibility checks for digital information. Use when students need to evaluate websites, online sources, or social media claims.

日本語の概要は準備中です。原文の説明を表示しています。

LLM-as-judge evaluation framework with 5-dimension rubric (accuracy, groundedness, coherence, completeness, helpfulness) for scoring AI-generated content quality with weighted composite scores and evidence citations

日本語の概要は準備中です。原文の説明を表示しています。

Generate msmodelslim evaluation YAML configuration (service_oriented + aisbench + vllm-ascend). Use when user asks for evaluation config generation.

日本語の概要は準備中です。原文の説明を表示しています。

Create evaluation frameworks for AI outputs with rubrics and scoring. TRIGGERS - Use when user needs help with ai-evaluation-framework related tasks.

日本語の概要は準備中です。原文の説明を表示しています。

Author and run offline evals for Agentforce agents: fixture format, scoring rubrics, hallucination checks, regression baselines, CI integration, prompt-change safety. Trigger keywords: agent eval, hallucination, doesn't hallucinate, agentforce offline evals, rubric for agent response. NOT for deciding which layers of testing the agent needs — use agentforce/agentforce-testing-strategy. NOT for Testing Center, AiEvaluationDefinition or sf agent test runs — use agentforce/agent-testing-and-evaluation.

日本語の概要は準備中です。原文の説明を表示しています。

Turns a business need and its constraints into a DRAFT RFP requirements pack: traced functional and non-functional requirements with priorities, mandatory gates, evaluation criteria with weights left to the business owner, and fact-seeking supplier questions. Solution-neutral; never names a supplier or product. Use when the user asks to "write the RFP requirements", "turn this business case into tender requirements", "draft the supplier questionnaire", "structure our RFQ from this brief" or "set up evaluation criteria before we go to market". Do not use for responses already received, use rfp-comparison-pack to compare them or supplier-evaluation-matrix to score them instead. Drafts for human review; never approves, authorises or signs off.

日本語の概要は準備中です。原文の説明を表示しています。

Builds a DRAFT weighted supplier evaluation matrix from supplier responses and the panel's agreed criteria, weights and scale: a provisional score per cell with the quoted evidence and anchor behind it, a register of every missing or partial answer, and calibration and sensitivity flags. Provisional for panel moderation; never recommends, awards or disqualifies. Use when the user asks to score, weight or rank supplier, bid or tender responses against agreed criteria, for example "score these bids", "build the evaluation matrix", "weight the tender responses" or "rank the suppliers against our criteria". Do not use for a side-by-side comparison without scores, use rfp-comparison-pack instead; for writing the criteria, use rfp-requirements-pack instead. Drafts for human review; never approves, authorises or signs off.

日本語の概要は準備中です。原文の説明を表示しています。

Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.

日本語の概要は準備中です。原文の説明を表示しています。

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise compari

日本語の概要は準備中です。原文の説明を表示しています。

Master comprehensive evaluation strategies for LLM applications, from automated metrics to human evaluation and A/B testing.

日本語の概要は準備中です。原文の説明を表示しています。

DDD評価エージェント - 戦略的設計・戦術的設計の両面からDDD原則への適合度を評価し改善点を特定。/ddd-evaluation [対象パス] で呼び出し。

評価統合エージェント - MMI評価とDDD評価の結果を統合し、優先度付き改善計画を策定。/integrate-evaluations [対象パス] で呼び出し。

Generate HTML reports and structured metrics from model evaluation results. Creates publication-ready reports with embedded outputs (images, audio, charts) and metrics.json for cross-model comparison. Use when generating reports, writing metrics, creating evaluation summaries, or formatting benchmark results. Triggers on "generate report", "write metrics", "create report", "evaluation summary", "benchmark results", "format results".

日本語の概要は準備中です。原文の説明を表示しています。

Focused Signals scout for PostHog projects using feature flags. Watches the flag roster and the `$feature_flag_called` evaluation stream for contradictions between a flag's configured state and its real traffic: evaluation cliffs on healthy flags, ghost flags (code calling keys that no longer exist), response-distribution shifts with no corresponding flag edit, and flag debt (stale, fully-rolled-out, or dead flags still burning evaluations). Emits findings only when they clear the confidence bar; otherwise writes durable memory and closes out empty. Self-contained peer in the signals-scout-* fleet — no dependencies on other skills.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

日本語の概要は準備中です。原文の説明を表示しています。

Open-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.

日本語の概要は準備中です。原文の説明を表示しています。

Generate the success criteria for a task or question, then review work against them. Given a task, goal, or open-ended question, decompose it into scenarios, evaluation perspectives, and fine-grained weighted YES/NO criteria using the Recursive Expansion Tree (RET) method; if work is supplied, score it criterion-by-criterion and surface what is missing or could be better. Use when asked to self-review or check your own work, judge whether a task is done well or completely, build a definition-of-done or completeness checklist, create an evaluation rubric or grading criteria, score or grade answers to a question, set up an LLM-as-judge rubric, or when the user mentions self-review, completeness check, success criteria, evaluation criteria, scoring rubric, Qworld, or the RET algorithm.

日本語の概要は準備中です。原文の説明を表示しています。

Create evaluation frameworks for AI outputs with rubrics and scoring. TRIGGERS - Use when user needs help with ai-evaluation-framework related tasks.

日本語の概要は準備中です。原文の説明を表示しています。

Design structured evaluation rubrics for scoring LLM and agent outputs — defining quality dimensions, scoring scales, hard gates, score descriptions, and edge cases. Load when the user asks to create an eval rubric, define evaluation criteria, design scoring dimensions, write an eval spec, or says "what should I evaluate", "design a rubric", "create eval criteria", "define quality dimensions", "evaluation rubric for", "how do I measure quality of". Sub-skill of eval-output orchestrator.

日本語の概要は準備中です。原文の説明を表示しています。

Design automated evaluation pipelines for LLM and agent systems — combining deterministic checks, statistical metrics, and LLM-as-judge scoring into repeatable, CI-integrated eval suites. Load when the user asks to set up automated evals, design an eval pipeline, integrate evals into CI/CD, create an eval suite, do eval-driven development, or says "automate my evals", "CI eval integration", "evaluation pipeline", "continuous evaluation", "monitoring eval quality", "set up regression testing for my agent". Sub-skill of eval-output orchestrator.

日本語の概要は準備中です。原文の説明を表示しています。

Builds repeatable evaluation systems with golden datasets, scoring rubrics, pass/fail thresholds, and regression reports. Use for "LLM evaluation", "testing AI systems", "quality assurance", or "model benchmarking".

日本語の概要は準備中です。原文の説明を表示しています。

Plan, evaluate, and adopt agent infrastructure for a named workload. Use for agent runtime/framework selection, agent evaluation or observability plans, memory and tool boundaries, infrastructure cost controls, and organizational adoption. Covers workload-first comparison, risk and recovery planning, held-out evaluation, cost accounting, and staged rollout. Not a vendor ranking or a production certification. NOT for implementing a specific agent behavior (agentic-patterns), building a specific agent (ai-engineer), prompt optimization (prompt-engineer), or conventional ETL workflow design (jury_rig-architect).

日本語の概要は準備中です。原文の説明を表示しています。