「evaluation」の検索結果

論文や研究計画を共通の評価基準で読み、研究方法、根拠、引用、文章構成を点検するスキル。評価の理由と修正の優先順位、次に直す内容を整理します。

• 投稿前の論文を見直したいとき
• 引用が主張を支えるか確認したいとき
• 研究方法と限界を評価したいとき

機械学習のない既存アプリに、推薦・分類・予測などを追加するための手順を整理します。課題とデータの確認から、最初のモデルの学習・評価・組み込みまでを支援します。

• 既存アプリに推薦機能を加えたいとき
• 学習データの準備状況を確認したいとき
• 代替ルール付きで予測を組み込みたいとき

アプリ開発を計画・実装・評価のエージェントに分け、動作中の画面を検証しながら、設定した品質基準に向けて修正を繰り返す開発支援スキル。

• 短い案からアプリ仕様を作りたいとき
• 別担当の評価で画面を改善したいとき
• フルスタックアプリの動作検証と修正

AIエージェントが継続して作業するループを、品質確認と評価、失敗時の停止・再実行の方針を組み合わせて設計し、進捗停滞や再試行に対処するスキル。

• 継続作業のループ方式を選びたいとき
• 品質確認と評価を組み合わせたいとき
• 失敗や進捗停滞からの復旧

AIエージェントに任せる開発を、完了条件の定義、作業分割、実装前後の評価で進め、難易度に応じたモデル選択と費用・時間の記録まで整理するスキル。

• AIエージェントに任せる開発の計画
• 大きな変更を検証可能な作業に分けたいとき
• 実装前後の評価を比較したいとき

大規模言語モデルの知識・推論・コード生成を共通の課題で評価するスキル。複数モデルの比較表や学習途中の成績推移を作り、評価結果と個別の回答を保存します。

• MMLU・GSM8Kでのモデル評価
• 複数モデルの比較表作成
• 学習途中のモデルの成績追跡

Command-line interface for CloudAnalyzer — Agent-friendly harness for CloudAnalyzer, a QA platform for mapping, localization, and perception outputs. Supports 27 commands across 8 groups: point cloud evaluation, trajectory evaluation, ground segmentation QA, config-driven quality gates, baseline evolution, processing, visualization, and interactive REPL.

日本語の概要は準備中です。原文の説明を表示しています。

6 production-ready AI engineering workflows: prompt evaluation (8-dimension scoring), context budget planning, RAG pipeline design, agent security audit (65-point checklist), eval harness building, and product sense coaching.

日本語の概要は準備中です。原文の説明を表示しています。

Prompt engineering regression test matrix register: baseline outputs, semantic drift thresholds, judge evaluations, and golden dataset.

日本語の概要は準備中です。原文の説明を表示しています。

Handles LLM-as-judge evaluation workflows on Arize including creating/updating evaluators, running evaluations on spans or experiments, managing tasks, trigger-run operations, column mapping, and continuous monitoring. Use when the user mentions create evaluator, LLM judge, hallucination, faithfulness, correctness, relevance, run eval, score spans, score experiment, trigger-run, column mapping, continuous monitoring, or improve evaluator prompt.

日本語の概要は準備中です。原文の説明を表示しています。

Technical leadership guidance for engineering teams, architecture decisions, and technology strategy. Includes tech debt analyzer, team scaling calculator, engineering metrics frameworks, technology evaluation tools, and ADR templates. Use when assessing technical debt, scaling engineering teams, evaluating technologies, making architecture decisions, establishing engineering metrics, or when user mentions CTO, tech debt, technical debt, team scaling, architecture decisions, technology evaluation, engineering metrics, DORA metrics, or technology strategy.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

日本語の概要は準備中です。原文の説明を表示しています。

LLM observability platform for tracing, evaluation, and monitoring. Use when debugging LLM applications, evaluating model outputs against datasets, monitoring production systems, or building systematic testing pipelines for AI applications.

日本語の概要は準備中です。原文の説明を表示しています。

Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when: langfuse, llm observability, llm tracing, prompt management, llm evaluation.

日本語の概要は準備中です。原文の説明を表示しています。

Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.

日本語の概要は準備中です。原文の説明を表示しています。

EU MDR 2017/745 compliance specialist for medical device classification, technical documentation, clinical evidence, and post-market surveillance. Covers Annex VIII classification rules, Annex II/III technical files, Annex XIV clinical evaluation, Art. 86 PSUR schedules, and EUDAMED integration. Use when classifying a medical device under MDR, building or gap-checking a technical file, planning clinical evaluation or PMS/PSUR cadence, or preparing for notified body review (e.g., 'what class is my device under MDR', 'review my PSUR schedule').

日本語の概要は準備中です。原文の説明を表示しています。

Use this skill to manage Google Cloud Workload Manager evaluations, rules, scanned resources, and validation results by using public client libraries and the REST API. Use when you need to inspect workload best-practice rules, create and run evaluations for Google Cloud general best practices, SAP, SQL Server, or custom organizational rules, review violations, export results to BigQuery, or automate Workload Manager through client libraries because no service-specific public CLI or MCP server is available. Don't use for general Google Compute Engine instance management, VPC configuration, or standard IAM auditing.

日本語の概要は準備中です。原文の説明を表示しています。

This skill should be used when writing, enhancing, or evaluating the launch prompt for a long-running autonomous agent or a parallel multi-agent orchestration attacking a hard problem: pseudo-formal task briefs that define terms and an exact success predicate linguistically, enumerate non-counting outcomes, set persistence rules with explicit stop and return conditions and effort floors, manage a diverse portfolio of parallel approaches with an approach registry and blocked-route bookkeeping, and gate the return on adversarial audit. Route agent topology and coordination protocols to multi-agent-patterns, runtime control surfaces and loop governance to harness-engineering, evaluator and quality-gate construction to evaluation, judge design to advanced-evaluation, and compaction or memory mechanics to context-compression and memory-systems.

日本語の概要は準備中です。原文の説明を表示しています。

This skill should be used when building agent evaluation systems: deterministic checks, regression suites, multi-dimensional rubrics, quality gates, production monitoring, baseline comparison, and outcome measurement for agent pipelines.

日本語の概要は準備中です。原文の説明を表示しています。

This skill should be used when the harness, scaffold, workflow, or optimizer itself is the optimization target: recursive self-improvement (RSI) loops, meta-harnesses, self-improving harnesses that mine their own failures and propose bounded edits, evolutionary or population-based search over agent scaffolds, acceptance gates for self-modifying systems, and agentic context evolution where the mechanism that produces context is versioned and evolved. Route governance of a single autonomous loop (locked surfaces, durable logs, rollback, novelty gates, approval boundaries) to harness-engineering, measurement and quality-gate design to evaluation, judge design to advanced-evaluation, and remote sandbox infrastructure to hosted-agents.

日本語の概要は準備中です。原文の説明を表示しています。

This skill should be used for advanced LLM evaluation: LLM-as-judge systems, direct scoring, pairwise comparison, rubric calibration, evaluator bias mitigation, confidence scoring, and automated quality assessment.

日本語の概要は準備中です。原文の説明を表示しています。

Runs the CARLA Leaderboard evaluator over a set of routes with your agent — routes/routes-subset/repetitions/track selection, checkpoint and live-results endpoints, resume after a crash, recording, and the debug levels — for leaderboard 1.0, 2.0 or 2.1. Preflights the whole stack (version pairing, maps, ports, agent class, sensor budget) before spending hours, and hands the world back in async mode. Use when the user asks to "run the leaderboard", "evaluate my agent", "run routes_training", "resume an evaluation", or "reproduce my submission locally".

日本語の概要は準備中です。原文の説明を表示しています。

Provides guidance for automatically evolving and optimizing AI agents across any domain using LLM-driven evolution algorithms. Use when building self-improving agents, optimizing agent prompts and skills against benchmarks, or implementing automated agent evaluation loops.

日本語の概要は準備中です。原文の説明を表示しています。