「lm evaluation harness」の検索結果

大規模言語モデルの知識・推論・コード生成を共通の課題で評価するスキル。複数モデルの比較表や学習途中の成績推移を作り、評価結果と個別の回答を保存します。

• MMLU・GSM8Kでのモデル評価
• 複数モデルの比較表作成
• 学習途中のモデルの成績追跡

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

日本語の概要は準備中です。原文の説明を表示しています。

Avalia LLMs em mais de 60 benchmarks acadêmicos (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use quando benchmarking de qualidade de modelo, comparação de modelos, relatório de resultados acadêmicos ou rastreamento de progresso de treinamento. Padrão da indústria usado por EleutherAI, HuggingFace e laboratórios principais. Suporta HuggingFace, vLLM, APIs.

日本語の概要は準備中です。原文の説明を表示しています。

Design eval harnesses — task schemas, metrics, dataset versioning, eval-as-code patterns. Use when asked to "build an eval harness", "set up eval-as-code", or "version our eval datasets".

日本語の概要は準備中です。原文の説明を表示しています。

lm-eval-harness: benchmark LLMs (MMLU, GSM8K, etc.).

日本語の概要は準備中です。原文の説明を表示しています。

Rigorous benchmarking of multi-agent coordination systems: experiment design, statistical analysis, human evaluation protocols, and reproducible reporting. NOT FOR ML model evaluation (use llm-evaluation-harness), A/B testing for web products, survey design, or general data science.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.

日本語の概要は準備中です。原文の説明を表示しています。

Test and evaluation harness for AI agents — scenario suites, deterministic replay, regression diffing, cost and latency budgets. Use when agent quality is vibe-checked, before shipping a prompt or model change, or when evals drift.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.

日本語の概要は準備中です。原文の説明を表示しています。

Builds repeatable evaluation systems with golden datasets, scoring rubrics, pass/fail thresholds, and regression reports. Use for "LLM evaluation", "testing AI systems", "quality assurance", or "model benchmarking".

日本語の概要は準備中です。原文の説明を表示しています。

6 production-ready AI engineering workflows: prompt evaluation (8-dimension scoring), context budget planning, RAG pipeline design, agent security audit (65-point checklist), eval harness building, and product sense coaching.

日本語の概要は準備中です。原文の説明を表示しています。

6 production-ready AI engineering workflows: prompt evaluation (8-dimension scoring), context budget planning, RAG pipeline design, agent security audit (65-point checklist), eval harness building, and product sense coaching.

日本語の概要は準備中です。原文の説明を表示しています。

AI Engineering Toolkit workflow skill. Use this skill when the user needs 6 production-ready AI engineering workflows: prompt evaluation (8-dimension scoring), context budget planning, RAG pipeline design, agent security audit (65-point checklist), eval harness building, and product sense coaching and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.

日本語の概要は準備中です。原文の説明を表示しています。

Reviews LLM integration, prompt templates, model selection, token/cost budgets, evaluation harnesses, structured outputs, retries, and streaming. Checks trust boundaries and task evidence against the selected provider/model contract rather than prescribing universal model tiers or sampling settings.

日本語の概要は準備中です。原文の説明を表示しています。

6 production-ready AI engineering workflows: prompt evaluation (8-dimension scoring), context budget planning, RAG pipeline design, agent security audit (65-point checklist), eval harness building, and product sense coaching.

日本語の概要は準備中です。原文の説明を表示しています。

Avalia LLMs em 100+ benchmarks de 18+ harnesses (MMLU, HumanEval, GSM8K, segurança, VLM) com execução multi-backend. Use quando precisar de avaliação escalável em Docker local, HPC Slurm ou plataformas cloud. Plataforma enterprise-grade da NVIDIA com arquitetura container-first para benchmarking reproduzível.

日本語の概要は準備中です。原文の説明を表示しています。

6 production-ready AI engineering workflows: prompt evaluation (8-dimension scoring), context budget planning, RAG pipeline design, agent security audit (65-point checklist), eval harness building, and product sense coaching.

日本語の概要は準備中です。原文の説明を表示しています。

6 production-ready AI engineering workflows: prompt evaluation (8-dimension scoring), context budget planning, RAG pipeline design, agent security audit (65-point checklist), eval harness building, and product sense coaching.

日本語の概要は準備中です。原文の説明を表示しています。