317 件 ・ 関連度順
概要と使いどころ
Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.
日本語の概要は準備中です。原文の説明を表示しています。
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
日本語の概要は準備中です。原文の説明を表示しています。
Pick an LM output format per (task x consumer x model) rather than by reflex: different formats carry different cognitive load (e.g. code-in-JSON makes the same model write worse code than plain-text+diff, while asking for prose when you need a typed object fails the other way). Use when designing or debugging an LM's output schema, choosing between plain text / diff / JSON / tool-call / grammar-constrained output, or when a model's quality drops after wrapping its output in a structured format.
日本語の概要は準備中です。原文の説明を表示しています。
Use when choosing between engines or models for an agent pipeline and the answer must come from measurement on your own data, not from marketing pages: speech recognition for meeting recordings, the model that turns a transcript into notes, the critic model that checks it. Runs every variant in an isolated container, measures time, cost per hour of input (USD and RUB, with price source and date) and quality (WER/CER and course terms for ASR; code checks plus a judge of a different model for LLM steps), writes results.json/csv and one static analytics page. Triggers on "бенчмарк", "сравни движки", "сколько стоит час записи", "какую модель взять для пайплайна", "benchmark the pipeline", "compare ASR engines".
日本語の概要は準備中です。原文の説明を表示しています。
Produce video with Moonvalley's Marey model family (Marey Realism v1.5) — a filmmaker-oriented, 1080p/24fps generative video model marketed as trained exclusively on licensed data. Use this skill when a task asks for Marey or Moonvalley specifically, when a brief demands brand-safe / legally reviewed AI video for commercial or studio work, or when the request needs Marey's director-style controls (camera trajectory, motion transfer, pose transfer, keyframing, reference conditioning). It covers what "commercially safe" really means versus the actual terms of service, access routes (Moonvalley web / Voyager, the fal API, ComfyUI), pricing, capability limits, prompt and reference strategy, where Marey fits production versus where other models win, iteration, and quality review. Do not use it for audio generation, for photoreal talking-head dialogue, or as a generic "best video model" default.
日本語の概要は準備中です。原文の説明を表示しています。
Move quality earlier in the development lifecycle. Covers dev/QA pairing patterns, Three Amigos sessions, TDD facilitation (Red-Green-Refactor), PR review checklists for testability, and Definition of Done with quality gates. Includes shift-left maturity model for team assessment. Use when: "shift left," "TDD," "dev-QA pairing," "definition of done," "testability," "quality culture," "QA in sprint planning." Not for: writing the unit tests themselves — use unit-testing; automated PR test-quality review at scale — use ai-qa-review; multi-quarter QA direction or roadmap — use test-strategy. Related: unit-testing, ai-qa-review, test-strategy.
日本語の概要は準備中です。原文の説明を表示しています。
Use when a model is served online (FastAPI, BentoML) or scored in batch, when changing how a model artifact is loaded or versioned, or when adding drift, quality or prediction monitoring and a rollout plan for a new model
日本語の概要は準備中です。原文の説明を表示しています。
Run a Meta-Harness-style optimization loop NATIVELY — automatically search over the scaffolding around a FIXED base model (memory, retrieval, context construction, prompt templates, summarization, tool-selection logic) by proposing candidate variants, scoring each on a cheap deterministic eval, and keeping a Pareto frontier of quality vs cost — using native Agent / Workflow / loop tools instead of a standalone Python harness. Use this whenever the user wants to optimize, evolve, tune, distill, or search over a harness, scaffold, prompt system, memory or retrieval policy, context-assembly code, or summarizer while keeping the model fixed; whenever they mention Meta-Harness, harness optimization, scaffold evolution, automatic prompt/memory optimization, an evolutionary or Pareto search over candidate implementations, or "make the harness/agent better without retraining"; and whenever the gain must come from the code AROUND the model rather than the model weights. Reproduces the Meta-Harness paper's method natively, with no claude_wrapper.py and no metered solver API.
日本語の概要は準備中です。原文の説明を表示しています。
Expert guide for intelligent model cascading and routing — complexity-scored task routing from Flash/Haiku to Sonnet/Opus/Astra, dynamic escalation with quality gates, 40-60% token cost reduction while maintaining output quality / Panduan ahli untuk kaskade dan routing model cerdas — routing tugas berbasis skor kompleksitas dari Flash/Haiku ke Sonnet/Opus/Astra, eskalasi dinamis dengan gerbang kualitas, pengurangan biaya token 40-60% dengan kualitas output terjaga.
日本語の概要は準備中です。原文の説明を表示しています。
Use when the operator wants two or more model conditions (Opus 5, Fable 5.1, Sol) compared on one source head and one prompt, or wants a harness change that must serve more than one model. Covers the condition design that separates harness quality from solver capability, the recorded-evidence join, the reading of each bucket, and the rule under which a change is admitted for every model rather than one.
日本語の概要は準備中です。原文の説明を表示しています。
Constructs multi-factor models with value, momentum, quality, size, and volatility factor definitions and portfolio construction rules. Use when building factor models, designing systematic strategies, or constructing factor portfolios.
日本語の概要は準備中です。原文の説明を表示しています。
Use when solving CUMCM, MCM, ICM, or other mathematical modeling tasks that need contest problem intake, model selection, Python solving, figures, paper writing, quality audit, and packaged deliverables.
日本語の概要は準備中です。原文の説明を表示しています。
Run the four paper-quality autoraters from PaperOrchestra (arXiv:2604.05018, App. F.3) — Citation F1 (P0/P1 partition + Precision/Recall/F1), Literature Review Quality (6-axis 0-100 with anti-inflation rules), SxS Overall Paper Quality (side-by-side), and SxS Literature Review Quality (side-by-side). TRIGGER when the user asks to "score this paper draft", "evaluate against the benchmark", "compare two papers", or "run the autoraters".
日本語の概要は準備中です。原文の説明を表示しています。
Review a data analysis for quality, correctness, and reproducibility. Covers data quality assessment, assumption checking, model validation, data leakage detection, and reproducibility verification. Use when reviewing a colleague's analysis before publication, validating an ML pipeline before production deployment, auditing a report for regulatory or business decision-making, or performing a second-analyst review in a regulated environment.
日本語の概要は準備中です。原文の説明を表示しています。
Analyze pre-trained generative diffusion models (Stable Diffusion, DALL-E, Flux) by computing quality metrics (FID, IS, CLIP score, precision/recall), inspecting noise schedules, extracting and visualizing attention maps, and probing latent spaces. Use when evaluating a pre-trained generative diffusion model's output quality, comparing noise schedule variants, analyzing cross-attention patterns for text-conditioned generation, interpolating between latent codes, or detecting out-of-distribution inputs.
日本語の概要は準備中です。原文の説明を表示しています。
Run a Meta-Harness-style optimization loop NATIVELY — automatically search over the scaffolding around a FIXED base model (memory, retrieval, context construction, prompt templates, summarization, tool-selection logic) by proposing candidate variants, scoring each on a cheap deterministic eval, and keeping a Pareto frontier of quality vs cost — using native Agent / Workflow / loop tools instead of a standalone Python harness. Use this whenever the user wants to optimize, evolve, tune, distill, or search over a harness, scaffold, prompt system, memory or retrieval policy, context-assembly code, or summarizer while keeping the model fixed; whenever they mention Meta-Harness, harness optimization, scaffold evolution, automatic prompt/memory optimization, an evolutionary or Pareto search over candidate implementations, or "make the harness/agent better without retraining"; and whenever the gain must come from the code AROUND the model rather than the model weights. Reproduces the Meta-Harness paper's method natively, with no claude_wrapper.py and no metered solver API.
日本語の概要は準備中です。原文の説明を表示しています。
Analyze catastrophe modeling systems for natural disaster exposure, PML estimation, and reinsurance optimization. Use when: 'assess cat model', 'evaluate disaster exposure', 'review PML calculations', 'audit reinsurance program', 'check exposure data quality', 'analyze hurricane/earthquake risk models', 'evaluate Oasis or RMS setup'.
日本語の概要は準備中です。原文の説明を表示しています。
Routes LLM requests to a local model first (Ollama, LM Studio, llamafile), validates the response quality, and escalates to cloud only when the local result fails. Tracks local vs escalated vs cloud outcomes in a persistent dashboard. Use when: (1) user asks to run a task with a local model first, (2) user wants to reduce cloud API costs or keep requests private, (3) user wants post-outcome quality validation before committing to a local result, (4) user asks to see token savings or the routing dashboard, (5) any request where local-vs-cloud routing should be decided automatically with a quality gate. Supports Ollama, LM Studio, and llamafile as local providers.
日本語の概要は準備中です。原文の説明を表示しています。
Great Expectations, dbt tests, anomaly detection, and data contracts for data quality. Activate on: data quality, data validation, Great Expectations, data contract, anomaly detection, SLA, freshness check, schema validation. NOT for: dbt model structure (use dbt-analytics-engineer), schema evolution (use schema-evolution-manager).
日本語の概要は準備中です。原文の説明を表示しています。
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
日本語の概要は準備中です。原文の説明を表示しています。
Evaluate LLM outputs using frontier models as judges. Use for pairwise model comparison, quality scoring with custom rubrics, and automated evaluation pipelines. Covers position bias mitigation, statistical significance, and generating preference data for DPO/RLHF.
日本語の概要は準備中です。原文の説明を表示しています。
[BETA] Detect, review, and fix code-quality and correctness issues in an AEM as a Cloud Service project — locally, with no external services or network calls. Use whenever a user wants to check, review, assess, audit, scan, modernize, upgrade, or fix AEM Java, Sling Models, OSGi, or Maven code — for example: "check my Sling Models are implemented correctly", "review my @Inject usage", "are my Maven dependencies up to date", "scan this AEM project for issues", "modernize my Sling Models", or "fix code-quality problems". Name the files to assess, or ask it to scan the repo; it detects issues, plans, and — only when you ask — applies surgical edits on a branch or in place, then verifies with mvn compile. It recognises the intent and handles each issue type itself, reporting anything it cannot yet fix. This skill is in beta. Verify all outputs before applying them to production projects.
日本語の概要は準備中です。原文の説明を表示しています。
Great Expectations, dbt tests, anomaly detection, and data contracts for data quality. Activate on: data quality, data validation, Great Expectations, data contract, anomaly detection, SLA, freshness check, schema validation. NOT for: dbt model structure (use dbt-analytics-engineer), schema evolution (use schema-evolution-manager).
日本語の概要は準備中です。原文の説明を表示しています。
Produces a narrow-interval quality signal for a WinDAGs skill that a third party can verify without inspecting model weights or method internals. The mechanism is attribution-kNN over a tamper-evident outcome log (task→skills→accept/reject store), combined with a conformal-prediction calibration certificate that provides a formal coverage guarantee. Thompson/Beta sampling is explicitly rejected as a category error: it produces a selection signal for stationary i.i.d. rewards, not an attestation signal for context-dependent LLM output quality. The resulting bundle — outcome-log Merkle root + conformal threshold + optional TEE guardrail signature — constitutes a "narrow-interval trust" credential: verifiable, bounded, and internals-opaque.
日本語の概要は準備中です。原文の説明を表示しています。