「evaluators」の検索結果

Guide for evaluation — inline evaluators, Inspect AI integration, and custom evaluators for measuring training progress. Use when the user asks about evaluation, metrics, benchmarks, or how to measure model quality during training.

日本語の概要は準備中です。原文の説明を表示しています。

Apply AlphaGo-style patterns: learned priors plus search, mixed evaluators, self-play, and asynchronous fast/slow loops. Use for intractable search and compute allocation. NOT for basic supervised learning, tractable optimization, or domains without search bottlenecks.

日本語の概要は準備中です。原文の説明を表示しています。

Applies Arbor Hypothesis Tree Refinement to research artifacts with repeatable evaluators, including model training, agent harnesses, data synthesis and benchmark optimization. Uses persistent hypotheses, isolated experiments, evidence propagation and held-out candidate comparison for multi-experiment research runs. Includes a standard-library state manager and guidance for the RUC-NLPIR Arbor CLI.

日本語の概要は準備中です。原文の説明を表示しています。

Build and run evaluators for AI/LLM applications using Phoenix.

日本語の概要は準備中です。原文の説明を表示しています。

Creates, reads, updates, and deletes Arize AI integrations that store LLM provider credentials used by evaluators and other Arize features. Supports any LLM provider (e.g. OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Vertex AI, Gemini, NVIDIA NIM). Use when the user mentions AI integration, LLM provider credentials, create integration, list integrations, update credentials, delete integration, or connecting an LLM provider to Arize.

日本語の概要は準備中です。原文の説明を表示しています。

Generates deep links to the Arize UI for traces, spans, sessions, datasets, labeling queues, evaluators, and annotation configs. Produces clickable URLs for sharing Arize resources with team members. Use when the user wants to link to or open a trace, span, session, dataset, evaluator, or annotation config in the Arize UI.

日本語の概要は準備中です。原文の説明を表示しています。

Handles LLM-as-judge evaluation workflows on Arize including creating/updating evaluators, running evaluations on spans or experiments, managing tasks, trigger-run operations, column mapping, and continuous monitoring. Use when the user mentions create evaluator, LLM judge, hallucination, faithfulness, correctness, relevance, run eval, score spans, score experiment, trigger-run, column mapping, continuous monitoring, or improve evaluator prompt.

日本語の概要は準備中です。原文の説明を表示しています。

Train or fine-tune sentence-transformers models across `SentenceTransformer` (bi-encoder, dense or static embedding model for retrieval, similarity, clustering, classification, paraphrase mining, dedup, multimodal), `CrossEncoder` (reranker, pair scoring for two-stage retrieval / pair classification), `SparseEncoder` (SPLADE, sparse embedding model for learned-sparse retrieval), and `MultiVectorEncoder` (ColBERT / late-interaction, per-token embeddings scored with MaxSim). Covers loss selection, hard-negative mining, evaluators, distillation, LoRA, Matryoshka, and Hugging Face Hub publishing. Use for any sentence-transformers training task.

日本語の概要は準備中です。原文の説明を表示しています。

Train or fine-tune sentence-transformers models across `SentenceTransformer` (bi-encoder, dense or static embedding model for retrieval, similarity, clustering, classification, paraphrase mining, dedup, multimodal), `CrossEncoder` (reranker, pair scoring for two-stage retrieval / pair classification), `SparseEncoder` (SPLADE, sparse embedding model for learned-sparse retrieval), and `MultiVectorEncoder` (ColBERT / late-interaction, per-token embeddings scored with MaxSim). Covers loss selection, hard-negative mining, evaluators, distillation, LoRA, Matryoshka, and Hugging Face Hub publishing. Use for any sentence-transformers training task.

日本語の概要は準備中です。原文の説明を表示しています。

Use when measuring or improving agent quality and performance — set up evaluators, online monitoring, CI/CD quality gates, observability, or cost optimization. Triggers on: "evaluate my agent", "add evaluator", "measure quality", "quality gate", "run evals", "agent too slow", "why is it slow", "reduce latency", "set up observability", "CloudWatch dashboard", "how much does my agent cost", "cost optimization", "logs not showing up", "logs missing", "spans not found", "eval failing", "eval error", "dev traces", "local traces", "agentcore dev traces", "traces to CloudWatch". Not for debugging errors or crashes — use agents-debug. Slow but correct routes here; broken routes to debug.

日本語の概要は準備中です。原文の説明を表示しています。

Builds, configures, debugs, and optimizes AWS observability - operator-symptom questions and detecting Omni vs classic CloudWatch. CloudWatch on an already-reporting service: Log Insights, metric/composite/anomaly alarms, custom metrics/EMF, dashboards, X-Ray/ADOT tracing, canaries, CloudTrail, Dynamic Instrumentation (live breakpoints/snapshots), the Application Signals service map, and fleet health views. CloudWatch Omni on an existing Space: SQL over logs and traces, PromQL over metrics, Omni dashboards, Omni alerts, context graph for root cause, programmatic/IaC access (API/SDK/CLI/CloudFormation), driving Omni from a coding agent or skills, and evaluating AI agent quality from traces - on-demand and online scoring of live traffic, readback, custom evaluators. For first-time setup - creating an Omni Space, granting access, ingestion, or ADOT instrumentation for Application Signals (ServiceEvents, CI/CD metadata) or Omni - use setting-up-cloudwatch-observability. Not for app logging or threat detection.

日本語の概要は準備中です。原文の説明を表示しています。

Calibrate an LLM judge against human labels using data splits, TPR/TNR, and bias correction. Use after writing a judge prompt (write-judge-prompt) when you need to verify alignment before trusting its outputs. Do NOT use for code-based evaluators (those are deterministic; test with unit tests per `write-code-eval`).

日本語の概要は準備中です。原文の説明を表示しています。

Design LLM-as-Judge evaluators for subjective criteria that code-based checks cannot handle. Use when a failure mode requires interpretation (tone, faithfulness, relevance, completeness). Do NOT use when the failure mode can be checked with code (regex, schema validation, execution tests); use `write-code-eval`. To validate an existing judge, use `validate-evaluator`.

日本語の概要は準備中です。原文の説明を表示しています。

Write code evaluators for known failure modes with objective rules. Use when code can check the rule from a trace, with or without a reference answer. Use `write-judge-prompt` when the rule requires interpretation.

日本語の概要は準備中です。原文の説明を表示しています。

Build a production behavior-tree runtime (Blackboard, action/condition leaves, sequence/selector/parallel composites, decorators) and a Utility AI system (response curves — linear, exponential, sigmoid, quadratic — considerations, and action evaluators), plus hybrid BT-drives-Utility agents. Use when implementing a reusable behavior-tree or utility-based decision system, or tuning enemy/NPC decisions beyond a simple FSM, or when the user mentions behavior tree, blackboard, decorator, selector, sequence, tick status, utility AI, response/scoring curve, or consideration. For choosing between FSM/BT/steering or for pathfinding, use game-ai; for Unreal's BehaviorTree/Blackboard assets, use unreal-behavior-trees.

日本語の概要は準備中です。原文の説明を表示しています。

Use when packaging ACM CCS artifacts for the artifact-evaluation committee and the ACM badges — Artifacts Available, Artifacts Evaluated Functional, Artifacts Evaluated Reusable, and Results Reproduced — covering what security evaluators inspect, how to make attacks and defenses turnkey, and how to justify withheld artifacts.

日本語の概要は準備中です。原文の説明を表示しています。

Suede-owned comparison-page discipline for accurate, evidence-backed alternative, versus, and competitor-comparison content that wins evaluators and search intent. Use when planning or writing a public page that positions products against named alternatives from verified evidence. NOT FOR: gathering the underlying competitor evidence (use suede-competitor-profiling), internal battle cards (use suede-sales-enablement), or scaled page generation (use suede-programmatic-seo).

日本語の概要は準備中です。原文の説明を表示しています。

Create and maintain AgentV YAML evaluation files for testing AI agent performance. Use this skill when creating new eval files, adding eval cases, or configuring custom evaluators (code validators or LLM judges) for agent testing workflows.

日本語の概要は準備中です。原文の説明を表示しています。

You are an expert in creating competitor comparison and alternative pages. Your goal is to build pages that rank for competitive search terms, provide genuine value to evaluators, and position your product effectively.

日本語の概要は準備中です。原文の説明を表示しています。

Use when user says "simmer this", "refine this", "hone this", "iterate on this", or asks to improve a specific artifact over multiple rounds. Runs an iterative refinement loop with investigation-first judges that read the code, understand the problem, and propose evidence-based improvements. Auto-selects single judge or multi-judge board based on complexity. Works on any artifact type: documents, prompts, specs, emails, creative writing, API designs, pipelines, codebases. Supports multi-file workspace targets, runnable evaluators, and open-ended optimization (model selection, pipeline topology, prompt tuning).

日本語の概要は準備中です。原文の説明を表示しています。

Train or fine-tune sentence-transformers models across `SentenceTransformer` (bi-encoder, dense or static embedding model for retrieval, similarity, clustering, classification, paraphrase mining, dedup, multimodal), `CrossEncoder` (reranker, pair scoring for two-stage retrieval / pair classification), `SparseEncoder` (SPLADE, sparse embedding model for learned-sparse retrieval), and `MultiVectorEncoder` (ColBERT / late-interaction, per-token embeddings scored with MaxSim). Covers loss selection, hard-negative mining, evaluators, distillation, LoRA, Matryoshka, and Hugging Face Hub publishing. Use for any sentence-transformers training task.

日本語の概要は準備中です。原文の説明を表示しています。

Build and run evaluators for AI/LLM applications using Phoenix.

日本語の概要は準備中です。原文の説明を表示しています。

INVOKE THIS SKILL for LLM-as-judge evaluation workflows on Arize: creating/updating evaluators, running evaluations on spans or experiments, tasks, trigger-run, column mapping, and continuous monitoring. Use when the user says: create an evaluator, LLM judge, hallucination/faithfulness/correctness/relevance, run eval, score my spans or experiment, ax tasks, trigger-run, trigger eval, column mapping, continuous monitoring, query filter for evals, evaluator version, or improve an evaluator prompt.

日本語の概要は準備中です。原文の説明を表示しています。

Grade and steer UI work using Anthropic-style design dimensions—coherence, originality, craft, and usability—for generators and evaluators.

日本語の概要は準備中です。原文の説明を表示しています。