「evaluation」の検索結果

Prompt engineering regression test matrix register: baseline outputs, semantic drift thresholds, judge evaluations, and golden dataset.

日本語の概要は準備中です。原文の説明を表示しています。

6 production-ready AI engineering workflows: prompt evaluation (8-dimension scoring), context budget planning, RAG pipeline design, agent security audit (65-point checklist), eval harness building, and product sense coaching.

日本語の概要は準備中です。原文の説明を表示しています。

Build DSPy evaluation harnesses with rich-feedback metrics that are essential for GEPA optimization. Use when writing a metric function, calling dspy.Evaluate, splitting dev/val sets, debugging "why is my optimizer not improving?", or designing CI-ready DSPy eval suites.

日本語の概要は準備中です。原文の説明を表示しています。

TRIGGER THIS when preparing interviews, planning hiring processes, creating interview questions, building evaluation frameworks, developing interviewer guides, or standardizing interview approaches. Builds complete interview kits with competency-based questions, scorecards, rubrics, interviewer training, and structured evaluation protocols.

日本語の概要は準備中です。原文の説明を表示しています。

Design reproducible evaluations for AI agents with representative task sets, explicit rubrics, appropriate graders, baselines, regression gates, and failure analysis. Use when defining agent quality, comparing prompts or models, validating a release, measuring tool-use reliability, investigating regressions, or deciding whether an agent is ready for production.

日本語の概要は準備中です。原文の説明を表示しています。

Think and work like an expert Computer Vision Scientist. Use when a task calls for Computer Vision Scientist judgment. Reasons from image formation, projective geometry (pinhole intrinsics and extrinsics, epipolar/PnP/bundle adjustment, similarity-scale ambiguity), and COCO/LVIS AP mechanics through DINOv3/SigLIP 2 foundation baselines, RF-DETR/YOLO26/SAM 3 models, COLMAP 4/GLOMAP and VGGT geometry, pycocotools/TrackEval/BOP evaluation, and CVPR reporting and EU AI Act limits while treating train–test and pretraining leakage, AP evaluation-setting gaming, preprocessing mismatches (EXIF, BGR, aliased resizing), label noise, and camera-convention and scale errors as first-class failure modes.

日本語の概要は準備中です。原文の説明を表示しています。

Implement comprehensive observability for LLM applications including tracing (Langfuse/Helicone), cost tracking, token optimization, RAG evaluation metrics (RAGAS), hallucination detection, and production monitoring. Essential for debugging, optimizing costs, and ensuring AI output quality. Use when ", llm-monitoring, tracing, langfuse, helicone, cost-tracking, ragas, evaluation, hallucination-detection, prompt-caching" mentioned.

日本語の概要は準備中です。原文の説明を表示しています。

Braintrust is a hosted platform for evaluating and tracing LLM applications: it runs a task over a dataset, scores the outputs, stores each run as an experiment and logs production traces. Use when a user asks to write or run a Braintrust eval, score LLM outputs with autoevals, compare prompt or model versions as experiments, run evals in CI and fail on regressions, trace OpenAI or Anthropic calls with Braintrust, manage evaluation datasets, or use the bt CLI.

日本語の概要は準備中です。原文の説明を表示しています。

Arize is an AI observability platform, and Phoenix is its open-source tracing and evaluation tool for LLM applications. Use this skill when the user wants to trace OpenAI or LangChain calls, run a local Phoenix server, score RAG answers for faithfulness, debug slow or wrong LLM responses, or send OpenTelemetry traces to Arize for production monitoring.

日本語の概要は準備中です。原文の説明を表示しています。

Audit whether a paper's EVALUATION DESIGN actually measures what it claims and whether its reporting is complete — the validity layer family D (experiment-forensics) cannot reach. Three patterns: train/test leakage means the reported score may not measure generalization (HP-EVAL-LEAKAGE — adopts the Kapoor & Narayanan 8-type / 3-category leakage taxonomy; the illegitimate-proxy / sampling-bias / pretraining-contamination subtypes hand off as needs_external_check, naming but NEVER running Oren-2023 exchangeability / Shi-2023 Min-K% / Golchin-2023 Time-Travel / BIG-bench canary); a load-bearing LLM judge is conflicted (same model/family as a compared system) or unvalidated (no human-agreement, no bias control) (HP-JUDGE-VALIDITY); a declared condition/metric is dropped or switched to favor the method, or 'best' is chosen with no held-out set (HP-SELECTIVE-REPORTING). Verdict-bearing at L0/L1 from the DESCRIBED protocol — NOT repo-gated like experiment-forensics; L2 only CONFIRMS against split/preprocessing/result files. A fresh cross-model reviewer (gpt-5.6-sol xhigh, read-only, fresh thread per pass) PROPOSES findings, each span-anchored to a ledger claim_id; tools/adjudicate_findings.py DECIDES the verdict. Leakage and under-reporting are usually HONEST methodological errors — every finding describes a discrepancy to CHECK, never an accusation. An LLM generating GROUND-TRUTH labels is HP-FAKE-GT (experiment-forensics) — routed there, not here. Emits eval-design-forensics.findings.json; computes NO verdict. Detect-only. Triggers: "eval design audit", "evaluation validity", "train/test leakage", "data leakage", "is the score measuring generalization", "LLM judge bias", "is the judge validated", "selective reporting", "cherry-picked results", "评估设计审计", "评测有效性", "数据泄漏", "训练测试集泄漏", "裁判模型有没有验证", "选择性报告".

日本語の概要は準備中です。原文の説明を表示しています。

Architecture Decision Framework workflow skill. Use this skill when the user needs Architectural decision-making framework. Requirements analysis, trade-off evaluation, ADR documentation. Use when making architecture decisions or analyzing system design and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.

日本語の概要は準備中です。原文の説明を表示しています。

AI Engineering Toolkit workflow skill. Use this skill when the user needs 6 production-ready AI engineering workflows: prompt evaluation (8-dimension scoring), context budget planning, RAG pipeline design, agent security audit (65-point checklist), eval harness building, and product sense coaching and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.

日本語の概要は準備中です。原文の説明を表示しています。

Conduct expert heuristic evaluations using Nielsen's heuristics and domain-specific criteria.

日本語の概要は準備中です。原文の説明を表示しています。

Write, run, and analyze structured test suites for Agentforce agents — functional AND security. TRIGGER when: user writes or modifies test spec YAML (AiEvaluationDefinition); runs sf agent test create, run, run-eval, or results commands; asks about test coverage strategy, metric selection, or custom evaluations; interprets test results or diagnoses test failures; asks about batch testing, regression suites, or CI/CD test integration; requests security testing, OWASP LLM Top 10, red-teaming, penetration testing, prompt-injection tests, a security grade, or a vulnerability assessment of an agent. DO NOT TRIGGER when: user creates, modifies, previews, or debugs .agent files (use agentforce-generate); deploys or publishes agents; writes Agent Script code; uses sf agent preview for development iteration; analyzes production session traces (use agentforce-observe); performs a static safety review of .agent file content (use agentforce-generate Section 15).

日本語の概要は準備中です。原文の説明を表示しています。

Manages Neo4j Aura Agents via the v2beta1 REST API — create, list, get, update, delete, and invoke Aura agents backed by an AuraDB instance. Use when configuring Aura Agent tools (CypherTemplate, SimilaritySearch, Text2Cypher), setting system prompts, deploying agents to REST or MCP endpoints, invoking agents with natural language queries, or generating evaluation datasets (max 50 questions) for the Aura Agent evaluation feature. Covers OAuth2 auth, organization/project scoping, tool parameter schemas, and InvokeAgentResponse format. Does NOT cover AuraDB instance provisioning — use neo4j-aura-provisioning-skill. Does NOT cover vector index creation — use neo4j-vector-index-skill.

日本語の概要は準備中です。原文の説明を表示しています。

Used to invoke the PXMeter tool for rigorous quality assessment of biomolecular structure prediction models (e.g., proteins, nucleic acids, small molecules). This skill covers single-sample CIF evaluations and large-scale dataset benchmarks, providing rich result parsing support. Trigger this skill when the user asks to "run PXMeter", "compare PDB/CIF evaluation results of different tools", "check benchmark scores", or asks "how accurate is the structure generated by the model".

日本語の概要は準備中です。原文の説明を表示しています。

Designs and runs reproducible evaluations for AI agents, prompts, tools, skills, and model-backed workflows using realistic datasets, isolated baselines, objective assertions, rubric grading, trajectory analysis, cost/latency tracking, and regression comparison. Use when measuring agent quality, optimizing skill triggering, comparing prompts or models, or gating an AI feature release. Not for ordinary deterministic unit tests.

日本語の概要は準備中です。原文の説明を表示しています。

🇺🇸 KOL Outreach & Influencer Marketing Playbook — Complete SOP from discovery to ROI tracking. Battle-tested with 200+ KOL campaigns at AFFiNE (60k stars). Includes pricing benchmarks, email templates, content packages, platform algorithm guides, and data-driven evaluation frameworks. 🇨🇳 KOL 外联与红人营销实战手册 — 从发现筛选到 ROI 追踪的完整 SOP。基于 AFFiNE（60k stars）200+ 次 KOL 合作实战验证。包含报价基准、邮件模板、内容包模板、各平台算法指南、数据评估体系。 🇯🇵 KOLアウトリーチ＆インフルエンサーマーケティング実践ガイド — 発見からROI追跡までの完全SOP。AFFiNE（60k stars）での200回以上のKOLキャンペーン実績。価格ベンチマーク、メールテンプレート、コンテンツパッケージ、プラットフォームアルゴリズムガイド。 🇰🇷 KOL 아웃리치 & 인플루언서 마케팅 플레이북 — 발굴부터 ROI 추적까지 완전한 SOP. AFFiNE(60k stars) 200+ KOL 캠페인 실전 검증. 가격 벤치마크, 이메일 템플릿, 콘텐츠 패키지, 플랫폼 알고리즘 가이드. Triggers: "KOL outreach" | "influencer marketing" | "KOL campaign" | "influencer outreach" | "creator partnership" | "KOL strategy" | "red人" | "红人营销" | "KOL合作" | "达人合作" | "influencer pricing" | "KOL ROI" | "content creator outreach" | "KOL筛选"

AI Agent Workflow & Skill Architecture Guide — turn expert work into measurable, reusable agent systems. Covers workflow discovery, skill decomposition, harness design, evaluation, human escalation, observability, cost control, and multi-agent orchestration. Includes a measured marketing-delivery case: 15 people × 3–4 weeks reduced to one strategist + AI in 5 days. By Gingiris.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate cyber insurance policies and coverage. TRIGGERS - Use when user needs help with cyber-insurance-evaluation related tasks.

日本語の概要は準備中です。原文の説明を表示しています。

Systematic evaluation of arguments for structure, validity, soundness, and charitable interpretation. Covers premise identification, conclusion extraction, argument mapping, steel-manning, and the distinction between validity (form) and soundness (form plus true premises). Use when assessing an argument, reconstructing an opponent's position fairly, or preparing a rebuttal.

日本語の概要は準備中です。原文の説明を表示しています。

LLM prompt management and evaluation platform. Version prompts, run A/B tests, evaluate with metrics, and deploy with confidence using Agenta's self-hosted solution.

日本語の概要は準備中です。原文の説明を表示しています。

Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.

日本語の概要は準備中です。原文の説明を表示しています。

Review user experience and interface design using Nielsen's heuristics, WCAG 2.1 accessibility guidelines, keyboard and screen reader audit, user flow analysis, cognitive load assessment, and form usability evaluation. Use when conducting a usability review before release, assessing WCAG 2.1 accessibility compliance, evaluating user flows for efficiency, reviewing form design, or performing a heuristic evaluation of an existing interface.

日本語の概要は準備中です。原文の説明を表示しています。