本文へ移動
cccskills
無料GitHub で公開

a-b-test-design

Design rigorous A/B tests with hypotheses, variants, metrics, and sample size calculations.

Infrasity-Labs/dev-gtm-claude-skills
インストール方法を見る

含まれるファイル（1）

  • SKILL.md1.7 KB

SKILL.md（原文）

インストールする前に、エージェントに与えられる指示の中身を確認できます。

A/B Test Design

You are an expert in designing rigorous A/B experiments that produce actionable results.

What You Do

You design A/B tests with clear hypotheses, controlled variants, appropriate metrics, and statistical rigor.

Test Structure

1. Hypothesis

Structured as: 'If we [change], then [outcome] will [improve/decrease] because [rationale].'

2. Variants

  • Control (A): current design
  • Treatment (B): proposed change
  • Keep changes isolated — test one variable at a time

3. Primary Metric

The single most important measure of success. Must be measurable, relevant, and sensitive to the change.

4. Secondary Metrics

Supporting measures and guardrail metrics to detect unintended consequences.

5. Sample Size

Based on: minimum detectable effect, baseline conversion rate, statistical significance level (typically 95%), and power (typically 80%).

6. Duration

Run until sample size is reached. Account for weekly cycles (run in full weeks). Minimum 1-2 weeks typically.

Common Pitfalls

  • Peeking at results before completion
  • Too many variants at once
  • Metric not sensitive enough to detect change
  • Sample size too small
  • Not accounting for novelty effects
  • Ignoring segmentation effects

When Not to A/B Test

  • Very low traffic (insufficient sample)
  • Ethical concerns with withholding improvement
  • Foundational changes that affect everything
  • When qualitative insight is more valuable

Best Practices

  • One hypothesis per test
  • Document everything before starting
  • Don't stop early on positive results
  • Analyze segments after overall results
  • Share learnings broadly regardless of outcome

レビュー

まだレビューはありません。使ってみた感想をお寄せください。

ログインしてレビューを書く

同じリポジトリのスキル

概要と使いどころ

ab-testing

無料

When the user wants to plan, design, or implement an A/B test or experiment, or build a growth experimentation program. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "should I test this," "which version is better," "test two versions," "statistical significance," "how long should I run this test," "growth experiments," "experiment velocity," "experiment backlog," "ICE score," "experimentation program," or "experiment playbook." Use this whenever someone is comparing two approaches and wants to measure which performs better, or when they want to build a systematic experimentation practice. For tracking implementation, see analytics. For page-level conversion optimization, see cro.

日本語の概要は準備中です。原文の説明を表示しています。

Infrasity-Labs/dev-gtm-claude-skills1362026年6月29日 更新

academic-cv-builder

無料

Format CVs for academic positions with publications, grants, and teaching

日本語の概要は準備中です。原文の説明を表示しています。

Infrasity-Labs/dev-gtm-claude-skills1362026年6月29日 更新

accessibility-audit

無料

Conduct a comprehensive accessibility audit against WCAG guidelines with severity ratings and remediation steps.

日本語の概要は準備中です。原文の説明を表示しています。

Infrasity-Labs/dev-gtm-claude-skills1362026年6月29日 更新

accessibility-test-plan

無料

Create accessibility testing plans covering assistive technologies and WCAG criteria.

日本語の概要は準備中です。原文の説明を表示しています。

Infrasity-Labs/dev-gtm-claude-skills1362026年6月29日 更新

ad-creative

無料

When the user wants to generate, iterate, or scale ad creative — headlines, descriptions, primary text, or full ad variations — for any paid advertising platform. Also use when the user mentions 'ad copy variations,' 'ad creative,' 'generate headlines,' 'RSA headlines,' 'bulk ad copy,' 'ad iterations,' 'creative testing,' 'ad performance optimization,' 'write me some ads,' 'Facebook ad copy,' 'Google ad headlines,' 'LinkedIn ad text,' or 'I need more ad variations.' Use this whenever someone needs to produce ad copy at scale or iterate on existing ads. For campaign strategy and targeting, see ads. For landing page copy, see copywriting.

日本語の概要は準備中です。原文の説明を表示しています。

Infrasity-Labs/dev-gtm-claude-skills1362026年6月29日 更新

ads

無料

When the user wants help with paid advertising campaigns on Google Ads, Meta (Facebook/Instagram), LinkedIn, Twitter/X, or other ad platforms. Also use when the user mentions 'PPC,' 'paid media,' 'ROAS,' 'CPA,' 'ad campaign,' 'retargeting,' 'audience targeting,' 'Google Ads,' 'Facebook ads,' 'LinkedIn ads,' 'ad budget,' 'cost per click,' 'ad spend,' or 'should I run ads.' Use this for campaign strategy, audience targeting, bidding, and optimization. For bulk ad creative generation and iteration, see ad-creative. For landing page optimization, see cro.

日本語の概要は準備中です。原文の説明を表示しています。

Infrasity-Labs/dev-gtm-claude-skills1362026年6月29日 更新

Infrasity-Labs のスキルをすべて見る

このスキルの問題を報告する