A/B Testing

A/B testing (experiments) lets you run experiments on your site to compare different variants and determine which performs better, using chi-squared significance testing.

Setting Up Experiments

To run experiments, track experiment events using our tracking script:

<script
  src="https://your-instance.com/api/script.js"
  data-site-id="YOUR_SITE_ID"
  defer
></script>

// Track an experiment impression
window.rybbit('track', 'experiment', {
  experimentName: 'home-hero-test',
  variantName: 'variant-a'
});

// Track a conversion for the experiment
window.rybbit('track', 'experiment-conversion', {
  experimentName: 'home-hero-test',
  variantName: 'variant-a',
  conversionEvent: 'signup'
});

Experiments Dashboard

The experiments page shows all your active experiments.

Experiment Selector

Choose an experiment from the dropdown to view its results. Experiments are auto-detected from tracked data.

Conversion Event Picker

Select which conversion event to measure:

• Purchase, Signup, Subscription, Checkout, Login, Trial Start

Variant Comparison Table

For each variant you'll see:

• Impressions — How many times the variant was shown
• Conversions — How many resulted in the selected conversion event
• Conversion Rate — Percentage of impressions that converted

Statistical Significance

Results include chi-squared significance with Yates' correction. When a variant reaches statistical significance, it's highlighted as the winner.

Named vs Unnamed Experiments

• Named experiments — Use experimentName to track specific experiments
• Unnamed experiments — Defaults to 'default' experiment name

API Access

Experiment data is available via the public API:

• GET /api/v1/sites/:siteId/experiments — List all experiments
• GET /api/v1/sites/:siteId/experiments/:name — Per-experiment detail