Running your A/B test correctly ensures that you gather reliable, actionable data.

​
Duration of Your A/B Test

​
How Long Should You Run Your Experiment?

Run your test for a duration that allows you to collect enough data to reach statistical significance. Typically, you should aim for at least 1-2 weeks, depending on your traffic volume.
By default, to get an experiment to be significant, you’ll need to run it for at least 14 days. You can contact us to change this parameter.

​
Can You Stop Early?

Avoid stopping your experiments too early. Even if you see promising results after a few days, stopping before reaching significance can lead to unreliable conclusions. Always wait for the experiment to complete its cycle. The longer your experiment, the more reliable the results will be. Consider adjusting your significance parameters by contacting us contact us. By default, Gleef recommand the following metrics to be significant:
  • Duration: 14 days
  • Confidence: 95%
  • Sample size: 1000 visitors per variation (including the baseline)
  • Success events: 50 success events per variation (including the baseline)

​
Avoiding Bias in Your Test

​
How to Avoid Bias

  • Timing: Run the test for more than a week to account for variations in traffic patterns (e.g., weekdays vs. weekends).
  • Consistent Conditions: Ensure that no changes are made to the website during the test, as this could influence the results.
  • Overlapping Tests: Avoid running multiple experiments on the same page simultaneously, as this can skew the results.
Gleef helps you manage multiple experiments on the same page. Learn how here.

​
Running Simultaneous Experiments

You can run multiple A/B tests across different pages simultaneously. However, to ensure the accuracy of each test, avoid running overlapping tests on the same page.