This free A/B test calculator will help you compare two variants of your website, A and B, and tell you the probability that B is better. You can read more about A/B testing in our earlier blog post on the subject. You may also be interested in our Chi-Squared sample size calculator which will help you calculate the minimum sample size needed to run a Chi-Squared test, given an expected standardised effect size.

Assuming you have already run your A/B test, you will have recorded for both variants of your website, A and B, the following data:

1. How many people have visited that variant

2. How many people converted on that variant (this could be a click, a purchase, a contact request, or any other desirable outcome)

Simply enter the numbers that came out of your A/B test into the calculator below:

How does it work?

The calculator relies on Bayesian simulations. It simulates 100,000 samples from a distribution and uses this to calculate the approximate probability that B beats A. The conversion rates for both variants are treated as random variables following a Beta distribution.

Python code

So that you can understand how the Bayesian A/B test probability is being calculated, the Python code will appear below when you run the Bayesian A/B test calculator.

First you need to install some dependencies:

pip install scipy numpy

# Python code will appear here.

R code

You can also run the code in R:

# R code will appear here.