This free A/B test calculator will help you compare two variants of your website, A and B, and tell you the probability that B is better. You can read more about A/B testing in our earlier blog post on the subject. You may also be interested in our Chi-Squared sample size calculator which will help you calculate the minimum sample size needed to run a Chi-Squared test, given an expected standardised effect size.
Assuming you have already run your A/B test, you will have recorded for both variants of your website, A and B, the following data:
How many people have visited that variant
How many people converted on that variant (this could be a click, a purchase, a contact request, or any other desirable outcome)
Simply enter the numbers that came out of your A/B test into the calculator below:
Enter your visitor and conversion data to calculate the probability that Variant B is better than Variant A.
Calculating...
Result
The calculator relies on Bayesian simulations. It simulates 100,000 samples from a distribution and uses this to calculate the approximate probability that B beats A. The conversion rates for both variants are treated as random variables following a Beta distribution.
So that you can understand how the Bayesian A/B test probability is being calculated, the Python code will appear below when you run the Bayesian A/B test calculator.
First you need to install some dependencies:
pip install scipy numpy
# Python code will appear here.
You can also run the code in R:
# R code will appear here.
Looking for experts in Natural Language Processing? Post your job openings with us and find your ideal candidate today!
Post a Job
When can lawyers, litigants in person, and expert witnesses use AI in court documents? In the last few years in the UK, the USA, Canada, Ireland and other jurisdictions, cases have been reported where submissions were made to a court where the author of a document used generative AI tools such as ChatGPT to create those documents. This has wasted court time, resulted in submissions being rejected or even resulted in changes to cost awards.
A person has recently returned from a camping trip and has a fever. Should a doctor diagnose flu or Lyme disease? Would this be any different if they had not mentioned their camping trip? Here’s how LLMs differ from human experts.
How can you predict customer churn using machine learning and AI? In an earlier blog post, I introduced the concept of customer churn. Here, I’d like to dive into customer churn prediction in more detail and show how we can easily and simply use AI to predict customer churn.
What we can do for you