This free A/B test calculator will help you compare two variants of your website, A and B, and tell you the probability that B is better. You can read more about A/B testing in our earlier blog post on the subject. You may also be interested in our Chi-Squared sample size calculator which will help you calculate the minimum sample size needed to run a Chi-Squared test, given an expected standardised effect size.

Assuming you have already run your A/B test, you will have recorded for both variants of your website, A and B, the following data:

How many people have visited that variant
How many people converted on that variant (this could be a click, a purchase, a contact request, or any other desirable outcome)

Simply enter the numbers that came out of your A/B test into the calculator below:

Bayesian A/B Test Calculator by Fast Data Science

Enter your visitor and conversion data to calculate the probability that Variant B is better than Variant A.

Variant A Data

Visitors (A)

Conversions (A)

Variant B Data

Visitors (B)

Conversions (B)

How does it work?

The calculator relies on Bayesian simulations. It simulates 100,000 samples from a distribution and uses this to calculate the approximate probability that B beats A. The conversion rates for both variants are treated as random variables following a Beta distribution.

Python code

So that you can understand how the Bayesian A/B test probability is being calculated, the Python code will appear below when you run the Bayesian A/B test calculator.

First you need to install some dependencies:

pip install scipy numpy

# Python code will appear here.

R code

You can also run the code in R:

# R code will appear here.

Data science consultingAi for business

Oct 18, 2025

A/B testing

See also: Fast Data Science A/B test Calculator (Bayesian) A/B testing is a way you can test two things, Thing A, and Thing B, to see which is better. You most commonly hear about A/B testing in the context of commercial websites, but A/B testing can be done in a number of different contexts, including offline marketing, and testing prices.

Explainable aiExplainable ai for businesses

Oct 10, 2025

Explainable AI for Businesses

Explainable AI for Businesses Guest post by Vidhya Sudani Introduction AI is moving rapidly and it can be hard to understand how an AI model works and what decisions it makes. Businesses are increasingly turning to Explainable AI (XAI) to demystify the “black box” nature of traditional machine learning models.

Natural language processingGenerative ai

Sep 30, 2025

How to spot AI-generated text?

When you receive an email or document written by somebody it can be hard to work out if they use generative AI. There can be giveaways. For example, if that individual has written a document in a different style from their usual writing. Occasionally I have received messages with the prompts left in, such as “I really enjoyed working with you on [insert name of project]”. Around the world, students are finding themselves accused of using AI to write their dissertations, and finding themselves in quasi-legal proceedings in their university where it is impossible to prove conclusively whether generative AI was used to write a document.

A/B test calculator (Bayesian)