Free A/B Test Calculator

A/B testing, sometimes referred to as split testing, is a type of test that contrasts two variations of something to detect which version is preferable. It’s often used for a marketing campaign or something similar. In A/B testing, users are randomly selected to receive one of two options, and statistical analysis is conducted based on their feedback.

A/B tests are used in many different industries for different reasons. Marketing and advertising professionals commonly use A/B tests to determine the best option to use in website design, online offers, headlines, emails, and product descriptions.

A/B tests are calculated by comparing the conversion rates of the two different variations against each other to see if one has higher statistical significance than another. Some key factors to consider when calculating your A/B test are z-score, P-value, sample size, confidence interval, and confidence score.

The statistical significance of your A/B test should ideally be 95% and not lower than 90% at the very least. Otherwise, it is not a successful test.

The required sample size depends on the A/B test and what it is measuring. Calculations should be performed to determine the minimum required sample size for the given A/B test.

In A/B testing, statistical significance ensures that the difference between the control and test groups of the experiment is not the result of chance or error. The higher the statistical significance, the more likely that the difference between your groups is accurate. In short, statistical significance is a measure of how probable it is that your null hypothesis is true.

The statistical significance formula is:

χ2= ∑(O−E)2/E

The z-score is a statistical measurement that indicates how much a given value differs from the standard deviation. In simple terms, the z-score shows you how far your tested data point is from your mean.

The P-value is the probability that there is no difference between your test groups and that the indicated improvement is completely by chance.

The null hypothesis refers to the baseline assumption that there is no connection between two sets of data.

The statistical power of a test is the probability that your test will find a statistical difference between the test groups, effectively deviating from the null hypothesis.

A one-sided test can determine if one (and only one) mean is greater or less than another mean. A two-sided test can determine if two means are different from each other.

An A/B confidence score grades the validity of a successful A/B test. Generally, a score above 95% is ideal.

You can run an A/B test by following the steps below:

• Enter the number of visitors and conversions for your A variant.
• Enter the number of visitors and conversions for your B variant.
• Determine your test’s confidence interval.
• Calculate, analyze, and share your results.

You can create your own A/B test with Jotform by starting from scratch or choosing one of our ready-made templates. Create and customize different A/B test forms to test your options and see which is more effective in the eyes of your audience.

Yes, you can customize your A/B test forms with Jotform’s drag-and-drop builder. Add or remove form fields, change up copy, upload branding assets or images, pick fonts and colors, and more depending on your needs.