A/B Testing

Online Course

A/B Testing

What is the course about?

A/B Testing
The course A/B Testing is an online class provided by Udacity. The skill level of the course is Intermediate. It may be possible to receive a verified certification or use the course to prepare for a degree.

This course will cover the design and analysis of A/B tests, also known as split tests, which are online experiments used to test potential improvements to a website or mobile application. Two versions of the website are shown to different users – usually the existing website and a potential change. Then, the results are analyzed to determine whether the change is an improvement worth launching. This course will cover how to choose and characterize metrics to evaluate your experiments, how to design an experiment with enough statistical power, how to analyze the results and draw valid conclusions, and how to ensure that the the participants of your experiments are adequately protected.

Course description
  • A/B Testing
  • 1 months
  • Online Experiment Design and Analysis
  • Overview of A/B Testing
  • Policy and Ethics for Experiments
  • Choosing and Characterizing Metrics
  • Designing an Experiment
  • Analyzing Results
  • This lesson will cover what A/B testing is and what it can be used for.
  • How to construct a binomial confidence interval for the results.
  • How to decide whether the change is worth the launch cost.
  • How to make sure the participants of your experiments are adequately protected.
  • What questions you should be asking regarding the ethicality of experiments.
  • The four main ethics principles to consider when designing experiments.
  • Learn techniques for brainstorming metrics.
  • What to do when you can’t measure directly.
  • Characteristics to consider when validating metrics.
  • How to choose which users will be in your experiment and control group.
  • When to limit your experiment to a subset of your entire user base.
  • Design decisions affect the size of your experiment.
  • How to analyze the results of your experiments.
  • Run sanity checks to catch problems with the experiment set-up.
  • Check conclusions with multiple methods, including a binomial sign test.
  • This course requires introductory knowledge of descriptive and inferential statistics. If you haven’t learned these topics, or need a refresher, they are covered in the Udacity courses Inferential Statistics and Descriptive Statistics.
  • Prior experience with A/B testing is not required, and neither is programming knowledge.
  • See the Technology Requirements for using Udacity.
  • A/B testing, or split testing, is used by companies like Google, Microsoft, Amazon, Ebay/Paypal, Netflix, and numerous others to decide which changes are worth launching. By using A/B tests to make decisions, you can base your decisions on actual data, rather than relying on intuition or HiPPO’s – the highest paid person’s opinion! Designing good A/B tests and drawing valid conclusions can be difficult. You can almost never measure exactly what you want to know (such as whether the users are “happier” on one version of the site), so you need to find good proxies. You need sanity checks to make sure your experimental set-up isn’t flawed, and you need to use a variety of statistical techniques to make sure the results you’re seeing aren’t due to chance. This course will walk you through the entire process. At the end, you will be ready to help businesses small or large make crucial decisions that could significantly affect their future!

Prerequisites & Facts

A/B Testing

Course Topic

Computer Science, Programming

University, College, Institution


Course Skill Level


Course Language


Place of class

Online, self-paced (see curriculum for more information)



Degree & Cost

A/B Testing

To obtain a verified certificate from Udacity you have to finish this course or the latest version of it, if there is a new edition. The class may be free of charge, but there could be some cost to receive a verified certificate or to access the learning materials. The specifics of the course may have been changed, please consult the provider to get the latest quotes and news.
A/B Testing
provided by Udacity


Share your experience

A/B Testing
Udacity Udacity
Rate the course

Do you recommend the course? *
Here you can find information, reviews and user experiences for the course “A/B Testing“. The provider of the course – “Udacity” – will be glad to answer any questions you may have about the class, click here to use the offical support channels. It would be great if you could share your experience of participating in the course – Your honest review will surely help others to choose the right class!
School: Udacity
Topic: Computer Science, Programming