A/B testing is a method where you test two different versions of a feature or product. This is typical on marketing and research. When using this method in software development you have to versions, for example a website.

A, could be the existing website (if you have any) and A is typical the control version where you test other versions against.

And the other version B is the new design or feature. Then you randomly split traffic between this two “A/B” versions and measure the outcome trough predefined metrics and hypothesis.

After the test is completed is time to analyze the result from the experiment. When doing you can then deiced which solution is preferred by the users and with real data backing up the decisions.

Make sure to run the test simultaneously and not after each other.

Steps to start with A/B test

  • Collect data where the test should be created
  • Define goals and metric with the test
  • Create hypothesis of what and why something should be a better solution. Try Lean Startup.
  • Design different versions to test
  • Runt the experiment using some available tools
  • Analyze the date when the test are finished.

Read More:


Thanks to Jimmy Janlén for letting me use his pictures from Agile Topics Cards.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s