A/B testing is a method where you test two different versions of a feature or product. This is typical on marketing and research. When using this method in software development you have to versions, for example a website.
A, could be the existing website (if you have any) and A is typical the control version where you test other versions against.
And the other version B is the new design or feature. Then you randomly split traffic between this two “A/B” versions and measure the outcome trough predefined metrics and hypothesis.
After the test is completed is time to analyze the result from the experiment. When doing you can then deiced which solution is preferred by the users and with real data backing up the decisions.
Make sure to run the test simultaneously and not after each other.
Steps to start with A/B test
- Collect data where the test should be created
- Define goals and metric with the test
- Create hypothesis of what and why something should be a better solution. Try Lean Startup.
- Design different versions to test
- Runt the experiment using some available tools
- Analyze the date when the test are finished.
- The Ultimate Guide To A/B Testing
- What is A/B Testing
- Google Website Optimizer
- Visual Website Optimizer