A/B Testing: A Complete Guide | Course Preview

Imagine you've been tasked with improving conversions for an educational technology website. Let's say the company offers an online quiz that gives users personalized product recommendations.

The quiz is meant to help visitors find the right educational product solution. In a handful of qualitative user testing sessions that you did, you found the small group of users who went through the quiz were very happy with the recommendations it provided.

But not many users actually wanted to take the quiz unless they were prompted to do so. You also know the website isn't generating the leads expected from the quiz sign-up, and nobody yet knows why or how to fix the issue. This is where you come in.

Since more user testing probably isn't going to tell you why the quiz is ineffective at scale, you now need large amounts of quantitative data to uncover the user pain points and shape a research question. You can then answer that research question through testing. Let's say you begin by digging into the analytics

data. You've set up a conversion funnel report in Google Analytics that shows the percentage of users reaching each step of the quiz.

Right away, something jumps out. There's a steep drop between the first and second step of the quiz funnel.

You look closer. Step one is simply when someone lands on the page. Step two is the key action, clicking the button to start the quiz. But very few users are taking that step.

In fact, about 75% of them leave before clicking into the quiz. That's a huge leak and it's hurting conversions. Aha. You've just found an important clue to help shape your research question. You now want to know what change on the website should be done to get more users clicking

into the quiz section. This question is grounded in real data. It identifies where the breakdown is happening and shows where the opportunity is.

To further explore this question, you next review the heat mapping visualizations already set up on the page.

These show where users are clicking or not clicking. Now, you don't need to be a heat mapping expert to interpret the basics.

The larger and darker colored areas, called hotspots, show where users spend more time or attention.