My RequestsAt Userlytics, we make it easy to test different parts of your product to help you make smarter decisions. One simple and popular way to compare designs or features is through A/B testing.
What is A/B Testing?
A/B testing is when you compare two versions of something—like a webpage—to see which one works better. By showing users two options (Variant A and Variant B), you can find out which one gets better results, like more clicks.
Let’s go over how to set up A/B testing on the Userlytics platform.
How To Set Up A/B Testing On Userlytics
There are two ways to run A/B testing on the Userlytics platform.
Method 1: Clone Studies to Test Multiple Variants
Let’s say you want to test two designs for your homepage. You have two screenshots, one for each version, and you want to see how users interact with them. You can track where they click and create heatmaps to see their activity.
Here’s how you do it:
Set Up Your Study With Variant A
Start by creating a study with your first design (Variant A). If you need help creating a study, we’ve got a step-by-step guide.
When you reach the ‘Activities’ tab, you can choose from several pre-made activity types. If you want to compare two screenshots of an asset, you can select “First Click Testing” to see where users click first or use the “X-Second Test” to see how they react within the first seconds of exposure to your design.
Let’s go with “First Click Testing” for this example.
Simply drag and drop your Variant A screenshot into the highlighted box. On the left side, you can add a question for participants.
Clone The Study and Switch to Variant B
After setting up Variant A, go back to your study dashboard and clone it. In the duplicated study, replace the Variant A screenshot with Variant B. Now, both versions will be tested with the same target demographic, making it easier to compare results.
Userlytics automatically prevents the same participant from joining both studies within 90 days for moderated studies and 15 days for quantitative studies, so you don’t have to worry about duplicate participants.
The advantage of this method is that you get two separate reports that you can compare side by side. If you need to test more than two variants, simply duplicate the study as many times as needed.
Method 2: Test Two Variants Within the Same Study
If you want your audience to see both versions in one study, you can use the same activity twice. For example, you can add the "First Click Testing" activity to your builder twice—once for Variant A and once for Variant B.
Step 1: Set Up Your First Study
Start by introducing the test with a scenario and outlining your goals. In this case, you'll show Variant A first, then Variant B.
Step 2: Ask Follow-Up Questions for Variant A
After showing Variant A, ask participants questions like: "What do you think of the design or copy?" or "Is there anything you would change?"
Step 3: Repeat for Variant B
Once Variant B is presented, ask the same questions so you can compare feedback across both variants.
Step 4: Ask Final Comparison Questions
After participants have seen both, ask deeper comparison questions like:
- "What stood out to you the most?"
- "Which version would you choose, and why?"
- "Which design do you prefer, and why?"
To dig deeper, consider adding activities like the SUS (System Usability Scale) in order to rate each version.
Step 5: Duplicate the Study and Reverse the Order
When the first study is done, use the ‘Clone’ feature on your dashboard to duplicate it. In the cloned study, show Variant B first and Variant A second to minimize order bias.
Step 6: Launch Both Studies
Now that both studies are set up, launch them at the same time and give participants about 30 minutes to complete each one.
And that’s how you can easily conduct A/B testing on the Userlytics platform! If you want to streamline the process, check out this free template to get started faster.