Mobile moderated user testing enables researchers to interact with participants in real time, capturing natural behaviors and mobile-specific insights in authentic environments. Whether you're testing an app, mobile website, or digital prototype, this guide will walk you through the key steps to successfully launching and managing a mobile moderated session.
Userlytics supports mobile moderated testing on both iOS and Android, offering flexibility and accessibility for researchers and participants worldwide. While both platforms provide similar functionality, there are some key distinctions:
Feature | iOS | Android |
Browser Type | Native browser within the app | External browser opens links |
Navigation | Guided navigation inside the app | Floating windows & boxes guide users |
User Flow | In-app experience | External links open in preferred browser |
How Do I Create a Mobile Moderated Study?
- Go to the Userlytics Project and Account Management System (PAM).
- Click "Create Study" in the top right-hand corner.
- Then, you will be prompted to select the team and project to which your study belongs.
- Next, select “Moderated Testing” as the study type.
After clicking “Continue" you'll land on the first page of the study builder. Here, you'll find several sections to fill out to begin creating your UX study.
- First, pick a "Public Name" for your study. This is what your participants will see, so we recommend you to make it reflect the brand and asset you're testing.
- Optionally, you can also set an "Internal Name" for your study. This name will only be visible to you and your colleagues.
- Next, choose the "Max Session Time" and the "Study Language."
After clicking “Continue" you'll land on the first page of the study builder. Here, you'll find several sections to fill out to begin creating your UX study.
- First, pick a "Public Name" for your study. This is what your participants will see, so we recommend you to make it reflect the brand and asset you're testing.
- Optionally, you can also set an "Internal Name" for your study. This name will only be visible to you and your colleagues.
- Next, choose the "Max Session Time" and the "Study Language."
After clicking “Next”, you will be directed to the Participants section of the study builder. Under “Participant Device,” select “Smartphone.”
Next, you’ll be able to select the type of mobile device(s) for your study. You can choose “BOTH,” “iOS,” or “Android.”
When selecting "BOTH" as the device type, participant recruitment will be conducted on a first-come, first-served basis, prioritizing those who qualify the screener first.
This means that if your study requires 10 participants, the distribution may vary based on qualification. E.g., 6 participants using iOS and 4 using Android.
If you require an equal number of participants per device type, you must duplicate the study and configure each version to accept only one device type (iOS or Android). Avoid selecting "Both" in this case to ensure balanced recruitment.
Proceed to choose the “Source of Participants”, number of participants for your study, countries you’d like the participants to come from, gender and age.
How Do I Schedule my Mobile Moderated Study?
To schedule the date and time of your moderated study, click on “Schedule” from within the menu at the top of the study builder. Then, a calendar will appear. You can select the correct timezone by using the drop-down menu in the top right corner.
Now, click on any available time slots within the calendar and simply drag to set time blocks. Note that the drag-to-set calendar feature allows you to set time blocks vertically for each day only, but you can choose as many time slots throughout the week as needed.
When participants are invited to your usability study, they will be able to choose from all the open moderated sessions you have created based on their availability. We highly recommend adding more slots than the number of participants you’re looking to test with, in order to provide more flexibility.
You can set a maximum number of booked sessions per day, customize the buffer time between sessions, and set a minimum advance notice setting to prevent unexpected last-minute sessions.
To invite observers to your study, click on the “Invite Observers” button and invite as many observers as needed. Simply enter the name, e-mail address and click “Send.”
You can have up to 10 observers during your study, although this default setting can be easily increased by contacting our Operations team at operations@userlytics.com.
Once you have added your study screeners (optional) and the activities within your study, you’ll be able to preview your study and how it will appear on mobile.
Video Guide
Notes on Running a Moderated Session on iOS
- Participants will stay within the Userlytics app throughout the session.
- The moderator can guide users seamlessly as they interact with the tested product.
- Links and tasks are displayed directly within the in-app browser, providing a smooth and uninterrupted experience.
Notes on Running a Mobile Moderated Session on Android
- Participants will navigate using floating windows and boxes, which guide them through the process.
- When clicking on "Start" or "Open Link," the system will open the URL in the participant’s default browser.
- This approach mimics the desktop experience, allowing participants to read instructions first before proceeding.
Best Practices for Mobile Moderated Testing
-
Test the session flow on both iOS and Android before launch.
-
Remind participants to enable microphone and screen sharing permissions.
-
Monitor user interactions and adapt questions based on real-time behavior.
-
Allow time for navigation differences between platforms.