The A/B test is one of the basic ways to quickly verify ideas in mobile applications. The arrangement is typically simple: you have a test for several weeks, measures its effect and decide to open a feature or repeat more.
One of the basic but commonly overlooked aspects of the A/B test is to adjust the correct exposure points. In this article, we will provide some tips and tricks that I have learned along the way, why the open exposure point setting is important, common traps to avoid.
What are the points of exposure already?
A exposure point is the real moment in which the user encounters or interacts with the feature you initially tested. For example, timing when a user sees a new button or sees a redesigned landing page after clicking something.
Why is it important to choose a good exposure point?
If you have looked at the test results of A/B and do not make sense, bad exposure points may be guilty. Bad exposure points can lead to the following:
- Confusing data: It will not be clear whether your feature is a hit or lady.
- Hidden Errors: Everything on the surface may seem well, but serious problems such as application accidents may be shifting under the radar.
- Missed deals: Even if users really enjoy the feature, your data may not have a significant effect wrong.
Bad exposure point
Imagine that you have tested a sales test for a new subscription in your application, but then trigger the exposure point in the application launch after really viewing the subscription page of the subscription.
Problem: Perhaps only 10% of users, 90% of your test data for this decision makes it worthless.
Loose and tight exposure points
Loose exposure points:
They take place a little too early. Users are exposed to experiment before experiencing the tested feature. This early exposure data dilutes and is difficult to find a real effect.
Strict exposure points:
These take place as soon as users experience the tested variant. The data collected with precise exposure points are more accurate and reliable and are easier to analyze.
Which one is better?
It depends on your usage status. Tight exposure points are preferred because they provide cleaner, more defined data even in smaller sample sizes. However, from time to time, strict exposure is not possible. In this case, you can use loose exposure points with information that you will probably need a larger sample size to achieve significant results.
Avoid stacking changes
In a single test, never confuse or stack multiple exposure points. Divide each change into your own A/B test. Although it will take a little more time, your data accuracy will be much larger and you will have more accurate results about each feature.
Fast real life example
Let’s say you add a new type of line in the table view:
- Good exposure point: When the new line is displayed, trigger fully exposure.
- Bad exposure point: Even if the new row has not yet appeared, the table exposure is not loaded.
Latest Tips
- Pre -plan: Consider your exposure points at feature planning time.
- Repeating quickly: If the first test results are not as expected, repeat quickly.
I would like to hear your stories and experiences in determining exposure points for your A/B tests. Share them in comments!
