Before You Begin
Before you start an experiment, you should have a very clear idea of who belongs in your experiment.
A good question to think about before starting an experiment would be, "Considering the goals I'm trying to accomplish, who should be included in this experiment?"
With that in mind, you'll be able to send the right experience/experiment, to the right user, at the right time in order to test your idea and accomplish your goal!
How to think about your segmentation
You should always be thoughtful about who will be included in your experiment. And remember that users may at some point fall outside of your experiment which could be a result of a variety of factors, in which case they will no longer be a part of the experiment.
A clear example for this is if, say, your experiment was to include only users who are:
- on iOS
- using appVersion = 1.0.0
- have made 0 purchases
If a user made a purchase (so they have 1 purchase), that user will no longer qualify for your experiment.
If a user upgrades to appVersion 1.0.1, that user will no longer qualify for your experiment.
If a user switches to the a new Android device and sells their iOS device, that user will never qualify for the experiment on the other device.
A simple way of thinking about your segmentation is that your filters is a net that you cast to include which users belong in your experiment.