Those users will be included. You should be explicit about the segmentation and appVersion that have the changes you want to test and who should ultimately be included in the experiment. In this case, you're just inflating the numbers for each of the variations. Those users who are on appVersions below whatever app version where you've introduced those code variables are essentially getting A/A tests.
What you can do to properly choose your segmentation is to set it equal or greater than appVersion (whatever app version you've introduced the code variable/block to) to include only those users that are relevant to the experiment and have the code to support the test
If you've already started the experiment, we could reset the date of the experiment to essentially restart it.
Comments
0 comments
Please sign in to leave a comment.