While an experiment is in draft mode, you can think of it as a safe sandbox where you're able to play around with your app to see what elements you can edit/change in your app. This would be the place to verify the way your experiment is set up, the way your experiment looks, and your experiment's behaviour. Here, you can distribute it to your intended segments/audiences and set up your goals until you actually start your experiment.
Keeping in mind, those who have collaborator privileges can only create draft experiments and not actually start experiments. Only project managers and project owners can start experiments. If you need to have your privileges changed, you'd have to ask the Project Owner to change your role to the role of Project Manager in Settings > Team Settings.
While an experiment is active, the experiment is live, and in the wild according to what you set your segmentation to, or ultimately what your intended audience is.
At this point, variations are being distributed and data is being collected, which powers goals and any other pertinent data that you want to watch for in the goals section.
Any new users (e.g. those who just downloaded the app who qualify for an experiment) will be allocated into it appropriately, according to the distribution you set.
When an experiment is in an active state, users will continue receiving their respective variations until you declare a winner or archive an experiment.
If you pause an experiment, this effectively does a few things:
- pauses distribution so that any users who haven't hit their next session and haven't received the experiment yet, will remain out of the experiment until you start the experiment again
- any users who are already in the experiment, will no longer receive the experiment until you restart the experiment again, but will still continue to receive the variation they had previously received if you re-start the experiment again
Restarting the experiment again will put the experiment back into "Active Mode" (see above).
Using the Variation Switcher
An important distinction must be made if you have forced yourself into an experiment through the variation switcher, and you paused the experiment at some point later. It is expected behaviour that you continue to receive that experiment because you forced yourself into the experiment. If you do not want to continue receiving the experiment when you've paused, you can explicitly remove yourself from the experiment in the variation switcher.
Declaring a Winner
Once you declare the winner, all users will be assigned the winning variation. If a user uninstalls your app on their device and then reinstalls it, that user should still receive the winning variation.
You can let it run it indefinitely so that users continue to receive the winning variation, or simply pause it so any new users won’t be allocated into the experiment with the winning variation.
If you want to conclude the experiment, you can archive it.
When you archive an experiment (thereby ending it), your users would go back to their default state before you started the experiment, and users will no longer be receiving any variations. With your new findings, you can use those learnings and hard code it back into the app.