With the delivery of Fast-Mode came a few trade-offs. Some of the trade-offs were discussed before the new SDK was delivered, but I want to take this opportunity to ensure we are on the same page. To improve the speed of the response time, Fast-Mode has reduced segmentation capabilities as the delivery of experiments is managed entirely client-side. This means that beyond the core “Web” segmentation option Fast-Mode will ignore all other segmentation filters.
Another important change came to paused and archived experiments. To keep the client-side cache as small as possible, the distribution for an individual client is not saved when an experiment is paused or archived. This means that if you pause or archive an experiment a client that previously received that experiment will be redistributed upon starting the experiment again.
Finally, because segmentation is done client-side with a minimal amount of data delivered in the Client Config from the CDN, there are also limitation on QA options, as I’ll go into in more detail below.
The expected behavior with Fast-Mode is that if an experiment is distributed to the web, it will be distributed to everyone no matter the filters. Between sessions a cookie is maintained on the client, so a user should receive the same experiment/variation combination across sessions. If an experiment is paused or archived, everyone will be removed from that experiment immediately. If a paused or archived experiment is re-started, users will be re-bucketed at random, based on the distribution weights at the time. The functionality for keeping users in variations after pausing or archiving can be added, but given timelines, we left it out of Fast-Mode’s v1.