Taplytics makes it as easy as possible to running QA on experiments with a number of different options to choose from to fit within your team’s QA workflow. For the purposes of this checklist we will make a suggestion based off of our recommended workflow, but feel free to adjust based off of your own needs.
If you need help building a custom QA process, our Customer Success team would be happy to help!
Helpful Tips & Tools
There are some tools and initial SDK setup tips that can make your life easier in understanding what’s happening “under the hood” so to speak. If you can be armed with these ahead of running through QA, you can save yourself a lot of time.
This guide will not require the use of anything more than the core Taplytics platform and your QA devices, but if you would like help with how to use other tools in your QA process our Customer Success team would be happy to help get you setup.
- Access to the Taplytics Interface
- I know this should be a no-brainer, but sometimes QA users don’t get access to the tool that is generating the push notifications. Making sure you or other QA users have access to your app’s Taplytics project should be the first thing you do.
- Setting up Advanced Pairing
- With Advanced Pairing set up, you can send yourself a custom link to identify your devices to Taplytics. Making them easily discoverable in the Taplytics interface. The docs for Advanced pairing can be found here for iOS and Android.
- Setting a User ID
- If you can have your team set a custom user ID to Taplytics it can be used as another method of finding your device in Taplytics, by searching User Insights. The docs for setting a User ID can be found here for iOS and Android.
- Charles (or some other proxying tool)
- Charles is a network monitoring that shows you the network requests that your device is making. This tool is amazingly useful for letting you know what your device is getting from Taplytics.
- Taplytics BigQuery (if you aren’t using this yet, ask your Account Manager)
- Taplytics BigQuery gives you and your team access to the real-time feed of all of your raw data within Taplytics. If you know SQL you can query the data on the fly to see what events and data Taplytics is logging.
- Require each QA user to have their own user account for QA.
- Ensure that there is only one build of the app running on a QA device at a time.
- Log in to your app at the beginning of each session.
- Log out of your app at the end of each session.
- Always log out of your account before deleting a build of the app.
QA of On-Device Experience:
- Ensure your experiment and variations are properly set up and your experiment is in Draft
- Install the most recent build of your app onto your test device
- Find your specific device in Taplytics
- If you’re using Advanced Pairing send yourself a link and find your device on the Device Settings page
- Or search for your device in Taplytics User Insights by using your user ID
- Navigate to your user profile on User Insights
- Select your specific QA device in the dropdown of the Variation Switcher
- Add your device to the draft experiment you are running QA for
- Apply the variation for the experiment
- If your app was open force close it
- Open your app
- Navigate to the area of the app where the experiment should be and observe the change
- Go back to the Variation Switcher on Taplytics User Insights
- Apply the baseline variation
- Force close your app
- Open your app
- Navigate to the area of the app where the experiment should be and observe the experiment in baseline
- Repeat the application and observation stages as many times as necessary and try other ways of navigating to the area of your app where the change should be applied
If you're blocked on QA because you have multiple devices in the QA device dropdown menu (shown below), here are some steps you can take to be able to ensure that you're choosing the right device to be able to force it into a experiment and variation that you're QA'ing.
- Clear any app data
- Reset your device's advertising id
- delete the build of the app that you're testing
- Reinstall the app
From there, you'll want to find your device to be able to force it in the right variation and see if the proper responses are coming down through Charles.
You can do that in two ways:
1. Pairing your app and finding it in your Device Settings
2. Finding your device in the User Insights in the lefthand navigation panel. There you can use a combination of filters to pinpoint your device.
Once you find your device, you can use the variation switcher to manually force a device into an experiment and the desired variation.
Let us know if you have any issues with QA by emailing firstname.lastname@example.org or if you have a private slack channel that was already set up for you, message us there! :)