4 Steps to A/B Test Your Way to Mobile App Success

There’s no denying that apps are largely responsible for the incredible success of smartphones. Recent reports show that smartphone owners spend 90% of their mobile time using apps. However, as much as we all love apps, we’re hardly loyal users. We spend most of our time with our favorites and are quick to desert new downloads. Our fickle nature is at least partly the reason for soaring user acquisition costs.

What do supremely popular apps like Facebook and YouTube have in common? Every single element of the app has been tailored to convert, engage, and retain. These elements and features aren’t chosen on a whim. They’re carefully tested and analyzed against key metrics.

For mobile developers, A/B testing provides real answers regarding everything from creatives to in-app purchases. While the list of things you can A/B test is nearly endless, we’ve compiled the most critical elements and divided them into specific stages.  You’ll also find a selection of tools to make testing a breeze. Let’s begin with how to best attract new users to your app.

Stage 1: App Store Page Optimization

Ready to convert from the get-go? Instead of figuring out what works (and what doesn’t) through costly and time-consuming trial and error, optimize your App Store page from the start. With A/B testing, you can identify which elements of your app page are the most attractive to users and create a high converting page. This will help you cut down your user acquisition costs, boost your organic traffic and even perform better in App Store search.

These are just a few of the page elements you’ll want to A/B test:

• Screenshots
• Descriptions
• Icons
• Titles

As for tools, stick to those that provide scientific data and app store page replication like SplitMetrics, Google Experiments, and to some extent App Store Analytics. These tools allow you to get down to the nitty-gritty and fine-tune each element of your page. Find out if a bold color palette attracts or repels users; stop guessing which screenshot is the most appealing and get the final answer with an A/B test.

With specialized tools like SplitMetrics, for instance, you can dive even deeper into oft-forgotten about factors such as search ranking and category performance. Industry benchmarks are one of this platform’s many secret sauces for wiping out the competition.

The Google Experiments framework is designed to test and monitor the effects of changes (to apps or website) on specific goals. You may identify a specific objective and the percentage of users involved in the experiment to produce a random sample. However it only works for Android and has some other limitations.

As you might have guessed, App Store Analytics zeroes in on app store performance. The data is centered around in-app user behavior and engagement tools and will help you further analyze the results of your optimization efforts.

Tools: SplitMetrics, Google Experiments, App Store Analytics

Stage 2: User Acquisition

This next stage is all about bringing users in and making a solid first impression when you get them there. A/B testing is incredibly useful for user acquisition as the results will affect and determine the steps you take in the subsequent stages.

For this second stage, place the following on your A/B testing to-do list:

• Ad networks/publishers
• Targeting/audiences
• Creatives (banners, texts, etc.)
• Bidding modes (CPC, CPM, oCPM, CPI)

To check off each item on your to-list, you’ll need a well-stocked tool kit. Otherwise, you risk gathering a lot of information without real analytics to guide your next steps. Analytics tools like Taplytics, Firebase, and Google Analytics are all designed to help you pinpoint which features are leading to your most desired results.

You’ll be able to validate the success of a particular ad network or bidding mode instead of relying on guestimates and correlation alone. Quantifiable data gives you invaluable insights into your users’ interests, behavior, and engagement levels. Follow up with a tracking and attributions analytics platform such as AppsFlyer to monitor uninstalls from user acquisition campaigns and retarget as a recovery method.

The creatives and messaging that perform best in your UA campaigns will also give you ideas for improving your App Page listing (e.g. you can rework some of your banner images into screenshots). Sometimes it even makes sense to use the tools mentioned in stage 1 like SplitMetrics to optimize your App Store page towards a particular UA channel if it drives a lot of traffic for you.

Tools: Taplytics, Firebase, AppsFlyer, Google Analytics, SplitMetrics

Stage 3: In-App User Flow Optimization

Optimizing user flow can be a tricky process. You need logical funnels to move users through the conversion process. However, unless you’re selling a specific product in your app (think retail), it can be hard to know which steps are most vital to moving your user along. This is where A/B testing comes in. You can analyze the effect the following actions have on your end goal and take in data regarding user behavior from step to step in the conversion process.

• Registration dialogues
• Sales funnel
• Cross-promotion flow
• Review generation and feedback collection

Reach for handy tools such as Taplytics and Firebase Remote Config in this stage and test not only your copy and creatives, but also the triggers and segmentation.  As you form your hypothesis for the right actions to take during each part of the sales funnel, use Taplytics’ visual A/B testing and performance insights to measure your results. You can even set up experiments, analytics, and A/B testing to develop smarter push notifications.

Tools: Taplytics, Firebase Remote Config

Stage 4: Retention

Retention is one of the most critical stages and arguably the most difficult to master. The statistics vary depending on app category, but most publishers can expect to lose somewhere around 77% (or more) of their users three-days post-install.

Retention is a long-tail goal, but it’s one worth pursuing. Engaged users increase profitability and app value. High retention rates mean you can spend less time on retargeting lost users and trying to acquire new ones.

A/B testing will help you identify which actions are keeping users around and which actions may be scaring them off. Here are some important ones to test during this final stage:

• Push notifications: timing, text, event triggers
• Newsletters
• Retargeting

The tools you use here will vary based on your actions. We recommend adding Taplytics, Firebase, Autopilot, and Mailchimp to your starter kit.

Taplytics, features a Mobile Analytics Platform, which provides insights into user acquisition, engagement, and retention. You can analyze not only what your users do and see inside your app but also how they respond as well.  

To segment users into cohorts for targeting and messaging, it makes sense to turn to a platform like Firebase. Firebase tracks the lifetime value and average revenue of your users; it also integrates with AdWords for superior campaign optimization.

For email newsletters and marketing, Autopilot and MailChimp are excellent options. Autopilot’s marketing automation software uses customer journeys to create cohorts and capture leads. A/B testing comes into play for tracking conversion, open, and click thru rates.

MailChimp has become synonymous with email marketing and for good reason. It features more than templates; the solution offers user-targeting tools and compares your data on factors such as open rate with industry standards.

Tools: Autopilot, MailChimp, Taplytics, Firebase

These are just a handful of our favorite tools and  elements for A/B testing. Keep in mind you can and should test as many features as possible to create an app that not only brings users in but keeps them around as well.

Splitmetrics is mobile platform that helps app marketers and developers to mobile marketers to A/B test and optimize iOS and Android app store pages for maximum conversion.