Mobile App Beta Testing: Breaking Down the Basics

Wise developers seldom neglect to test their app’s quality both before and after launch day. Here’s what you’ll need to know.

For Android

In 2013, Google announced both open and closed beta support for all apps via Google’s Developer Console services. If you’re testing an app to discern the stress levels that your servers can handle, an open beta would be ideal. Why? Open betas allow any and all users to find and access your beta – no invite required. Of course, thinking ahead, Google set optional user restrictions so that developers can limit the allowed capacity of their beta tests, preventing floods of users from bursting over a server’s limits. Conversely, google permits developers to invite up to 40,000 Google+ users into their closed betas – only those users with granted permission can find and access the app’s page on the Play Store.

For iOS

The following year, Apple acquired and released Burstly’s beta testing services to the public. Newly named, TestFlight acts as a way for iOS development teams to test their apps through an invitation-only closed (with a modest cap of 25 internal participants per app) and open beta service (with a more generous cap of 200 external users per app). To perform a closed beta, you’ll need to acquire trustworthy testers first. We suggest that our clients:

1. Post on forums and blogs whose audience might find your app appealing
2. Ask friends and family through social media outlets for assistance
3. Pay for or inquire about services that specialize in aggregating competent testers

Interacting with professional-grade testers ensures that feedback will be both precise and constructive. Also, due to the tedious nature of assessment, it’s possible to burn out a small set of testers. If you’re having trouble finding newer and fresher candidates, one of the many benefits of beta testing services is their wealth of available testers. For any sort of testing, consistency in feedback is key, so having access to a bigger set of testers – such as those from an organization of professionals – is an invaluable asset to have while assessing an app.

Promoting your app’s beta through its landing page and requiring all interested testers to apply via signup forums

Use Google Docs or perhaps a dedicated launching page as a forum to identify:

1. The tester’s device
2. Their reasons for signing up or their interest in the app
3. Their emails that can be reached for when the launch date approaches

Signup forums act as a checkpoint of communication between the developer and potential testers, granting the opportunity to ask questions, inform them about feedback expectations, and to collect general user information before launch.

Other things to consider

• Which of your app's features are the least compelling? Remember – the more cluttered the UI, the more likely you'll overlook and never encounter an error.
• The time and resources that are delegated into testing should be proportional to the number of resources you've invested into your app. A single experience-shattering error will be enough to dissuade a sizable fraction of your user base from wrestling with your unresponsive app.
• Gaming apps and difficulty testing: is your app intellectually stimulating on higher difficulties? Do testers feel a sense of fulfillment after completing difficult challenges? Are the easier levels too easy?