Testflight from Apple is a program with which beta testing for iOS applications can be performed. It consists of a service, which provides the functions, and an app on the test device that interacts with the user. Testflight is especially interesting from the perspective of developers, because they get faster feedback. Also for companies, the use of Testflight is an advantage, because what helps in development increases productivity.
Testflight has been used by app developers as an beta test platform since several years. Apple bought the service this year in connection with the acquisition of Burstly, the company that originally developed the platform. Apple has redesigned the program and made it available as an integrated iOS 8 service for both developers and testers.
The Testing Process
The testing process begins with the production of a build in the Xcode development environment. The build is transferred to iTunes Connect, which is a program for the management of all tasks connected with the sale of apps. It can be used to upload the app on the App Store, to enter metadata and descriptions, to view statistics on sales and to invite the beta testers per email.
After releasing the beta version and sending the invitations, the new app is distributed over the Testflight service. Prerequisite for the test is only the Testflight app and iOS8 that have to be installed on the test device (smartphone, tablet). The service ensures that the versions are available, informs about new builds and updates, provides the tester with instructions on what to pay attention to and allows feedback.
Up to 25 internal and 1000 external testers can participate. Internal testers are part of the development team in the company that produces the app. They get the test version as it was created by the developers, without prior review by Apple. The condition is that they have an account in iTunes Connect. In addition, they can test on up to 10 devices.
External tester can be anyone, provided he received an invitation. In this case, the test version must have gone through the review process of Apple. A repetition of this process is required when significant changes to the program are made. Each tester – internal or external – can check up to 10 applications simultaneously.
During the test, some automatic feedback takes place, e.g., the number of testers who really participate, or when the app is started for the first time. In addition, individual comments from the testers can be transmitted. The test must be carried out within 30 days, then the version is set to “inactive”.
Mainly, there are two points that make the testing process easy. On the one hand, no UDID has to be entered. The UDID uniquely identifies every Apple device. Dealing with this identification was always a complicated process and had a distinctive tendency to go wrong. By contrast, the process in Testflight is much easier. On the other hand, there is no limitation to 100 devices anymore. Instead, up to 1,000 testers can be invited who identify themselves with their Apple ID. Again, no UDID is necessary.
Some developers don’t like the limitation to iOS8. This means in particular that the backward compatibility cannot be tested with older versions of iOS. In certain situations this is definitely necessary, and could be a reason to use a different test system.
A similar problem is the fact that only the latest version of an app can be tested. Older versions are locked, and the tester cannot go back to them. Again, this may be a significant disadvantage in certain situations.
Currently, Crash Reporting is not integrated, which would help developers a lot. However, since this is to be provided in the next months, this should not be a problem.
It is definitely a good idea to make usage of an extended test phase with external testers before an app is published in the App Store. The bugs are less, and the uncomplicated procedure as well as the greatly expanded range of participants makes it possible to shorten the test phases and to develop new versions faster.
However, Apple’s restrictions may be a real obstacle. In particular, the limitation to iOS8 and the blocking of older versions of a test app is likely to prevent some developers from using Testflight.