The reason that software testing is so vital is summed up in this quote: “The bitterness of poor quality remains long after the sweetness of meeting the schedule has been forgotten.” – Anonymous
However, given the vast complexity of mobile app testing in an ecosystem of multiple OSs, browsers, variable bandwidth, display sizes and device capabilities, the following quote may be more appropriate to the current level of testing efforts:
“It’s more about good enough than it is about right or wrong.” – James Bach
The hard truth is that both developers and QA must cover an immense amount of ground to ensure that consumer and business critical apps meet functional, performance, usability and compatibility requirements.
Fortunately, besides the efficiency of cloud-based mobile testing services, there are a number of tried and true best practices that mobile app testing can embrace to ensure effective, efficient testing that is much more than just good enough.
10 Mobile App Testing Best Practices
1. Begin the test plan before development starts.
Testing is no different than programming in that the cost of flaws in requirements and implementation is far less when those flaws are detected earlier rather than later.
2. Define the scope of what will be tested carefully.
Especially with regard to the diversity of the mobile app ecosystem compared to that of PCs, it is critical to reduce options to a supportable scope. It is impossible to test every combination of OS, manufacturer, device model, carrier, delivery method and device configuration let alone versions and sub-versions of these.
3. Automate testing as much as possible.
Automation is essential if the job is ever to be done. During every stage of development, scripting, IDEs, in-house and cloud test frameworks are essential for detecting bugs or design flaws.
4. Match testing tactics to development phases.
Up to the time of post-component integration when the mobile app is more or less whole, device emulators are the most effective and least costly test vehicle. A large selection of emulated devices can run tests in parallel.
As development approaches alpha, real devices within live networks must be employed. To scale this phase without building up costly, complex infrastructure, consider a cloud-based mobile app testing service with facilities for script generation, logging and 24/7 collaboration capabilities is most appropriate.
5. Establish a culture of continuous development, integration, testing and delivery.
In other words, employ agile methodologies throughout. These shorten time gaps between new functionality, testing, bug detection and bug fixes. Less lag leads to improved team cohesion, efficiency and makes necessary course changes easier to implement.
6. Be sure testing accounts for device constraints.
Take into account the severity of operating constraints under which mobile apps work. The amount of memory, CPU horsepower and electrical power are guaranteed to be a fraction of that on a typical laptop or desktop PC. Reducing platform resources to the lowest practical figures during testing can reveal unforeseen edge cases that must be addressed.
7. Test network bandwidth response.
Regardless of the probable differences in performance for a network-aware mobile app that is using WiFi, 3G or 4G connectivity, users are likely to blame the app rather than their connection. Testing can document bottlenecks for which the app can adjust during low bandwidth conditions.
8. Test both client and server sides.
Especially for browser apps with a server side component, test the mobile app under loads on the client, server and both sides at once. This testing should be done in conjunction with network bandwidth testing since there are related performance issues.
9. Do not make UX testing an afterthought.
Usability testing, along with functional, compatibility and performance testing should be implemented as early as possible in the app development cycle. If the user has a lousy experience, it will not matter what technological magic is performed by the code. Most users will reject the app never to return.
More often than not, users interact with software in ways never imagined by the software engineers that created them. Instrumenting user interaction can provide valuable insights that lead to significant improvements for the next app version.
10. Test for universal appeal.
Even for enterprise workforce business apps, localization is essential. Assume that this app has international appeal and test that it supports that scenario. It must support different languages, display local currencies and account for localized versions of popular social networks or regional social sites.
It is a fact that half of mobile app users who encounter a buggy or difficult to use application delete it from the device. Furthermore, their skepticism of future apps from the same provider rises. Although testing never proves the absence of all bugs, it minimizes their occurrence and impact if the correct testing approaches are taken.
Those who skimp on testing still have their apps tested, but instead of it being performed by a team of professionals it is done by end-users. The app consumers, however, will never provide any feedback that could be used to improve app quality.
Rather than view mobile app testing as a necessary evil or the last line of defense in producing a quality product, it should be recognized as equal in importance to that of the application’s programming.