10 Considerations for Mobile App Testing (Part 2)

The following is Part 2 of our article on the top 10 Considerations for Mobile App Testing. Click here to read Part 1.

6 – Device Constraints

In general, mobile devices whether they be phones, tablets or wearable devices have severe constraints in terms of power, processing and storage compared to an average PC. Furthermore, their form factors, screens sizes, layouts, platform capabilities and connectivity capabilities vary widely even within a single manufacturer.

Testing any mobile app to its fullest requires pushing the limits of how low you can go in terms of platform resources. Creating an application that can adapt to the richness or paucity of its platform is far better in terms of user acceptance than simply throwing up an error message when resources are inadequate to support nominal usage expectations.

7 – Network Constraints

Other constraints on device performance are less predictable than its physical configuration. The most valuable apps are network-aware and respond consistently regardless of whether the user is connected via 3G, 4G or WiFi even in the face of network jitter and packet loss. If it makes sense, these apps should offer offline functionality as well.

Besides testing connectivity channels, tests to evaluate app functionality and usability across different carrier networks, may expose subtle differences that impact performance. Furthermore, the effects of network congestion, interrupts from text or voice messages, calls and sharing of connectivity with other apps must be assessed.

How well a mobile application responds and preserves state across networks and under varying network constraints is a vital part of the user experience. You can be certain that users will first blame the app for any deficiencies rather than the network.

8 – Back-end Constraints

Any mobile app that has a server side component must take account of latency issues when testing. A series of graded test cases can evaluate the effects of degradation in client-server communication. For hybrid and web-based apps, various browser loads should be imposed also.

If the web server or DB server is implemented by your organization, then separate instrumentation and testing on that side is a requirement. If these are not under your control, then it is still important to test for unacceptable latency and deal with it.

You can take a page from Google’s experience that a latency of ½ to 1 second results in a 20 percent traffic drop to calibrate your understanding of how impatient your app’s users can be. Often latency can be improved by reformatting data, reducing the amount of data over the connection or removing intervening agents or proxies.

9 – Security and Privacy

Gartner projects that 75% of mobile applications available in 2015 will fail fundamental security tests. An even greater number carry privacy risks as well. Most of these issues arise from access management, cross-site scripting and data storage leakage.

Due to the importance of security and privacy to your end users, you owe it to them to incorporate security testing end-to-end in the development process rather than saving it for the final phases or, worse yet, for real users post-release.

Tests cases for five common security defects must be covered:

  • User authentication and the constraints put on usernames, passwords or pin numbers
  • Ensuring data transfers are never done without encryption
  • Exposure of user identifying information to other apps or 3rd-party ad networks
  • Avoidance of storing unencrypted data on the client device
  • The robustness of server-side security including verifying API calls

10 – Regional Differences

The impressive diversity of the mobile app marketplace along the dimensions of OS types, device capabilities and network configurations will hopefully not lead you to de-prioritize supporting the diversity of your end users. Even for enterprise workforce business apps, attention must be given to testing for regional differences in language, culture, currencies and social network preferences in order for your app to gain international appeal.

Many aspects of localization can be contracted via third parties, but time must be allocated for verification of the results. If possible, have native speakers review interface labels and error messages to avoid unintentional misunderstandings. In locations where language is right-to-left, the effects on the UI must be evaluated also.

Business Benefits

Taking the above considerations to heart and applying them even before the next mobile application development project begins pays immediate and long-term dividends to a business’ competitive posture:

  • With agile methodologies, designers, developers, testers and IT all improve their abilities to collaborate and produce the highest-quality products at lower cost
  • Realistically managing the testing scope within the ever-increasing complexity of the mobile device ecosystem is an essential organizational skill for focused effort and profitability
  • A greater emphasis on automation lowers costs and increases the business’ responsiveness to customers and changing market conditions
  • Carefully evaluating all aspects of mobile app testing including functional, performance, compatibility, network, UI/UX and security tests improves the organization’s decision-making dexterity while reducing risk
  • Thorough and efficient testing processes free up resources that can be used for increased product innovation
  • Ensuring end user satisfaction by providing a functional, responsive and safe app experience is a nearly immeasurable aspect of application adoption and the reputation of the company’s products

 

Summary

Mobile device testing presents an array of challenges well beyond those for traditional desktop application testing due to the unparalleled diversity of products, operating environments, networks and inherent device constraints.

Despite these formidable challenges to producing and testing high-quality products, businesses must accept that mobile app users have little tolerance for apps whose functionality, performance or safety do not meet their heightened expectations. The mobile app world is a highly competitive one in which end users always have alternative products available to them.

Businesses up to the challenge of producing the best, most usable mobile app products and highest customer loyalty must adopt a mindset in which app production is an end-to-end, highly collaborative process that comprehends 10 key considerations and incorporates solutions to these at every step of the way. That is a tall order, especially for enterprises that have until now used a manual, waterfall approach to development and accepted gaps in testing coverage.

The benefits to organizations that are able to adopt highly collaborative, continuous delivery models of product development and can embrace test automation to the fullest extent are difficult to overestimate.

Besides producing higher quality software with faster time-to-market, improved quality and customer responsiveness, they will see overall costs decrease. Additionally, companies are able to free resources to advance the most innovative initiatives within the enterprise to further increase its reputation and profits.