Mobile application testing faces unique challenges compared to PC software quality assurance programs. For effective penetration, mobile apps must perform within a highly fragmented, dynamic device market. Building the necessary in-house expertise and infrastructure to support software running on any device and any network is a daunting task for any organization.
A cloud-based test environment provides the app provider’s QA department with a complete choice of devices, OS platforms, display densities, phone carriers and languages within which they execute their test scripts. Additionally, they allow for scripting/IDE integration and instant worldwide QA and developer collaboration.
Benefits of Cloud Based Mobile Application Testing
Mobile app testing in the cloud allows in-house development and test teams to maintain focus on results without the distraction and expense of maintaining an extensive test bed infrastructure.
There are several advantages that accrue when using a cloud-based testing service:
- Applications run in a secure environment
- The QA asset footprint is minimized
- Resources are scalable to match testing demand
- Parallelized testing saves time
- A wide range of programming languages, scripts and development environments are supported
- All testing modes are available including compliance, security, failover, latency, stress, capacity, performance measurement and more
- Interaction with carrier phone plans and services is included
- Automated test results are collected in logs, screenshots or videos
- QA and development teams have access to tools and results from anywhere
Cloud Based Mobile Application Testing Tools
Perfecto Mobile Continuous Quality Lab
At the center of Perfecto Mobile’s Continuous Quality Lab is their SaaS-based platform called MobileCloud™. This provides access to hundreds of smartphones, tablets, device emulators and test development, automation and collaboration tools.
Real devices on live carrier networks are operated from a browser interface. Test sessions can be recorded in logs or multimedia presentations and shared directly with test or development teams. Test scenarios are developed using online recording of device interactions and keyword scripting. Scripts from one device can be transferred to other devices automatically.
Perfecto’s environment is built on open, REST-based APIs that embed within any development or testing environment regardless of whether the enterprise uses commercial or open source programming tools.
Keynote DeviceAnywhere Cloud
Keynote’s DeviceAnywhere trial version is a free online service that provides 10-minute device sessions on any number of devices.
Testers or developers connect to the DeviceAnyWhere Cloud via the DeviceAnywhere Enterprise Studio app on their PC. Any available device and carrier network can be selected. This real device on a live network is controlled via a fully functional GUI representation of the device.
Applications to be tested, test scripts or instrumentation code can be uploaded directly to the device provided by DeviceAnywhere Cloud. Released applications can be loaded from the platform’s app store or pre-release apps loaded via DeviceAnywhere’s secure upload tool.
The platform also includes test case management, automation tools and direct device management functions. Multiple devices are accessed simultaneously in tabs. Anomalies can be logged individually in video or shared live with other users.
Sauce Labs Automated Mobile Testing
The Sauce Labs mobile web app testing platform is based on the Appium open source project, which interfaces to native Android and IOS automation frameworks. It supports over 500 browser-OS platform combinations. It uses the Selenium testing framework, which enables authoring tests using record and playback tools.
Since tests are run through the use of emulators, throughput is easily scaled. All tests are run within secure VMs that are torn down after each session. No app data is preserved between sessions.
Tests can be written in any popular development language including Ruby, Java, PHP, Node.js, Python or C# from any test framework. When tests complete, a screenshot-by-screenshot log, command-response log or video recordings are provided. These can be shared with team members and used to document bug/fix reports.
Xamarin Test Cloud
For Xamarin cross-platform developers and QA, there is the Xamarin Test Cloud. Using a proprietary test framework called Calabash, any Android or iOS app, native or hybrid, can be fully tested from the user interface downward. The environment is specifically built for Xamarin developed apps in C# and test scripts written in C# within Xamarin Studio.
The test cloud hosts over 1,000 mobile devices on both iOS and Android. Real or emulated devices can be selected. The test tool includes iOS and Android dashboards which run multiple tests. Each dashboard includes overall performance monitoring and displays testing progress.
Each test case has its own script log, device log and screenshot. The latter provides instant feedback on logic or display errors. A special construct can be added to script code via the Xamarin IDE that acts as a super breakpoint. It enables the tester or developer to use an interactive command shell at the breakpoint where detailed investigation of the fault can be performed.
Test cases can be run automatically during nightly Visual Studio builds. Xamarin Test Cloud also integrates with any CI system, Jenkins, TeamCity or TFS.
Cloud-based mobile testing promises vastly improved test efficiency and faster turnaround time for bug detection and fixes. It levels the playing field among mobile software development organizations by relieving them from supporting a testing infrastructure for a combinatorial explosion of device brands, models, OSs and carrier networks. It does all that while providing a secure execution environment and worldwide collaboration capabilities among testers and developers.