Share

The Mobile Testing Challenge

Posted by Tara Weber on February 11, 2013

Challenge
Mobile adds new complexities to developing high-quality solutions. A very large number of phones, screen sizes and versions of mobile operating systems exist, and each is unique.

For example there are hundreds of Android phones, all customized slightly like the HTC Sense, and each release updates on its own timetable. These complexities along with various operating systems, including iOS, Blackberry and Windows, combine to create a vast amount of variables and unique situations with which to contend.

The 101 Solution
By utilizing a combination of experience, cool technology, automated testing and unit testing, this challenge can be addressed. Below are the “101” solutions to this problem. Much like peeling layers of an onion, these will increase quality and take you to the next level. At that point, other opportunities will present themselves, leading to further improvement.

Iterative Development
Smaller iterations break up a larger problem into discrete sets of work that can be more easily tackled following a scrum approach. This allows you to focus on the immediate work at hand and complete that before moving on to the next area. Among other things, it reduces the time from when a bug is introduced to when it is fixed, resulting in more efficient solutions. There are many articles written about the benefits of agile and scrum. Mobile, due to its maturity, benefits enormously from an agile approach. Between starting a project and ending a project, new smartphones and even new operating systems can be released, which changes how things need to be done. Therefore, there is a need to match the rapid changes in the industry to an agile development approach.

Test Early
By testing early and often, we can find the difficult errors on devices quickly and efficiently. Key areas here are having the testers involved from the start and having good unit tests written that check each independent unit of the code. By having testing involved early in the process, technical debt and bugs don’t accumulate. As the diagram below shows, having more unit tests finds issues earlier and fixes them faster. This is especially true in complex systems, such as mobile, where an issue found in integration testing may have several potential root causes that must be explored independently to find the one that is causing the error.

Automation
By automating key parts of the development cycle from builds to testing across devices, we can test more quickly and more often. The key areas to automate are the build process through continuous integration and unit tests across multiple phones. By automating the build process, we can build for many platforms (iOS, Android, etc.) quickly and easily each time new development is done. We can then run automated tests across a variety of phones to verify that the new development didn’t break anything existing and produced the desired results on all phones. This addresses the biggest challenge in mobile: device proliferation.

It is important to pick the most commonly used phones on the market. iOS devices are a prominent choice, since Apple controls the devices made. With Android, we can only approximate which phones to focus on by looking at Amazon’s top sellers in its mobile store.