If you’re reading this, you probably already understand the importance of, and why you need mobile device testing to be a part of your mobile testing strategy. If you’d like some tips, here’s a great article on mobile device testing strategy.
So once you have your plan, now what? Well, you’ll need to decide how you’re going to go about testing and with what testing tools.
When it really comes down to it, there’s 2 ways to test. Either on real devices or with an emulator which simulates the software. Each has its pros and cons so let’s explore!
Testing on Real Devices
There’s really no substitute for the real deal. Testing on real android/ios/other devices is the absolute best way to get an idea of how an app works on each type of device. However, there can be some drawbacks.
First, you need to have all those devices to test on. This can be accomplished either by hitting up all of your coworkers and borrowing devices (commonly referred to as BYOD, Bring Your Own Device… Yuck) or by contacting device manufacturers and asking to borrow. If you’re lucky, you’ll be put on a wait list and receive a device to test on… Eventually. Then there’s actual testing part. Manual device testing is crucial, but it is extremely time consuming, especially when testing for the first time to identify and diagnose bugs.
That’s where automated app testing comes in, it’s just very difficult to do on dozens of devices all at once.
Fortunately, there are a handful of mobile device testing solutions out there. For example, device Connect by Mobile Labs allows you to test almost 50 devices at once and automate the testing process on every device simultaneously. You actually receive a physical testing station which can be connected behind your own corporate firewall. Say “bye-bye” to 3rd party security concerns. Mobile Labs also has good reviews too!
This solution really is only meant for serious testers in a larger company. While the equipment may seem expensive, it’s significantly cheaper than purchasing all the devices and paying monthly and it can streamline a process that would take several developers weeks to test, done to just a few developers who can test in a fraction of the time.
Real Devices Pros:
- Most accurate testing environment
- Increased security
- With a tool like device Connect, it can save significant time and money
- Complete control of the testing process
Real Devices Cons
- Can be expensive and slow without the right tools
- Physical devices do take up a little bit of space
Testing on Emulators
For those without the capital resources and companies not large enough to test on mobile devices on a large scale, there’s the emulator solution. Emulators mimic the software, iOS or Android so you can test in a simulated virtual environment.
This is a much more inexpensive solution, but of course, you do get what you pay for.
You do not get to interact with the actual device so if there is a software and hardware compatibility bug, it will be hard to catch in the testing phase which can upset your users after launch.
In addition, since the software is just emulated you are now relying on the provider to accurately simulate the software which they may or may not do 100% right.
Since you are not using physical devices, you lose control of the testing process as well. If the provider is having software issues, it will impact your testing results and potentially delay your app release. If there are problems, you’ll have to rely on the tech support and customer service of the 3rd party. Unfortunately, one of the biggest players in the game—Perfecto Mobile–seems to not have the best reputation when it comes to customer service and have been involved in patent infringement lawsuits.
- Very cheap
- Easy to get started
- Emulator is not always 100% accurate with software simulation
- 3rd party has significant control over your testing
- Many competitors, each has strengths but none all excel at everything
- You cannot test how software interacts with the hardware
Both emulators and real device solutions have their own strengths and weaknesses but if you can afford it and are serious about mobile device testing, then there’s no substitute for testing on real devices. Of course, an emulator is a great solution for those on a small budget.
So what did you think? Any thoughts on what you prefer? Let us know!