This page contains an executive summary of a recent study conducted by Perceptive Sciences on the usability of the latest generation of smartphones. More information can be found in the Computerworld article “Usability Test: Does iPhone Match the Hype,” which can be viewed here. For further inquiries, please send an e-mail to info@perceptivesciences.com.
The study was designed, conducted and analyzed by research scientists holding advanced degrees. Our team consists of Ph.D.s and M.A.s in Cognitive Psychology and Human Factors Psychology.
Study Objective. This study assessed the out-of-the-box usability of 3 next-generation smartphones: the Nokia N95, the HTC touch, and the Apple iPhone. Users with no smartphone experience conducted a series of 8 tasks on each device in order to determine how the devices fared in terms of overall usability. The study also assessed user perceptions of the devices in terms of ease of use, quality, pleasure-to-use, complexity of function-set, appropriateness for business use, and personal purchase preference.
Main Findings.
• Participants completed more tasks successfully using the iPhone than
they did using either the HTC Touch or the Nokia N95.
• Participants successfully completed tasks twice as fast (on average)
on the iPhone as they did on the HTC Touch or Nokia N95.
• Participants were more interested in purchasing the iPhone for themselves,
although the HTC Touch and iPhone were both selected as being appropriate
for business users. Users perceived the iPhone to be more complex than the
Nokia N95 in terms of the number of features and functions offered.
• In addition to user testing, our cognitive psychologists conducted
a heuristic review based on a qualitative scoring of each device along five
dimensions (global navigation, usability/information architecture, ergonomics,
look-and-feel, and breadth of functionality). The iPhone scored higher on
the expert review than both the HTC Touch and Nokia N95.
Participants. All participants selected for this study did not own any type of smartphone device or PDA. They were average cell phone users (i.e. used low-end, industry standard handsets) and were recruited to fit the demographic of the target market for these devices. The fact that they had no experience or hands-on exposure to the devices used in the study ensured that the usability data would be unbiased.
Methodology. All participants were run individually in a
1-hour usability test session. 10 participants were run comparing the iPhone
and the HTC Touch, while a second group of 8 participants were run comparing
the iPhone and the Nokia N95. This sample size had more than enough power
to resolve the large effects observed in time-to-success, success rate, and
the forced-choice device preferences.
The order that participants used the devices was counterbalanced, so that
not everyone started with the same device. The tasks used in the study were
based on frequently used functions offered in most cell phones today: turning
on the device, setting the ringer to vibrate, placing a local call, creating
a new contact, sending a short email, browsing the web for a specific link,
taking and deleting a photo, and turning the device off. For each task, we
measured how long it took to complete the task (or for a participant to “give
up”), and whether or not the task was completed successfully (“giving
up” or exceeding each task’s fixed time limit resulted in a failure).
Participants had no access to the instruction manual and were not trained
on using the devices. By limiting the information available to the participants,
we sought to measure ease-of-use and intuitiveness in the absence of any supporting
materials or guidance.
Perceptive Sciences is often turned to by journalists and clients to conduct market research, usability studies, and competitive analyses on a wide variety of software, hardware and consumer products.
Click here to see recent press related articles and here to see a sample of our clients.
