When Is a Defibrillator Not a Defibrillator? When It’s Like a Clock Radio … . The Challenge of Usability and Patient Safety in the Real World

Understanding of Usability Science 

First, providers, many users, manufacturers, and even regulatory agencies do not necessarily know or understand usability. Anyone who has struggled with programming a DVD player or setting the alarm on a novel clock radio in a hotel room inherently understands the problem of usability but may be unaware that usability science exists. Usability is grounded in the field of human factors engineering,6, 7 which is the science that studies human performance capabilities and limitations and designs built systems (eg, medical devices, information technology, scalpels, computer monitors) to support performance needs. Unfortunately, many decisionmakers and designers erroneously believe that “identifying performance needs,” also referred to as “user-centered design,” is common sense. Users themselves often share the same erroneous assumption; this leads to blaming (I can’t believe that person can’t figure it out) and even ironically leads to much self-blaming (Why can’t I figure this out? or If only I had been paying more attention, this wouldn’t have happened) when in fact the real problem was poor design.

Consider a defibrillator; the obvious or common-sense need is for the user to be able to save the life of a patient by correctly diagnosing and treating a life-threatening rhythm disturbance. But a human factors engineer or usability scientist analyzes usability at a much deeper level. He or she realizes there are physical and cognitive performance needs that must be met for all the different environments of use and possible users. The physical performance needs include having buttons, knobs, and dials designed to accommodate the hand and finger sizes of different users to maximize correct usage and minimize incorrect usage or accidental activation. The actual design could vary, depending on where the device is used; in emergency transport vehicles, there is significant vibration, so the affordances must be even greater. The device should also accommodate lifting and carrying needs by having correctly designed handles and a weight that can be carried by the majority of users. The device needs to be designed so that it can be read (visual sensation) in dark or bright light and the auditory cues (alarms or confirmatory sounds) can be heard in environments with varying noise and with the possibility of vibration in a transport vehicle. Cognitive performance needs may include planning, decisionmaking, attention focusing, pattern matching, problem solving, and many others. The defibrillator can be designed to make all of these tasks easier, and Fairbanks et al clearly demonstrate how readily available devices failed to meet many of these design requirements. Fortunately, dozens of scientifically validated design guidelines exist to support each of those tasks.8, 9, 10, 11, 12 The bad news is that it is not clear how much of this available guidance is used.

Additionally, many people, including health care administrators, clinicians, and manufacturers, believe proper training and compliance with correct use protocols are sufficient for avoiding errors when automation is used. This problem is not a lack of awareness of the existence of a science behind usability but a lack of awareness of the concept itself. This belief, which is not evidence based, brings with it a host of problems. Users get blamed (think user error) for mistakes caused by bad design, and users themselves even think all bad outcomes stemming from human automation interactions are their fault! Norman13 explains that this self-blame phenomenon is the result of misunderstanding in causality, as well as both a learned and taught helplessness. Although training is crucial for effective use of automation, it cannot completely compensate for poor design. Training is unlikely to overcome interfaces that do not conform to population stereotypes for where information is located or what colors mean or what “enter” or “return” means. And all the training in the world will not help someone hear an alert in a noisy environment or read a small display in a vibrating ambulance, especially during a time-critical task. Each of these cases requires better system design.6, 14

Usability Testing Procedures 

A second major issue is the challenge of performing usability testing. Even if one possesses a robust understanding of usability science, good usability testing is not ensured. Dozens of resources describe the plethora of usability methods,15, 16, 17, 18 including many specific to medical devices or health information technology.5, 19, 20, 21, 22, 23, 24 Bad conclusions result when the methods are incorrectly applied. One important consideration for testing requires knowing how to select representative end users and representative environments of use. Whether the studies are conducted by manufacturers or hospitals, if the sample includes only “expert users” or “well-trained users,” the results will likely not be generalizable. Similarly, if the device or information technology will be used in environments with different lighting intensities and sources (fluorescent tube versus sunlight), different levels of noise (intensive care unit versus surgical suite) and even different levels of distraction (emergency department versus general medicine ward), then testing must be conducted in those different environments to determine if and when the device is usable. Putting a handful of subjects in a nice, clean, simulated patient care room and having them use the device there may not simulate the real environments of use, in which the alarms might not be audible, the displays easily visible, or buttons easily activated. The key point here is that usability is not proven by demonstrating that a handful of people can use the device in a given environment. Usability is determined by the interaction among users, the technology, the environment (lighting, noise, vibration, distractions), the task characteristics (time pressure, need for concentration) and the organization (culture, policies). Good usability testing must attempt to mimic these interactions.

Understanding the many human biases that can lead to wrong conclusions during testing is another important consideration for good usability. For example, hindsight bias may lead a tester to conclude that a user error during testing could “obviously” be corrected with better training. The fundamental attribution error may lead to conclusions that it was just the “ignorant” subjects who did not understand how to use the device. These biases and many more exist, and a well-trained tester must be aware of them and not fall victim to them. The pervasive blame culture in health care, which is likely rooted in the fundamental attribution error and similar biases, may reduce the understanding of the contribution of bad design in the same way. These biases even exist in heuristic evaluations and expert testing. Experts in a laboratory setting do not view or use technology the same way as novice users in a real-world environment. Consequently, even if end users, regulatory agencies, and manufacturers all understood the issues of usability and more usability testing was mandated, there is no assurance that proper testing would occur.

Market Demand and Regulation 

The final major issue affecting usability of medical devices is weak market demand for improvements, in part attributable to the lack of understanding of the importance of usable technology and devices. However, this issue also illustrates the role health care providers and organizations play in the problem of poorly usable devices by largely demanding and purchasing devices that function at the lowest price possible. At the same time, many providers insist on added features that may increase complexity and decrease usability. The combination of purchasing for price while requesting added features, without any requirement for usability, creates a potentially dangerous combination. Although some organizations request highly usable products, they are a minority, and thus, their numbers are insufficient to change device and technology design.

Alternatively, some consumers of these products, primarily health care delivery organizations, may assume they are purchasing “tested” products. The US Food and Drug Administration requires manufacturers to “address the intended use of the device, including the needs of the user and patient,” whereas information technology vendors have no such requirements.25 However, compliance with this regulation may yield very different results, according to their understanding and execution of usability principles. At best, this means that the device does work if used exactly as intended under ideal circumstances. Perfect conditions rarely exist in health care delivery organizations, rendering the possibility that a device will function in its intended manner largely fatuous.

The lack of current market demand takes on added significance in light of the fact that those most likely to be hurt by poorly usable devices and technology, ie, patients, are not involved in the selection and purchasing processes. Meanwhile, it is unlikely that providers, clinicians, and even manufacturers will see the direct consequence (patient harm) of their decisions. Ironically, in an environment of increased transparency of medical errors and harm, manufacturers might leverage enhanced safety through improved usability for a market advantage.

What does this all mean? Devices and technology with poor usability are endemic in health care and will remain so until there is a fundamental shift in knowledge that leads to changes in purchasing. Groups such as the US Food and Drug Administration and the Joint Commission probably should demand better product usability at both the manufacturer and consumer levels, but improvements will not happen unless good usability testing practices are implemented. Until then, the next time you see a clock radio, think of the ventilators, defibrillators, and pumps on which your patients depend.