One cheer for feedback

To a great extent, emergency medicine is practiced in a vacuum. Emergency physicians evaluate patients, formulate evaluation and management plans, set them into motion, and move on to the next case. It is rare that they ever hear what the outcome was; so rare that a question that to an outsider would seem innocent enough—“Remember that patient you saw last week?”—is enough to induce tachycardia.

Without reliable feedback, it is almost impossible for practitioners to learn or even to be well calibrated about their performance.1 The absence of an infrastructure that seamlessly, routinely, and effortlessly provides feedback to the emergency caregiver is a major impediment to improved performance in emergency care.

In this issue of Annals, Chern et al2 report on a system of routinized feedback to residents based on telephone follow-up of selected cases thought to be at greater risk. Although this was not their only intervention (they also instituted education targeted at problem cases or subtle presentations of serious illness, and added mandatory attending review before discharge, thus confounding what led to the results), the net effect of this suite of changes was to cut both unscheduled return visits and adverse events roughly in half. However, this may not be unalloyed good news because the interventions also led to increases in the observation and admission rates. This suggests that, rather than improving discrimination, the intervention simply caused practitioners to “aim off,” that is, to simply change their threshold and admit or observe more cases across the board.

The context of the feedback intervention might explain this. One can imagine how these feedback “moments” may have played out in a typical hierarchical, high-power distance3 training program. It is likely that what residents hear in this setting, no matter how gently their attending physicians present it, is that they are being closely watched and had better be extra careful about sending patients in these high-risk categories, who might have more serious diagnoses, home. Although there is nothing wrong with this general vigilance, without knowing the balance of costs and benefits it is not necessarily clear there was any real improvement. In other words, it looks as though the increase in sensitivity was purchased by a decrease in specificity; this might or might not be a good thing, depending on the relative costs of false positive and false negative decisions, which at present are unknown.

It is not entirely clear whether residents in this study got feedback on every high-risk patient they managed, or only on those who developed some problem or had to return. Different methods of providing feedback might have dramatically different effects because people tend to weight recently received or dramatic information more heavily; in addition, if feedback was only given on problem cases, then the clinicians would likely continue to be miscalibrated, although now in the negative direction.

These patients present tough decisions. Admitting and observing more nonspecific presentations undoubtedly will capture more cases with latent serious conditions, but it is harder to see whether these interventions helped make the decisions any easier or led to better judgment in sorting out sick from not sick.

Still, providing feedback on a routine basis seems like a step forward,4 and the authors should be congratulated for finding a way to institute it in their department. But a great deal of further study is needed to more clearly identify what, exactly, the feedback should be, how it should be presented, what changes in behavior and outcome it produces, and (most difficult of all) how to make such a feedback system ongoing and sustainable.