Practical Considerations in HIV Testing in the Emergency Department, Characteristics of Diagnostic Tests, and the Role of Sensitivity Analysis in Observational Studies:

Article Outline

• Discussion Points
• Answer 1
• Answer 2
• Answer 3
• Answer 4
• Answer 5
• Answer 6
• References
• Copyright

Back to Article Outline

Discussion Points 

1. The emergency department (ED) in which you work has been awarded a Centers for Disease Control and Prevention (CDC) grant to initiate nontargeted (universal) opt-out rapid HIV screening of adults. The grant provides funding to test patients for 6 months, after which time the hospital needs to decide whether this testing should be continued. Your hospital medical director asks you to implement a testing program in your ED. How sensitive and specific are the rapid tests? What other resources should be in place before initiation of a testing program? What would the process entail for a patient who tests positive but does not require hospital admission? In a time when ED crowding and throughput are major concerns, how might HIV screening affect ED length of stay?

2. Question 1 asks you to state the sensitivity and specificity of the rapid tests, but what do these terms mean? Prepare a 2×2 table for a dichotomous test (positive versus negative) and a dichotomous disease state (diseased versus nondiseased) and define sensitivity, specificity, predictive value positive, predictive value negative, and accuracy. Which is more important in a screening test, sensitivity or specificity? Why?

3a. The authors write: “Using previously unpublished data, it was assumed that approximately 75% of patients would agree to be tested when offered nontargeted opt-out rapid HIV testing as part of their ED visit. A sample size of 500 patients was therefore estimated to provide 95% confidence intervals (CIs) around this point estimate of less than 10%,” by which they mean that they determined the sample size necessary to be sure that the limits of the 95% CI around 75% were within 10% of that value. Thus, the authors determine the sample size required for their study to have a certain precision rather than a certain power. Discuss the difference between these 2 approaches. What information should authors provide with regard to their sample size calculation when the outcome is a proportion? Describe in what way the authors' description of their sample size calculation is ambiguous.

3b. What is the definition of a CI? What 95% CI would we expect for the proportion 375/500 (75%), which signifies that 375 of 500 subjects stated they would be willing to have an HIV test? Assuming that 75% is the correct value, what is the anticipated precision had they enrolled 12 patients? 24? 48? 100? 240? 1000? 10,000? In general terms, what is the relationship between sample size and precision for a single proportion? Is it linear?

4. There have been recent publications within the emergency medicine literature addressing the feasibility of HIV screening in the ED and the specific barriers that have deterred physicians from participating in HIV screening. Discuss the most commonly cited barriers and potential strategies for overcoming them. Discuss the potential advantages of initiating rapid HIV screening in the ED. How might this testing affect patient care in the ED? What are the advantages of knowing a patient's HIV status when treating him in the ED? Are there any disadvantages?

5a. The authors write that “A convenience sample of patients was approached by trained research assistants and asked to participate if they were awake, alert, not intoxicated and understood the premise of the study.” They also report that 529 (8%) of 6,965 adults seen in the ED during the study were enrolled and completed the survey. How might these factors affect one's interpretation of this study?

5b. Perform a sensitivity analysis that explores how the value offered for the percentage of subjects willing to be tested in the opt-out strategy—81%, 95% CI (77%-84%)—would change if the patients who participated were 5, 10, 20, 40, 80, 100, 200, 500, and 1000% more likely to respond “yes” than those who were not sampled. What if the sample included 3,500 of the (roughly) 7,000 ED patients? Create a graph of your estimates for both sample sizes with the proportion of the population willing to be tested on the y axis and the ratio of “yes” responses in those sampled to “yes” responses in those not sampled on the x axis. Do this for other sample sizes (6,000, 7,000) if you wish.

5c. Question 3 explored the effect of sample size on the precision of the estimate; this sensitivity analysis explores the effect of sample size on the likelihood of sampling bias. Which consideration is more important in the Haukoos et al paper? Is the study's conclusion more vulnerable to imprecision or bias? What is the role of sensitivity analysis in observational studies?

6. In your opinion, what are the most important conclusions from this paper? How might the limitations mentioned by the authors affect your decision whether to change your clinical practice with regard to initiating nontargeted opt-out HIV screening in your ED? Would you have asked the patients questions covering any additional topics regarding HIV screening? What additional information or data analyses would you like the authors to provide in order for you to change your clinical practice?

Back to Article Outline

Answer 1 

Q.1.1 The ED in which you work has been awarded a CDC grant to initiate nontargeted (universal) opt-out rapid HIV screening of adults. The grant provides funding to test patients for 6 months, after which time the hospital needs to decide whether this testing should be continued. Your hospital medical director asks you to implement a testing program in your ED. How sensitive and specific are the rapid tests?

The sensitivities of the 4 Food and Drug Administration (FDA)–approved rapid HIV-1 antibody tests range from 99.3% to 100%. Specificity ranges from 98.6% to 100% (Table 1). The tables below were created by the Health Research and Education Trust and are available at http://www.hret.org/hret/programs/content/rpd1.pdf.

Table 1. Testing characteristics of FDA-approved rapid HIV screening tests.

As expected, the predictive value positive (PV+) has a much greater range than the sensitivity or specificity, depending on the HIV prevalence in the community. For example, Haukoos et al estimated that between 0.7% and 2% of their ED population has undiagnosed HIV infection.¹ The PV+ of these 6 tests would range from 26% to 100% if the prevalence of undiagnosed HIV is 0.5% in the community and from 59% to 100% if it is 2% (see question 2 for information on how to calculate these values). The CDC recommends that patients with a negative rapid HIV test result be told they are not infected. However, because HIV antibodies are not present immediately after acute infection, retesting should be recommended to persons with a recent possible exposure (sexual contact or needle sharing in the past 3 months).², ³

Confirmatory testing with a Western blot or immunofluorescence assay is essential for all patients with a positive rapid HIV test. If the confirmatory test is negative or indeterminate, it should be repeated in 4 weeks.³

An excellent review article³ is available on the CDC Web site at the following Internet address: http://www.cdc.gov/hiv/topics/testing/resources/journalarticle/pdf/rapid_review.pdf

Q.1.2 What other resources should be in place before initiation of a testing program? What would the process entail for a patient who tests positive but does not require hospital admission?

The CDC's goal is to diagnose the estimated 250,000 persons in the United States who are unaware of their HIV infection and initiate treatment earlier to decrease the transmission rate of the infection.⁴, ⁵ The most recent CDC guidelines on rapid HIV testing in the ED state the following⁵:

Persons responsible for HIV testing in the ED should be aware of several implications of the CDC guidelines. First, although the CDC recommends incorporation of the HIV consent into the general consent for care, individual state laws vary on this matter. Wolf et al⁶ reviewed HIV consent laws and found that 14 states require written consent, 19 require oral consent, and 11 require pretest counseling. Individual EDs need to ensure compliance with their state regulations when designing their own protocols for rapid opt-out HIV screening. Second, although the CDC recommends that all persons tested be provided information regarding the meaning of positive and negative test results, there is no requirement that EDs have HIV counselors.⁵

Two recent publications, one by Haukoos et al⁷ at Denver and another by Brown et al⁸ at George Washington University, offer 2 ED HIV screening implementation strategies. In Denver, the protocol involved physicians identifying patients at high risk who warranted screening. These patients were then counseled and consented by clinical social workers and the rapid HIV test was performed in the hospital laboratory. Patients with negative tests received posttest prevention counseling from the social worker and those with positive tests received more intensive counseling from both a physician and the social worker and referral for confirmatory testing.

The HIV screening program at George Washington University involved the triage nurse informing the patient of the free HIV screening test and distributing information about HIV and HIV screening. The rapid oral specimen test was then performed by supplementary nonmedical volunteer staff who also informed patients of negative results. Patients with positive rapid test results were informed by the attending ED physician and given several follow-up options, among them a clinic that agreed to see patients and perform confirmatory testing on patients irrespective of their ability to pay for services. Based on their experiences, Brown et al⁹ recommended that at least 1 emergency physician coordinate such an implementation, managing and reviewing the process and educating the physicians and nursing staff about the value of this program and the benefit to the patients' and the public's health. Individual EDs might use these protocols as a starting point for the design of their own protocols.

The most delicate part of the ED HIV testing process is the management of patients who have a positive rapid test but do not require hospital admission. These patients require counseling by a professional knowledgeable about HIV who is capable of answering questions and securing confirmatory testing. Furthermore, the CDC recommends that appropriate interpreter services be available so that a family member does not have to serve as interpreter, given the stigma of HIV infection.⁵ EDs may have trouble ensuring that adequate counseling and interpreter services are available at all times and may have trouble securing confirmatory testing for all patients.

Ideally, patients who test positive would be referred to a single designated clinic but this may not be possible, given the myriad of social and economic considerations involved. Silva et al¹⁰ found that only 3 of the 8 patients with rapid positive tests made at least 1 follow-up visit to the ID clinic. Others have reported increased compliance with follow-up when posttesting counseling and social work assistance has occurred, monetary compensation was offered, or the patient was walked from the ED to the ID clinic.¹¹, ¹², ¹³, ¹⁴

Q.1.3 At a time when ED crowding and throughput are major concerns, how might HIV screening affect ED length of stay?

At present there is insufficient evidence regarding the adoption and effect of the 2006 CDC recommendations. Length of ED stay was 1.7 hours longer for patients who were tested in the Denver trial.⁷ However, the authors point out that the decision to test for HIV was often made late in the patient's ED stay. Testing at triage might eliminate this delay and, because the rapid tests can be performed on saliva or fingerstick blood, should be feasible. Results for rapid oral fluid or whole specimens were typically available 20 to 40 minutes from time of specimen collection and would not be the rate limiting step in the patient's disposition.¹⁵

In theory, because decreased hospital length of stay has been reported when HIV diagnosis is made on admitted patients,¹⁶ rapid HIV testing in the ED might improve ED throughput by increasing inpatient bed availability. However, in most hospitals the number of such cases is small and the effect is likely to be negligible. On the other hand, counseling the occasional patient who tests positive may consume resources that would otherwise be used to move other patients along in their ED stay. This could increase the average length of stay. Furthermore, testing will likely consume resources that may not be fully reimbursed and this could divert resources from staffing, resulting in decreased throughput.

Back to Article Outline

Answer 2 

Q.2.1 Question 1 asks you to state the sensitivity and specificity of the rapid tests, but what do these terms mean? Prepare a 2×2 table for a dichotomous test (positive versus negative) and a dichotomous disease state (diseased versus nondiseased) and define sensitivity, specificity, predictive value positive, predictive value negative, and accuracy.

The table is typically constructed as follows:

The columns represent patients who are deemed diseased or nondiseased by a gold standard (criterion standard) test. In the case of rapid HIV testing, the “diseased” status would typically be determined by Western blot or immunofluorescence assay. The column totals therefore represent the number of diseased and nondiseased subjects. Each column is divided into 2 strata, those with a positive test and those with a negative test result. The row totals represent the number of subjects with positive and negative test results. The sum of the row totals represents the total number of subjects, as does the sum of the diseased and nondiseased columns.

The 4 cells in the center of the table (within the bold lines) have special names that indicate the relationship of test result and disease state. If the values of these 4 cells are known, then the 5 external (or “marginal”) cells can all be calculated, but the reverse is not true. Therefore, authors should always provide the 4 inner numbers.

Sensitivity is defined as the proportion of DISEASED subjects who have a POSITIVE TEST result, or TP/(TP+FN). It is calculated exclusively from numbers in the “Disease Present” column. It answers the question “If the subject has the disease, what is the probability the test will be positive?” It is crucial to distinguish the sensitivity (the probability the test will be positive, given that the person has the disease) from the predictive value positive (the probability that a patient has the disease, given a positive test result).

Specificity is defined as the proportion of NONDISEASED subjects who have a NEGATIVE TEST result, or TN/(TN+FP). It is calculated exclusively from numbers in the “Disease Absent” column. It answers the question “If the subject does not have the disease, what is the probability the test will be negative?” It is crucial to distinguish the specificity (the probability of a negative test result, given the patient does not have the disease) from the predictive value negative (the probability that a patient does not have the disease, given a negative test result).

Note that sensitivity is calculated using only diseased persons and is therefore independent of the prevalence of the disease (this is not exactly true, but that will be the subject of a future journal club question). The same is true for specificity.

Predictive value positive (or positive predictive value or PV+ or PPV) is the proportion of subjects with a POSITIVE TEST result who have the DISEASE or (TP/TP+FP). It is calculated exclusively from numbers in the “test positive” row. It answers the question “If the subject has a positive test result, what is the probability that he has the disease?” Note that this is the quantity that clinicians care about but, because it is calculated from numbers from both the “disease present” (the TPs) and “disease absent” (the FPs) columns, it is dependent on the prevalence of the disease in the community or, alternatively, one's belief about the probability of disease in a single subject right before the test is conducted on that subject. Thus, unlike sensitivity and specificity, predictive value positive is not a static, inherent quality of the test but must be calculated anew for each value of prevalence or previous belief.

Predictive value negative (or negative predictive value or PV– or NPV) is the proportion of subjects with a NEGATIVE TEST result who DO NOT have the DISEASE, or (TN/TN+FN). It is calculated exclusively from numbers in the “test negative” row. It answers the question “If the subject has a negative test result, what is the probability that he does not have the disease?” Note that this is the quantity that clinicians care about but, because it is calculated from numbers from both the “disease present” (the FNs) and “disease absent” (the TNs) columns, it is dependent on the prevalence of the disease in the community or, alternatively, one's belief about the probability of disease in a single subject right before the test is conducted on that subject. Thus, unlike sensitivity and specificity, predictive value negative is not a static, inherent quality of the test but must be calculated anew for each value of prevalence or previous belief.

Accuracy is defined as the proportion of correct results (test result matches disease state) or (TP+TN)/(TP+TN+FP+FN). Its use should be discouraged because sensitivity and specificity are each needed to intelligently contemplate the value of a test and accuracy blends the 2 without providing unique information or benefit.

Q.2.2 Which is more important in a screening test, sensitivity or specificity? Why?

“Screening” implies that an unselected population is being tested for a condition. When screening asymptomatic subjects who are not actively seeking treatment, it is expected that the disease will be rare. For example, consider randomly testing 10,000 adults who are not already known to have HIV in an ED where there is a 1% prevalence of undetected HIV infection. Our table might look like this:

If we assume that the test has a sensitivity of 90% and a specificity of 99%, our results would be as follows:

The test would be positive in 189 patients (1.9% of those tested), all of whom would require counseling. In an ED with an annual adult census of 40,000, 10,000 patients would be seen during 3 months (92 days), that staff will be counseling about 2 patients each day. The predictive value positive of such a test applied to a population with a prevalence of 1% would be 90/189, or 48%. Less that half of the patients who have a positive rapid test would ultimately be confirmed as being infected with HIV. The 99 false-positive patients will have been subjected to substantial anxiety with no benefit and possible harm from increased testing or hospitalization. Because the sensitivity was only 90%, 10 patients (10% of all subjects) with HIV disease would test negative and be missed.

What if the numbers are reversed and the sensitivity is 99% and the specificity is 90%? We have:

We now miss only 1 patient with HIV disease, but the poor ED staff will have to counsel 1,089 patients with positive rapid test results (1 every 2 hours!). Even worse, with a predictive value positive of 99/1,089=9.1%, only 1 in 11 patients who test positive on the rapid test will actually have HIV infection.

How about a rapid test with sensitivity of 99% and specificity of 99.9%?

Our ED staff is happier. Only 109 patients need counseling, and 9 out of 10 of them actually have HIV infection. From this example, we see that specificity is crucial when conducting a screening test. Prevalence is low and there will be few true positives. With so many nondiseased subjects, the number of false positives will overwhelm the true positives unless specificity is extremely high. This is not to say that we do not want high sensitivity but, if a tradeoff has to be made, in the screening situation we value specificity more than sensitivity.

Some practitioners find the mnemonic “SPIN,” derived from “SPecificity rules IN,” a useful aid to remember that if one wants to be sure that a patient has a disease (“rule in” the disease), a very high specificity is needed to make sure that there are few false positives and therefore a positive test result represents a true positive. As we will see in a future journal club, the converse is SNOUT (SeNsitivity to rule OUT), meaning that if you want to tell someone that they do not have a disease, you better have great sensitivity to keep the number of false negatives down so that you can be sure that all negative test results represent true negatives.

Back to Article Outline

Answer 3 

Q.3.1 The authors write: “Using previously unpublished data, it was assumed that approximately 75% of patients would agree to be tested when offered nontargeted opt-out rapid HIV testing as part of their ED visit. A sample size of 500 patients was therefore estimated to provide 95% CIs around this point estimate of less than 10%.” Thus, the authors determine the sample size required for their study to have a certain precision rather than a certain power. Discuss the difference between these 2 approaches. What information should authors provide with regard to their sample size calculation when the outcome is a proportion? Describe in what way the authors' description of their sample size calculation is ambiguous.

Researchers and clinicians care about both validity (whether estimates of a quantity cluster around the true value) and precision (how tightly clustered the estimates are). The precision of an estimate is determined by the inherent variability of the quantity being measured and the sample size. Because precision is an important aspect of experimental and observational studies virtually all scientific reports should explain the choice of sample size. Sample size can be selected to achieve a desired precision or a desired power.

Precision is typically characterized by the size of the CI around an estimate. For example, a study that reports that 40% (95% CI 37% to 43%) of subjects have the disease is more precise than one that reports that 40% (95% CI 30% to 50%) of subjects have the disease. For proportions, all that is required to calculate a sample size is the width of the desired CI and the investigators' belief regarding the likely value of the proportion. From these 2 quantities, most off-the-shelf statistical packages will calculate how many subjects are required to achieve that level of precision.

For example, in STATA (Stata Corp, College Station, TX) the command: informs us that a sample size of 1000 will produce a 95% CI from 37% to 43 assuming that the percentage is 40%. Similarly, demonstrates that a sample size of 100 would produce a 95% CI with a width of about 20% around the percentage of 40%, from 30% to 50%.

There is one minor ambiguity in the Haukoos et al paper. It is unclear whether by their statement “a sample size of 500 patients was therefore estimated to provide 95% CIs around this point estimate [75%] of less than 10%” they mean 70% to 80% (an absolute width of 10%); 65% to 85% (a width of 20% with each limit being within 10% of the estimate); 71.25% to 78.75% (an absolute width of 7.5% equivalent to a relative width of 10% [10% of 75% = 7.5%]), or 67.5% to 82.5% (an absolute width of 15% with each limit being within 7.5% [10% of 75%] of the estimate). A quick check: suggests that they may have meant the third alternative because this CI best matches that expected with a sample size of 500. The key concept is that all 4 of the above CIs are adequate to qualitatively capture the sampled patients' beliefs. They will tell us whether almost all patients, most patients, some patients, few patients, or almost no patients are willing to take the HIV test. None of these CIs are adequate to permit us to state with certainty that the number is 75% not 73%. A much larger sample would be required if the goal of the study were to make such a statement.

Finally, we consider the concept of sample size based on precision compared to sample size based on power. When performing a study that compares 2 or more groups, one must decide how many subjects are required to have a sufficiently precise estimate of the difference between the groups. The concepts of precision discussed above easily translate to this circumstance. Instead of considering the CI around a single proportion, we consider the CI around the difference in 2 proportions. For example, if we anticipate that a study will find a 40% response in the control group and a 60% response in the intervention group, a 20% absolute difference in response, we can determine the sample size necessary to get any particular CI around the 20% value. For example, a study with 100 patients per group would have a 95% CI of 6% to 34%, whereas a study with 1000 patients per group would have a 95% CI of 16% to 24%.

Whereas the concept of precision can be used to determine the sample size of any study, the concept of power becomes relevant in the framework of classical hypothesis testing, a topic that will be covered in detail in a future journal club. In brief, in the hypothesis testing model we begin with a null hypothesis (eg, the response rate is the same in the 2 groups) and examine whether the observed results are likely to have occurred by chance alone, assuming that the null hypothesis is true. If the observed results are sufficiently unlikely, we reject the null hypothesis, implying that the groups are sufficiently different that something other than chance is responsible for the difference. Power quantifies the ability of a study to reject the null hypothesis, assuming that the groups truly differ by a prespecified amount. Keeping with our 40% versus 60% assumed response rate example, we could determine the number of subjects required to ensure that, during multiple repetitions of an experiment, 90% of the iterations will reject the null hypothesis that the difference between the 2 groups is 0. In STATA we enter:

Thus, a study with 140 persons in each group would have a 90% chance of rejecting the null hypothesis if the true values in the 2 groups were .4 and .6.

The expected precision of the difference in such a study would be 95% CI, 9% to 31%:

Although either approach is acceptable, the precision approach encourages the investigator to think carefully about the degree of precision that would be clinically meaningful rather than restricting one's horizon to the binary “reject/don't reject the null hypothesis” decision.

Q.3.b.1 What is the definition of a CI? What CI do you get if you assume that 75% of 500 subjects were willing to be tested? Why do we need to know that the authors anticipate the assent rate to be 75%?

We begin by stating what a CI is not (though a quick perusal of CI definitions on the Web or a casual questioning of medical students might make one think that it is the definition). When a 95% CI goes from 40% to 60%, it does not mean that there is a 95% chance that the true value exists within this range. We would love to know that interval (the region that has an “x”% chance of containing the true value), but such an interval is unavailable in the frequentist statistical framework. The estimation of such an interval requires Bayesian statistics, a topic that will be covered in future journal clubs.

A 95% CI is defined as follows. Perform an experiment for which you already know the truth. Calculate the 95% CI. Determine whether the true value is contained within the interval. Repeat this process 1,000 times. Count how many times the CI contained the true value. If it is a true 95% CI, then 950 of the intervals will contain the true value and 50 will not. Stated more formally, the 95% CI represents an interval such that if the experiment were repeated many times, 95% of the resulting CIs would contain the true parameter value. The CI provides a sense of where the true value is likely to fall but does not provide the probability that the true value is within its range.

As shown above (3.a.) the 95% CI for a proportion of .75 observed in a 500 subject sample is .71 to .79. The calculation of confidence limits for proportions is based on the binomial theorem. As such, the width of the CI varies somewhat, depending on how close the proportion is to the extremes (0 and 1) or middle (.5) of its range. We examine the width of the 95% CI for 500 subject samples that have varying proportions:

95% CIs for 500 subject samples with varying proportions

The table reveals that as the proportion moves from .5 toward 1, the width of the CI narrows. As a result, when the expected proportion is close to .5 one needs more patients to achieve a given precision than when the expected proportion is closer to 0 or 1. Because precision varies with the proportion, one must specify an expected proportion in order to determine sample size.

Q.3.b.2 Assuming that 75% is the correct value, what is the anticipated precision had they enrolled 12 patients? 24? 48? 100? 240? 1000? 10,000? In general terms, what is the relationship between sample size and precision for a single proportion? Is it linear?

STATA output for various sample sizes all with a proportion of .75 are shown below.

The series demonstrates how the CI narrows as the sample size increases. Those who are mathematically inclined can contemplate how the binomial theorem can be used to calculate these exact CIs, but the important point is that as the number of subjects increases the effect on the width of the CI diminishes. It takes more and more subjects to further narrow the CI. We graph the CI for a series of studies all with result .75 but different numbers of subjects.

The graphs represent the limits of the 95% CI (the distance between the 2 curved lines) around the mean value (.75) for different sample sizes (y axis). The CI narrows quickly as the N increases from 0 to 100 but narrows more slowly after that. It is clear from the graphs that the relationship between sample size and CI width is not linear.

Back to Article Outline

Answer 4 

Q.4.1 There have been recent publications within the emergency medicine literature addressing the feasibility of HIV screening in the ED and the specific barriers that have deterred physicians from participating in HIV screening. Discuss the most common cited barriers and potential strategies for overcoming them.

Recent publications have shown that rapid HIV screening in the ED is feasible and can be implemented using a variety of approaches. Some programs incorporate HIV screening into ED triage; others are dependent on additional personnel dedicated solely to HIV screening and counseling.⁷, ⁸, ¹⁰, ¹¹, ¹⁷ The American College of Emergency Physicians' policy statement is presented below¹⁸:

ACEP policy statement

Reprinted by permission of the American College of Emergency Physicians.

Commonly cited obstacles to ED HIV testing are insufficient time, burdensome consent process including pretest counseling, inadequate reimbursement, concerns for lack of patient acceptance of HIV screening, and concern for ensuring patient follow-up for confirmatory testing.⁹, ¹⁰, ¹⁹ Suggested solutions include providing information regarding HIV counseling and testing information as educational pamphlets handed out at triage, educational videos playing in the waiting areas, adoption of decreased pretesting counseling, and opt-out consent as permissible by individual state regulations. A more costly solution is hiring additional personnel dedicated to HIV counseling, testing, and ensuring follow-up for positive results.

Reimbursement for HIV testing in the ED is a potential barrier to instituting universal opt-out screening.⁷ In many published ED HIV screening studies, the study's financial sponsor absorbed the cost of testing. With a unit cost for rapid HIV testing of $14 to $25, EDs will need to be reimbursed if programs are to be financially feasible.³, ⁹ Until such time that insurers and governmental agencies guarantee reimbursement for the cost of tests, opt-in screening may be financially infeasible in many EDs.

Q.4.2 Discuss the potential advantages of initiating rapid HIV screening in the ED. How might this testing affect patient care in the ED?

Identification of patients with occult HIV infection is obviously an important method for reducing the spread of this disease. Many such patients are economically disadvantaged and may use the ED as their sole source of medical care. The high prevalence of undiagnosed HIV infection reported in some inner-city EDs supports this perspective.²⁰, ²¹, ²² The initiation of an HIV screening program in the ED provides an opportunity to diagnose a substantial number of undiagnosed infections in a group that may not be otherwise be identified. A collateral benefit of the screening process is the potential to raise HIV awareness and initiate HIV transmission prevention strategies.

Q.4.3 What are the advantages of knowing a patient's HIV status when treating him in the ED?

Unlike publicly funded testing programs that have reported that between 18% and 38% of patients fail to receive their test results,²³ rapid HIV testing provides results while the patient is still in the ED. This provides the health care provider the opportunity to inform the patient personally of preliminary positive results and immediately incorporate this probable diagnosis into medical decisionmaking. Common ED complaints such as headache, altered mental status, cough and fever, and flulike symptoms might require additional evaluation in a patient known to be HIV positive. For example, we know that individuals coinfected with HIV and tuberculosis often have atypical chest radiograph results.²⁴ A new diagnosis of presumptive HIV infection might broaden the potential differential diagnosis for the adolescent or young adult presenting with atypical chest pain, dyspnea, or unexplained syncope. For individuals who participate in high-risk activities for contracting HIV but test negative, this screening program might provide the ED physician an opportunity to counsel the patient personally on the serious consequences of his or her behavior.

Q.4.4 Are there any disadvantages?

There is legitimate concern that an ED rapid HIV screening program might increase patient length of stay both directly (waiting for test results, counseling patients with positive results, and arranging confirmatory testing) and indirectly by drawing resources away from other ED patient care activities. False-positive results might lead to unnecessary workups and hospitalizations as physicians react to the false-positive HIV test by working up symptoms that might otherwise have been ignored. Mechanisms need to be developed to record rapid HIV test results in the patient's medical record. Although negative tests can and should be recorded, positive tests must be noted in a manner that makes clear that the test is preliminary. Because the confirmatory test may well be done at another clinic, health department laboratory, or facility, there needs to be some mechanism for getting the confirmatory result into the patient's record.

The main medical harm from HIV testing is false-positive tests. We can think of no circumstances in which knowledge of the patient's true HIV status could be a disadvantage to the physician providing care.

Back to Article Outline

Answer 5 

Q.5.a The authors write that “A convenience sample of patients was approached by trained research assistants and asked to participate if they were awake, alert, not intoxicated and understood the premise of the study.” They also report that 529 (8%) of 6,965 adults seen in the ED during the study were enrolled and completed the survey. How might these factors affect one's interpretation of this study?

We sample for efficiency. It is too costly and inefficient to measure everyone. Random samples, when properly formed and analyzed, can tell us virtually all we want to know about the population. Care must be taken, however, to distinguish random sampling from the convenience sampling that was performed in the Haukoos et al study. Imagine a young research assistant tasked with enrolling patients for this study. In the hallway are 2 patients, one well dressed, smiling, highly approachable; the other, grungy and seemingly hostile. Who is more likely to be approached? Are both equally likely to provide the same answers? If the sampling is nonrandom and those who are enrolled differ from those who are not, then the possibility of sampling bias looms large. In a study like this one, a small amount of sampling bias is inevitable and may not be crucial. If the authors estimate a percentage as 81% when the population value is 80%, so what? Both numbers tell us that most people state they are willing to be tested. The minor discrepancy is inconsequential. Because sampling bias is inevitable in all studies that use convenience sampling, the reader should not be concerned with the question “Is there bias” but with the question “Could sampling bias be of sufficient magnitude to change the meaning of the results?” Formal sensitivity analysis is often the best way to answer this question.

Q.5.b Perform a sensitivity analysis that explores how the value offered for the percentage of subjects willing to be tested in the opt-out strategy—81%, 95% CI (77% to 84%)—would change if the patients who participated were 5, 10, 20, 40, 80, 100, 200, 500, and 1000% more likely to respond “yes” than those who were not sampled. What if the sample included 3,500 of the (roughly) 7,000 ED patients? Create a graph of your estimates for both sample sizes. Do this for other samples sizes if you wish.

For the purpose of this exercise, we round some numbers. We assume that there are 7,000 eligible subjects (instead of the 6,965 noted in the paper) and consider obtaining convenience samples of 500 (instead of the 527 in the paper), 3,500, 6,000, and 7,000 (100% sampling). This type of exercise is best done on a spreadsheet and the one used for this analysis is offered as a supplement to this article. To perform the sensitivity analysis for the 500-person sample, we start with the base case (we assume that the 81% rate seen in the 500 patients who were sampled would also have been observed in the 6,500 patients who were not surveyed). We then model what happens if the 6,500 patients were less likely to say yes than those who were interviewed. For example, if those interviewed were 5% more likely to say yes than those who were not, we would expect 77% (81%/1.05) of those not interviewed to say yes. The expected total number of affirmative responses in the population would be (.81×500)+(.77×6,500)=5,419. This represents 77.4% (5,419/7,000) of the population. We then repeat this calculation using different values for the difference in the proportion of “yes” responses between those sampled and not sampled. We can then graph our results:

We see that if we had sampled everyone (7,000 sampled), then we are immune to sampling bias because 81%, the proportion of “yes” responses, is by definition an unbiased estimate of the population value because it is the population value. We also see that as the sample size drops, the potential for grossly overestimating the response rate rises. If we assume that respondents are 200% more likely to say yes (they are 3 times more likely) and we sample 500 patients, then the true response rate could be as low as 30% (hollow arrow). We suspect, however, that this amount of sampling bias is unlikely. More reasonably, those sampled might be 20% more likely to answer in the affirmative and, if we sampled 500 patients, the population value would be 68% (solid arrowhead).

Q.5.c Question 3 explored the effect of sample size on the precision of the estimate; this sensitivity analysis explores the effect of sample size on the likelihood of sampling bias. Which consideration is more important in the Haukoos et al paper? Is the study's conclusion more vulnerable to imprecision or bias? What is the role of sensitivity analysis in observational studies?

A survey of the medical clinical research literature would find that almost all papers present comparative statistics but few present sensitivity analyses. Because virtually all classical statistics are concerned with the question “are the findings likely to be due to chance,” one might believe that imprecision rather than bias is the great threat to medical research. In most instances, however, bias poses a far greater risk to correct interpretation than imprecision.²⁵ As we saw in question 5.b, as a result of the convenience sampling, the study could have overestimated the population value by 10% or more. In question 3.b, we saw that random error alone would be unlikely to account for more than 5 percentage points of error. Thus, sampling bias rather than imprecision is the greater threat to the interpretation of the Haukoos et al study results. Our exercise highlights the importance of conducting a sensitivity analysis to determine how potential biases might alter the interpretation of a study's results.²⁶ We believe that in studies in which there is the potential for sampling bias or other forms of bias, the sensitivity analysis should be part of the formal published analysis.

Back to Article Outline

Answer 6 

Q.6.1 In your opinion, what are the most important conclusions from this paper? How might the limitations mentioned by the authors affect your decision whether to change your clinical practice with regard to initiating nontargeted opt-out HIV screening in your ED?

This paper reports that in an urban ED with a prevalence of undiagnosed HIV ranging from 0.7% to 2.2%, approximately 80% of adults would consent to free HIV testing in the ED whether by the opt-in or opt-out method. This study surveyed a convenience sample of nontargeted adult patients in the ED who were able to consent. Although there likely are differences in patient preference regarding HIV screening in other populations, this study provides ED directors and administrators with evidence that many patients in a representative urban population would accept free HIV testing.

This population was predominantly English speaking (92%) and white, with 28% Hispanic and 18% black patients. Results might differ in EDs that service a predominantly black or Spanish-speaking population. Perhaps a greater concern is that the Haukoos et al study asked patients about free HIV testing. We cannot assume that similar results would be achieved if patients were asked to pay for their test. The authors report that widespread HIV screening has been shown to be cost-effective; however, it is presently unknown whether the 2006 CDC recommendations will act as the impetus for both governmental and private insurers to cover HIV screening in the ED. Until reimbursement can be assured, most medical centers will need to obtain alternative modes of funding support for nontargeted HIV screening.

This survey also demonstrates that CDC-recommended opt-out screening was less easily understood by the subjects and more often required explanation than opt-in testing. The CDC maintains that HIV screening should remain voluntary and free of coercion.⁵ Therefore, medical centers that implement opt-out screening need to ensure that patients truly understand that they will be tested unless they refuse. Patients in this population were ambivalent on another CDC recommendation that the HIV consent should be encompassed within the general consent. As previously mentioned, medical centers need to adhere to their state's regulations regarding HIV counseling, consent, and testing.

Q.6.2 Would you have asked the patients questions covering any additional topics regarding HIV screening?

The authors tackled many important topics in their succinct survey. Given the limited funding for HIV screening, it would be of great interest to know whether patients would be as accepting of HIV screening if they had to pay for the testing. In addition, it would be important to know whether patients most likely to have HIV are least likely to consent for testing. That question could only be answered through a far more complex survey, one that collected detailed, accurate information on high-risk behaviors and whether the patient had had previous HIV tests.

Q.6.3 What additional information or data analyses would you like the authors to provide in order for you to change your clinical practice?

This is a straightforward survey, and the authors do a good job of concisely describing their findings. The sensitivity analysis (to see the potential effect of sampling bias) described in question 5.b could have been incorporated into the paper.

Back to Article Outline

References 

1. Haukoos JS, Hopkins E, Byyny RL. Patient acceptance of rapid HIV testing practices in an urban emergency department: assessment of the 2006 CDC recommendations for HIV screening in health care settings. Ann Emerg Med. 2008;51:303–309
   • View In Article
   • Abstract
   • Full Text
   • Full-Text PDF (184 KB)
   • CrossRef
2. Centers for Disease Control and Prevention. HIV counseling with rapid tests. http://cdc.gov/hiv/topics/testing/resources/factsheets/rt_counseling.htm 2007Accessed January 17, 2008
   • View In Article
3. Greenwald JL, Burstein GR, Pincus J, et al. A rapid review of rapid HIV antibody tests. Curr Infect Dis Rep. 2006;8:125–131
   • View In Article
4. Centers for Disease Control and Prevention. Rapid HIV testing in emergency departments—three U.S. sites, January 2005–March 2006. MMWR Morb Mortal Wkly Rep. 2007;56:597–601
   • View In Article
5. Branson BM, Handsfield HH, Lampe MA, et al. Revised recommendations for HIV testing of adults, adolescents, and pregnant women in health-care settings. MMWR Morb Mortal Wkly Rep. 2006;55:1–17
   • View In Article
6. Wolf LE, Donoghoe A, Lane T. Implementing routine HIV testing: the role of state law. PLoS ONE. 2007;2:e1005
   • View In Article
   • CrossRef
7. Haukoos JS, Hopkins E, Eliopoulos VT, et al. Development and implementation of a model to improve identification of patients infected with HIV using diagnostic rapid testing in the emergency department. Acad Emerg Med. 2007;14:1149–1157
   • View In Article
   • CrossRef
8. Brown J, Shesser R, Simon G, et al. Routine HIV screening in the emergency department using the new US Centers for Disease Control and Prevention guidelines: results from a high-prevalence area. J Acquir Immune Defic Syndr. 2007;46:395–401
   • View In Article
   • CrossRef
9. Brown J, Shesser R, Simon G. Establishing an ED HIV screening program: lessons from the front lines. Acad Emerg Med. 2007;14:658–661
   • View In Article
   • CrossRef
10. Silva A, Glick NR, Lyss SB, et al. Implementing an HIV and sexually transmitted disease screening program in an emergency department. Ann Emerg Med. 2007;49:564–572
    • View In Article
    • Abstract
    • Full Text
    • Full-Text PDF (136 KB)
    • CrossRef
11. Lyons MS, Lindsell CJ, Ledyard HK, et al. Emergency department HIV testing and counseling: an ongoing experience in a low-prevalence area. Ann Emerg Med. 2005;46:22–28
    • View In Article
    • Abstract
    • Full Text
    • Full-Text PDF (154 KB)
    • CrossRef
12. Harris SK, Samples CL, Keenan PM, et al. Outreach, mental health, and case management services: can they help to retain HIV-positive and at-risk youth and young adults in care?. Matern Child Health J. 2003;7:205–218
    • View In Article
    • MEDLINE
    • CrossRef
13. Sherer R, Stieglitz K, Narra J, et al. HIV multidisciplinary teams work: support services improve access to and retention in HIV primary care. AIDS Care. 2002;14(suppl 1):S31–S44
    • View In Article
14. Haukoos JS, Witt MD, Coil CJ, et al. The effect of financial incentives on adherence with outpatient human immunodeficiency virus testing referrals from the emergency department. Acad Emerg Med. 2005;12:617–621
    • View In Article
    • CrossRef
15. Centers for Disease Control and Prevention. Rapid HIV testing in outreach and other community settings—United States, 2004-2006. MMWR Morb Mortal Wkly Rep. 2007;56:1233–1237
    • View In Article
16. Lubelchek R, Kroc K, Hota B, et al. The role of rapid vs conventional human immunodeficiency virus testing for inpatients: effects on quality of care. Arch Intern Med. 2005;165:1956–1960
    • View In Article
    • MEDLINE
    • CrossRef
17. Centers for Disease Control and Prevention. Rapid HIV testing in emergency departments—three U.S. sites, January 2005-March 2006. MMWR Morb Mortal Wkly Rep. 2007;56:597–601
    • View In Article
18. ACEP. Policy statement: HIV testing and screening in the emergency department. Ann Emerg Med. 2007;50:209
    • View In Article
    • Full Text
    • Full-Text PDF (44 KB)
    • CrossRef
19. Burke RC, Sepkowitz KA, Bernstein KT, et al. Why don't physicians test for HIV? (A review of the US literature). AIDS. 2007;21:1617–1624
    • View In Article
    • CrossRef
20. Kelen GD, Hexter DA, Hansen KN, et al. Trends in human immunodeficiency virus (HIV) infection among a patient population of an inner-city emergency department: implications for emergency department-based screening programs for HIV infection. Clin Infect Dis. 1995;21:867–875
    • View In Article
    • MEDLINE
    • CrossRef
21. Baraff LJ, Talan DA, Torres M. Prevalence of HIV antibody in a noninner-city university hospital emergency department. Ann Emerg Med. 1991;20:782–786
    • View In Article
    • Abstract
    • Full-Text PDF (504 KB)
    • CrossRef
22. Kelen GD, Shahan JB, Quinn TC. Emergency department-based HIV screening and counseling: experience with rapid and standard serologic testing. Ann Emerg Med. 1999;33:147–155
    • View In Article
    • Abstract
    • Full Text
    • Full-Text PDF (97 KB)
    • CrossRef
23. Lyss SB, Branson BM, Kroc KA, et al. Detecting unsuspected HIV infection with a rapid whole-blood HIV test in an urban emergency department. J Acquir Immune Defic Syndr. 2007;44:435–442
    • View In Article
    • MEDLINE
    • CrossRef
24. Geng E, Kreiswirth B, Burzynski J, et al. Clinical and radiographic correlates of primary and reactivation tuberculosis: a molecular epidemiology study. JAMA. 2005;293:2740–2745
    • View In Article
    • CrossRef
25. Schriger D. Problems with current methods of data analysis and reporting, and suggestions for moving beyond incorrect ritual. Eur J Emerg Med. 2002;9:203–207
    • View In Article
    • MEDLINE
    • CrossRef
26. Greenland S. Basic methods for sensitivity analysis of biases. Int J Epidemiol. 1996;25:1107–1116
    • View In Article
    • MEDLINE
    • CrossRef