Advertisement

Predicting Ambulance Patient Wait Times: A Multicenter Derivation and Validation Study

      Study objective

      To derive and internally and externally validate machine-learning models to predict emergency ambulance patient door–to–off-stretcher wait times that are applicable to a wide variety of emergency departments.

      Methods

      Nine emergency departments provided 3 years (2017 to 2019) of retrospective administrative data from Australia. Descriptive and exploratory analyses were undertaken on the datasets. Statistical and machine-learning models were developed to predict wait times at each site and were internally and externally validated.

      Results

      There were 421,894 episodes analyzed, and median site off-load times varied from 13 (interquartile range [IQR], 9 to 20) to 29 (IQR, 16 to 48) minutes. The global site prediction model median absolute errors were 11.7 minutes (95% confidence interval [CI], 11.7 to 11.8) using linear regression and 12.8 minutes (95% CI, 12.7 to 12.9) using elastic net. The individual site model prediction median absolute errors varied from the most accurate at 6.3 minutes (95% CI, 6.2 to 6.4) to the least accurate at 16.1 minutes (95% CI, 15.8 to 16.3). The model technique performance was the same for linear regression, random forests, elastic net, and rolling average. The important variables were the last k-patient average waits, triage category, and patient age. The global model performed at the lower end of the accuracy range compared with models for the individual sites but was within tolerable limits.

      Conclusion

      Electronic emergency demographic and flow information can be used to estimate emergency ambulance patient off-stretcher times. Models can be built with reasonable accuracy for multiple hospitals using a small number of point-of-care variables.
      To read this article in full you will need to make a payment
      ACEP Member Login
      ACEP Members, full access to the journal is a member benefit. Use your society credentials to access all journal content and features.
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Cooney D.R.
        • Millin M.G.
        • Carter A.
        • et al.
        Ambulance diversion and emergency department offload delay: resource document for the National Association of EMS Physicians position statement.
        Prehosp Emerg Care. 2011; 15: 555-561https://doi.org/10.3109/10903127.2011.608871
        • Kingswell C.
        • Shaban R.Z.
        • Crilly J.
        Concepts, antecedents and consequences of ambulance ramping in the emergency department: a scoping review.
        Australas Emerg Nurs J. 2017; 20: 153-160https://doi.org/10.1016/j.aenj.2017.07.002
        • Crilly J.
        • Keijzers G.
        • Tippett V.
        • et al.
        Improved outcomes for emergency department patients whose ambulance off-stretcher time is not delayed.
        Emerg Med Australas. 2015; 27: 216-224https://doi.org/10.1111/1742-6723.12399
      1. Walker K, Stephenson M, Loupis A, et al. Displaying emergency patient estimated wait times: a multi-centre, qualitative study of patient, community, paramedic and health administrator perspectives. Emerg Med Australas. https://doi.org/10.1111/1742-6723.13640

        • Cooney D.R.
        • Wojcik S.
        • Seth N.
        Can NEDOCS score be used to predict ambulance offload delay?.
        Ann Emerg Med. 2011; 58: S217https://doi.org/10.1016/j.annemergmed.2011.06.144
        • Cooney D.R.
        • Wojcik S.
        • Seth N.
        • et al.
        Evaluation of ambulance offload delay at a university hospital emergency department.
        Int J Emerg Med. 2013; 6: 15https://doi.org/10.1186/1865-1380-6-15
        • Hargreaves D.
        • Snel S.
        • Dewar C.
        • et al.
        Validation of the National Emergency Department Overcrowding Score (NEDOCS) in a UK non-specialist emergency department.
        Emerg Med J. 2020; 37: 801-806https://doi.org/10.1136/emermed-2019-208836
        • Weiss S.J.
        • Derlet R.
        • Arndahl J.
        • et al.
        Estimating the degree of emergency department overcrowding in academic medical centers: results of the National ED Overcrowding Study (NEDOCS).
        Acad Emerg Med. 2004; 11: 38-50https://doi.org/10.1197/j.aem.2003.07.017
        • Mengyu L.
        Designing emergency medical services processes to minimize the impact of ambulance offload delay. Dalhousie University Web site.
        (Accessed March 26, 2021.)
        • Yang W.
        • Qiang S.
        • Huang S.
        • et al.
        Simulation modeling and optimization for ambulance allocation considering spatiotemporal stochastic demand.
        Journal of Management Science and Engineering. 2019; 4: 252-265https://doi.org/10.1016/j.jmse.2020.01.004
        • Senderovich A.
        • Beck J.C.
        • Gal A.
        • et al.
        Congestion graphs for automated time Predictions.
        Proc Conf AAAI Artif Intell. 2019; 33: 4854-4861
        • Arha G.
        Reducing Wait Time Prediction In Hospital Emergency Room: Lean Analysis Using a Random Forest Model.
        Thesis. University of Tennessee, 2017
        • Ang E.
        • Kwasnick S.
        • Bayati M.
        • et al.
        Accurate emergency department wait time prediction.
        Manuf Serv Oper Manag. 2016; 15: 170-175
        • Sun Y.
        • Teow K.L.
        • Heng B.H.
        • et al.
        Real-time prediction of waiting time in the emergency department, using quantile regression.
        Ann Emerg Med. 2012; 60: 299-308https://doi.org/10.1016/j.annemergmed.2012.03.011
      2. State government of Victoria. VEMD user manual 2018-19, section 3, data definitions. Accessed November 28, 2020. https://www2.health.vic.gov.au/about/publications/policiesandguidelines/vemd-manual-2018-19-sec-3-data-definitions

        • Gori M.
        Machine Learning: A Constraint-Based Approach.
        Elsevier, 2018 (5, 6, 86)
        • Dong J.
        • Yom-Tov E.
        • Yom-Tov G.B.
        The impact of delay announcements on hospital network coordination and waiting times.
        Manage Sci. 2019; 65: 1969-1994
        • Fox J.
        Applied Regression Analysis and Generalized Linear Models.
        Sage Publications, 2015
        • Hyndman R.J.
        • Athanasopoulos G.
        Forecasting: Principles and Practice.
        2nd ed. OTexts, 2018
        • Cho I.
        • Boo E.-H.
        • Chung E.
        • et al.
        Novel approach to inpatient fall risk prediction and its cross-site validation using time-variant data.
        J Med Internet Res. 2019; 21: e11505https://doi.org/10.2196/11505
        • Tantithamthavorn C.
        • McIntosh S.
        • Hassan A.
        • et al.
        Impact of automated parameter optimization on defect prediction models.
        IEEE Transactions on Software Engineering. 2019; 45: 683-711https://doi.org/10.1109/TSE.2018.2794977
        • Cliff N.
        Dominance statistics: ordinal analyses to answer ordinal questions.
        Psychol Bull. 1993; 114: 494-509
        • Romano J.
        • Kromrey J.
        • Coraggio J.
        • et al.
        Appropriate statistics for ordinal level data: Should we really be using t-test and Cohen’s d for evaluating group differences on the NSSE and other surveys?.
        Florida Association of Institutional Research Annual Meeting, 2006: 1-33 (Accessed April 26, 2021.)
        • Xie B.
        • Youash S.
        The effects of publishing emergency department wait time on patient utilization patterns in a community with two emergency department sites: a retrospective, quasi-experiment design.
        Int J Emerg Med. 2011; 4: 29https://doi.org/10.1186/1865-1380-4-29