@RISK used by Interdisciplinary Solutions to help the New York City Department of Health and Mental Hygiene prepare for future pandemics

The New York City Department of Health and Mental Hygiene (NYC DOHMH) is one of the world's oldest and largest public health agencies, with an annual budget of $1.6 billion and more than 6,000 staff.

In October 2008, its Health Emergency Preparedness Program, in collaboration with the Greater New York Hospital Association (GNYHA), was awarded funds by the Center for Disease Control and Prevention (CDC) to develop new strategies to mitigate a severe influenza pandemic should one occur. A key objective was to understand how essential healthcare services and their delivery would be affected.

Interdisciplinary Solutions executed the quantitative side of this year long public health project using its Panálysis model which projects healthcare demands during a pandemic to include critical product requirements and staffing needs. This epidemiologic model integrates Palisade’s risk and decision analysis software @RISK and RISKOptimizer in order to account for uncertainty. 

@RISK used to assess nursing capacity
A wide range of data was extracted from New York City’s 64 hospitals and, where data was not available, expert opinion and peer reviewed literature were used to define ranges for the inputs to the @RISK-based portions of the model. From there, Interdisciplinary Solutions generated a multitude of 'what-if' scenarios by using Monte Carlo Simulation to randomly select values for the input variables so that the results from each demonstrated a range of possible outcomes. The following example shows the utility of Monte Carlo simulation as performed through @RISK to determine nurse staffing needs during a severe pandemic.
 
Nurses are amongst the most critical working assets employed during a pandemic and their availability can significantly impact hospitals’ provision of service during an emergency situation. However, during an influenza pandemic, there is substantial uncertainty that surrounds nurse availability. To model and determine the nurse capacity that might be required during a pandemic to care for all hospital patients, Interdisciplinary Solutions took a number of variables into account. These incorporated the total number of nurses working in partner hospitals, including those involved in direct patient care and in a supervisory and administrative capacity, as well as full-time registered nurses and licensed practical nurses.

Additional consideration was given to the number of hospital beds, the number of shifts a nurse can perform in a week and the total number of shifts available in a week dependent on pre-pandemic occupancy. Nurse absenteeism on account of their own family members suffering from the disease or their desire to avoid exposing themselves to the virus, was also taken into account.

As such, Interdisciplinary Solutions used @RISK to study the differences in potential shortage levels and the effect of interventions designed to mitigate such shortages. One such intervention is the Agency for Healthcare Quality and Research’s (AHRQ) recommended nurse-patient modified standards of care ratios. (These are 16:1 for non-ICU patients and 5:1 for ICU patients).

After determining the actual nurse patient ratios in various parts of the hospital, Interdisciplinary Solutions studied the effect of using the AHRQ’s recommended modified standards of care nurse patient ratios for emergency situations where nursing shortages exist. By generating tens of thousands of results it was demonstrated that a middle case scenario can be very misleading in emergency preparedness. While the mean result (i.e. average) of all modelled interventions eliminates shortages, simulation demonstrates that in a sample hospital during peak pandemic weeks, even with interventions there would be:

• A 26.5% chance that shortages would exist
• A 10% chance of shortages of 46 nurses or above
• A 5% chance of shortages of 67 nurses or above
• A 1% chance of shortages of 102 nurses of above
• A maximum potential shortage level of 149 nurses

Graph 1: Nurse overages and shortages as measured by full time equivalents using modified standards of care and all available interventions

RISKOptimizer used to create critical resource capacity scenarios for New York City hospitals
Planning for severe pandemics is speculative especially because there is little retrospective data available on which to base projections. Therefore, rather than be confined by the limitations of historical statistical analysis, Interdisciplinary Solutions used RISKOptimizer, Palisade's simulation optimisation add-in for Microsoft Excel®, to understand the just how severe the effects of pandemics on critical resources can be.

Given the limited availability of historical data, in close coordination with a team of experts at the NYC DOHMH and the New York City pandemic Emergency Medical Services committee, Interdisciplinary Solutions populated 11 disease profile variables. These included: percentage of population infected seeking medical help; percentage of population actually requiring hospitalisation; average length of stay of non-ICU patients; and ICU fatality rate assuming no bed is available and that there are ventilator shortages. Inputs used realistic but deliberately broad ranges to account for the lack of historical data points.

Rather than use RISKOptimizer to determine the minimum or maximum shortage levels for any individual hospital, Interdisciplinary Solutions input the variables above into the model to determine the minimum and maximum ranges used to define the disease’s characteristics. From this, different plausible scenarios were created to ascertain resource requirements for ICU and non-ICU beds, ventilators and emergency department capacity that New York City hospitals would need in each of those eventualities. To ensure accuracy, Interdisciplinary Solutions, ran extensive simulations (up to half a million in some instances) to ascertain a minimum or maximum point to define the range. As such, these techniques allowed the NYC DOHMH to take a logical scientific approach to a task that would otherwise rely heavily on guess work.

“By nature, emergency preparedness situations are speculative and therefore not predictable. At the same time they are extremely high risk because peoples' lives are in danger. The key to handling such events is to acknowledge that our predictive abilities are limited and, with that in mind, to use quantitative methods to study a multitude of possibilities,” explains Mark Abramovich, principal at Interdisciplinary Solutions. “Palisade's @RISK and RISKOptimizer are designed to make it easy to do this, and the results generated enable organisations to plan for a wide range of scenarios.”

Graph 2: Values for disease profile characteristics that produce minimum and maximum shortage levels derived from RISKOPtimizer

This chart developed from the results of RISKOptimizer shows the maximum and minimum values for the 11 disease profile variables that correlate to minimum or maximum shortage levels of ICU and non-ICU beds, ventilators and emergency room visits based on a pandemic of similar severity to the one that occurred in 1918. The “Harvey Ball” portion demonstrates which quintile within the possible range of values the result fell.

Conclusion
The results of the study are now being reviewed by the CDC.

Building on this project, there is now potential to extend this study to include other variables in emergency preparedness such as pharmaceuticals, staff members other than nurses and supporting supplies (e.g. oxygen for ventilators) that could result in equipment not functioning properly. In addition, there is scope to analyse the effects of the geographical, spatial and temporal spread of the disease.

Additional Information

Distributions used
A severe pandemic is a low probability, high impact, event. When dealing with instances of this kind, use of distributions such as the normal distribution can be misleading (and potentially dangerous). This is because classic distribution patterns, such as the normal distribution are predicated on the idea that if there are an infinite number of historical data points, or at least many, many data points one can fit a curve to the distribution that best matches the pattern of historical data. However, historically major pandemics appear roughly once every 70 to 100 years and hence data collection is not as robust. Therefore, Interdisciplinary Solutions used expert opinion to determine some initial ranges since the consultancy only had the pandemic event of 1918 from which to work. For RISKOptimizer, a triangular distribution was used to acknowledge that there isn’t a historical distribution pattern available. Where @RISK was used, the consultancy used the Latin Hypercube method seeded with a random initial seed that fell within its estimated range.

Additional background information:

1. An influenza pandemic is a global disease outbreak that occurs when a new strain of influenza virus emerges for which people have little or no immunity and for which there is no vaccine. The most severe pandemic influenza in recent history was the Avian Flu of 1918, which killed 660,000 people in the US out of a population of approximately 108 million. Typically, severe influenza pandemics occur roughly every 100 years, so it is imperative that the healthcare system is geared to deal with the intense surge in demand for medical services.

2. For the NYC DOHMH study, Interdisciplinary Solutions collected data from 64 New York City hospitals and worked with three partner hospitals to perform in-depth analysis. 1918 was considered to be the benchmark for this study to measure the impact of a future pandemic on medical services in New York City but given the then current H1N1 threat, actual lessons from that experience were incorporated into the work.

3. By coincidence, during the study, the H1N1 variant (now deemed mild) of the flu struck the globe and New York City, a region that in many ways was considered the pandemic’s epicentre in the United States. 

4. Interdisciplinary Solutions' Panálysis model was initially developed in collaboration with the Center for Biosecurity at the University of Pittsburgh Medical Center, who provided expertise on a pro bono basis to better advance its public health and defence orientated mission. Since then, advancements include integrating risk and decision analysis software @RISK and RISKOptimizer from Palisade.

This publication was made possible by Grant Number 1U90TP000138-01 from the Centers for Disease Control and Prevention to Public Health Solutions on behalf of the New York City Department of Health and Mental Hygiene Healthcare Emergency Preparedness Program Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention or Public Health Solutions or the New York City Department of Health and Mental Hygiene Healthcare Emergency Preparedness Program.

» @RISK