Raise the Flag I: the impact of a sepsis quality improvement programme on delivery of a sepsis resuscitation bundle at a tertiary hospital in New Zealand

Global epidemiological studies suggest that sepsis may contribute directly, or indirectly, to as many as 20% of deaths world-wide.¹ In New Zealand, sepsis exerts a significant burden of cost and population morbidity, with Māori and Pasifika people, the elderly and those experiencing socio-economic disadvantage most at risk.² System-wide efforts to improve sepsis recognition and outcomes are a crucial response to this challenge.

Translation of best practice clinical guidelines into practice is facilitated using care bundles. Longitudinal studies show that it is possible to improve sepsis care using these bundles. For example, the “Sepsis Kills” programme was associated with a 22% increase in the delivery of antibiotic therapy within 60 minutes of arrival in participating emergency departments in New South Wales between 2011 and 2013.³ Prompt receipt of a sepsis resuscitation bundle is associated with reduced mortality. Mortality after Sepsis Kills fell from 19.3% to 14.1%. In the United Kingdom (UK), an observational study reported by Daniels et al. showed that the receipt of a sepsis resuscitation bundle within 1 hour was associated with a mortality of 20%, compared to a mortality of 44.1% in those who did not receive it.⁴  In response to this and other evidence, the National Institute of Clinical Excellence published guidance recommending screening and resuscitation of sepsis based on the presence of clinical findings associated with a high risk of in-hospital mortality.⁵

In 2018, New Zealand adopted these recommendations as a national standard for sepsis care. This provided the opportunity to develop, implement and study the performance of a sepsis screening and action tool within a whole-of-system quality improvement programme. Introduced to public hospitals in the Waikato Region, the whole sepsis advocacy and change programme became known as “Raise the Flag”. Within this, collaboration with the UK Sepsis Trust (UKST) led to adoption of the UKST Red Flag Sepsis Screening Tool and the Sepsis Six, which was modified to suit practice in our setting. The Raise the Flag programme (available at www.sepsis.org.nz) aimed to empower frontline clinical staff to deliver the sepsis resuscitation bundle. We conducted a pre- and post-implementation evaluation of the Red Flag Sepsis Screening Tool and the Sepsis Six at Waikato Hospital, a 600-bed, publicly funded, tertiary-level academic hospital in the North Island of New Zealand.

Methods

Setting

A multi-disciplinary Sepsis Action Group (SAG) was established in 2016. The SAG consisted of clinical champions, quality improvement experts, senior executives and data analysts. To lead and sustain programme implementation, a nurse coordinator was appointed in 2018. The Red Flag Sepsis Screening Tool and the Sepsis Six were launched to all clinical areas in Waikato Hospital in August 2018. A package of interventions aimed at changing clinical behaviour included a sepsis e-learning package for all clinical staff, the addition of sepsis screening prompts to all vital sign charts, and commissioning of a multi-media design package to increase programme visibility in clinical and non-clinical areas.

Direct feedback on Sepsis Six compliance in individual cases admitted to high dependency units (HDUs) or intensive care units (ICUs) was provided to clinical teams via email from the sepsis nurse coordinator during 2019 and 2020. Audit results were presented to the SAG in July 2018, July 2019 and September 2020, and to the hospital via a grand round presentation in August 2019 and August 2022, coinciding with yearly hospital-wide promotion of World Sepsis Day. A sepsis newsletter was circulated to all staff quarterly from December 2018.

Case definition and audit strategy

The study was registered prospectively with the Waikato Hospital Quality and Patient Safety office. As a low-risk observational study, it was considered exempt from Health and Disability Ethics Committee review.

We identified potential cases of sepsis using the New Zealand Sepsis Indicator (NZSI).^2,6 This makes use of the International Statistical Classification of Diseases and Related Health Problems, 10th Revision, Australasian Modification (ICD-10-AM) to identify patients in hospital discharge data who have both a primary infection diagnosis and a secondary diagnosis consistent with organ failure. Eighty-six percent of cases identified in this way satisfy the third international consensus definition of sepsis.^{6, 7}

A random number-generating algorithm was used to select 10 cases a month, satisfying NZSI criteria to review. Selected cases went forward for full data collection, where clinician documentation of infection and at least one high-risk clinical finding (red flag) were found together. The red flags used to select cases for this study are the same as those in the Red Flag Sepsis Screening tool that qualifies patients for receipt of the Sepsis Six, and are: responds only to voice or pain or unresponsive; systolic blood pressure less than or equal to 90mmHg; heart rate more than 130 beats per minute; respiratory rate more than or equal to 25 breaths per minute; needs oxygen to keep saturations more than or equal to 92%; non-blanching rash, mottled, ashen or cyanotic; not passed urine in the last 18 hours; urine output less than 0.5 ml/kg an hour; lactate more than or equal to two; and receipt of recent chemotherapy. The earliest recorded time where both were present was termed “time zero” (T0).

Excluded were those aged <15, those admitted only for palliative management and those transferred from other hospitals. We audited continuously from December 2017 to May 2019. The pre-implementation group (subsequently referred to as Group 1) represents cases presenting to Waikato Hospital between October 2017 and July 2018. The post-implementation group (Group 2) includes cases presenting between August 2018 and May 2019. To assess whether changes were sustained, we audited throughout calendar year 2021 (Group 3).

Variables

Our primary outcome measure was completion of the first five components of the Sepsis Six bundle (“the sepsis bundle”) within 3 hours. The final component of the Sepsis Six bundle, measure urine output, was excluded as fluid balance charts are not routinely filed and this could not be determined reliably in our retrospective audit.  The included actions are: administer oxygen; take blood cultures; give intravenous (IV) antibiotics; give IV fluids; and check serum lactate. In accordance with advice provided on the Sepsis Six tool, oxygen delivery was deemed mandatory only if oxygen saturations were <94%, and a fluid bolus only if systolic blood pressure was <90mmHg or the serum lactate was ≥2mmol/l. The time to receipt of each item was recorded where these data were available.

Our secondary outcome was the association of Māori/Pasifika ethnicity with the delivery of the sepsis bundle to assess for equitable roll out. Other secondary outcomes included: the number of red flags present for each patient; location of the patient when sepsis was diagnosed; location of hospital placement after recognition of sepsis; source of sepsis; 30-day mortality and Charlson Comorbidity Index.⁸ The association between the Raise the Flag programme and ICU admissions was a post hoc analysis to examine the wider impact of the programme.

Data collection

A pre-specified data collection sheet, including definitions, was used to standardise data collection and all data collectors were trained in its use. Data on red flags, mode of transport to hospital, hospital location, infectious diagnosis and delivery of the sepsis bundle were determined using paper and electronic records. Demographic and ethnicity data were collected using iPM (iPatient Manager, DXC Technology, Tysons Corner, United States of America [USA]). We used each patient’s national health identifier (NHI) to determine comorbidity index and mortality 30 days following T0. All ambiguities were reviewed and resolved by a second investigator (KW).  

Statistical analysis

Audit data were collected in Microsoft Excel (Microsoft Corporation, Redmond, USA). Simple statistics were used to describe data. Pearson’s Chi-squared test was used to compare groups containing categorical and binary data. Mantel–Haenszel odds ratios (OR) were calculated for associations of possible confounders with delivery of the sepsis bundle within 3 hours. Variables associated with either the exposure or outcome variable with p<0.1 were included in multivariate logistic regression. All data analysis was performed in STATA version 16 (StataCorp, College Station, USA). As this was an audit of an intervention established as effective overseas, sample size was determined pragmatically by the resources available to collect data.

View Table 1–4.

Results

In total, 610 records were selected for review. Of these, 133 were excluded (98 presented to another hospital, 13 were children, 22 were for palliative care only). Of the remaining 477 records, 71 (14.9%) had no red flags, and 21 (4.4%) had no documentation of infection. We collected complete data for 385 eligible cases: 117 patients in Group 1, 149 in Group 2 and 119 in Group 3. Key demographic and clinical variables for these patients are shown in Table 1.

The average age was 67±18 years; this was 9 years lower at 58± 16 years in patients of Māori or Pasifika ethnicity. Eighty-seven (23%) people died within 30 days of T0.

Table 2 describes the infection-related characteristics of our cohort. Two hundred and eighty-five (74%) patients arrived by ambulance. Three hundred and eleven (81%) patients were under the care of the emergency department at T0. Six percent of patients in Group 1 and 14% of patients in Group 3 were admitted directly to the ICU after sepsis diagnosis.

Tables 1 and 2 show the association of potential confounding variables with pre- and post-implementation periods. Patients were more likely to present with haemodynamic instability in the pre-implementation group than subsequent groups (p<0.001). They were more likely to be older than 75 years (p=0.05) and present with skin, soft tissue, bone and joint infection (p=0.098).

We performed a univariate analysis of the association between potential confounding variables and the receipt of the sepsis bundle within 3 hours. Age ≥75 was associated with a reduced odds of sepsis bundle delivery (OR 0.58, p=0.01). The presence of haemodynamic instability (OR 1.71, p=0.01), three or more red flags (OR 2.05, p=0.001), arrival by ambulance (OR 1.99, p=0.003) and being under emergency department at T0 (OR 3.81, <p=0.001) were associated with increased odds of sepsis bundle delivery. There was no evidence to support a crude association between gender, Charlson Comorbidity score or ethnicity and delivery of the bundle. Noting inter-ethnic differences in population age structure, we used Mantel–Haenszel methods to look for an association between Māori/Pasifika ethnicity and receipt of the sepsis bundle adjusted for age by decade. In this analysis, Māori/Pasifika ethnicity was associated with reduced odds of sepsis bundle delivery (OR 0.55, 95% confidence interval [CI] 0.33–0.91, p=0.018).

On the basis of univariate associations, we performed a logistic regression adjusting for 10-year age group, Māori/Pasifika ethnicity, final diagnosis, the presence of haemodynamic instability, the presence of three or more red flags at T0, arrival by ambulance and management under ED. Table 3 shows the associations of these potential confounding factors with delivery of the sepsis bundle across the whole study population.

Being under emergency medicine at T0 was associated with an increased adjusted odds ratio (aOR) for delivery of the sepsis bundle (aOR 3.33, 95% CI 1.85–5.98, p<0.001). Age was negatively associated with bundle completion. For every increase in 10-year age group, the odds of receiving the bundle fell by 17% (aOR 0.83, 95% CI 0.73–0.95, p=0.005).

Table 4 shows the crude and adjusted association between Group 1 and Group 2 and between Group 1 and Group 3 in delivery of the sepsis bundle. In the unadjusted analysis, clinicians in the post-implementation period (Group 2) were more likely to deliver the sepsis bundle within 3 hours than those in pre-implementation Group 1 (OR 1.79, 95% CI 1.09–2.95, p=0.02). There was no difference in sepsis bundle delivery comparing Group 3 and Group 1 (OR 1.07, 95% CI 0.64–1.78, p=0.8).

In the adjusted analysis there remained a significant positive association between the post-implementation period and delivery of the sepsis bundle (aOR 2.20, 95% CI 1.27–3.78, p=0.005). Treatment in 2021 (Group 3) was not associated with an increased odds of sepsis bundle delivery over baseline (aOR 1.22, 95% CI 0.89–1.66, p=0.21).

In a post hoc analysis we assessed whether the implementation of the Raise the Flag programme was associated with admission to our ICU. The crude OR for ICU admission comparing the post-implementation groups (Groups 2 and 3) with the pre-implementation group (Group 1) was 2.36 (95% CI 1.01–5.51, p=0.04). Age group, the presence of haemodynamic instability, being under emergency department at T0 and number of red flags were all associated with admission to ICU with a p-value of <0.1. In multivariate analysis, the association between post-implementation periods and admission to the ICU remained significant (aOR 2.81, 95% CI 1.13–6.97, p=0.03).

Discussion

A comprehensive, hospital-wide sepsis initiative was associated with improvements in delivery of a sepsis resuscitation bundle at our hospital. This improvement was not sustained at 18 to 30 months. In assessment of secondary and post hoc end points, important findings were revealed with respect to clinician and system performance. Delivery of treatment by an emergency medicine team increased the odds of sepsis bundle delivery (aOR 3.33, 95% CI 1.85–5.98, p<0.001). Increasing age significantly reduced sepsis bundle completion, despite excluding treatment ineligible patients and adjusting for both haemodynamic instability and Charlson Comorbidity Index (aOR 0.83 for every 10 years of age, 95% CI 0.73–0.95, p=0.005). The odds of being admitted to ICU (the only area in our hospital we deliver vasoactive medications) increased in the combined post-implementation groups (aOR 2.81, 95% CI 1.13–6.97, p=0.026). We suggest that the increased rates of admission to ICU show that, despite a drift to baseline in terms of immediate sepsis bundle delivery, the Raise the Flag programme had wider impacts that improved sepsis care beyond 2019.

The strength of our study is the description of, and adjustment for, confounding factors. This enabled comparison between groups that were not matched in important variables and allowed us to investigate the factors that influence delivery of the bundle to target ongoing interventions. For example, the Red Flag Sepsis Screening Tool was updated in 2022 to include Māori ethnicity as an “amber flag” to highlight excess risk in this group. The major limitation of our study is the before and after design. Data for Group 3 were collected during the COVID-19 pandemic and may have been particularly affected by residual confounding. Whether the lower rates of haemodynamic instability in the 2021 cohort is a sampling phenomenon or a real effect is not clear. During this period, New Zealand had restrictions on large gatherings and encouraged the use of masks in public. Widespread community transmission of COVID-19 didn’t occur until early 2022. COVID-19 containment measures have been shown to reduce blood stream infections with organisms transmitted by droplet spread, such as Streptococcus pyogenes, overseas.^9,10 Surveillance data show that the rates of both invasive pneumococcal disease and invasive Group A Streptococcal disease were lower in 2020 and 2021 in New Zealand compared to previous years.^11,12 It is possible that both behavioural change and a change in the microbiology of sepsis had an impact on the presentation of sepsis, and more research is required in this area.

The results of this study are consistent with the results of similar programmes in New South Wales and world-wide, which show improvement in the delivery of sepsis care after their implementation.^3,13,14 Fifty-six percent of our patients received the sepsis bundle in 3 hours, which compares well with the literature referenced.^13–15 The most successful sepsis quality improvement projects combine process change and educational activities, dedicated sepsis teams and supportive environmental contexts and resources.^15,17 The reduction in bundle delivery in Group 3 coincides with the end of direct feedback to clinical teams and suggests that feedback and education must be sustained over time to embed the change in routine practice.

The 9-year younger mean age of patients of Māori or Pasifika ethnicity compared to the study population average is consistent with existing evidence that sepsis is both a result and a potentiator of health inequity in New Zealand. We did not find that ethnicity was associated with sepsis bundle delivery; however, the crude and adjusted ORs were below 1, and this sample size would not detect a small difference in bundle delivery. It would be naïve to think that sepsis interventions are unaffected by the various forms of bias and systemic racism resulting in variation in practice described in other conditions, and this will continue to be monitored at our institution.^18,19

We have shown that age is associated with reduced odds of receiving the sepsis resuscitation bundle (aOR for every 10-year increase in age 0.83, 95% CI 0.73–0.95, p=0.005). In a previous report, we have shown that the NZSI identifies more neurologic and renal organ failure with age, and less respiratory failure.² Normothermia and hypothermia are more common with age. This may translate to differences in the clinical cues used to prompt action. However, this study made use of red flags that should have triggered action regardless of age. Delay of over 3 hours in the administration of antimicrobials in sepsis is associated with increased risk of death in observational studies, is inconsistent with best practice guidelines and would not be considered appropriate for treatment-eligible adults.^20–22 Given the higher mortality in older patients, there may be more to gain in this group from prompt antimicrobial and haemodynamic management.

In conclusion, a system-wide sepsis programme at our hospital produced changes in early sepsis management and revealed evidence of differential care based on age. Embedding and sustaining change in a complex system requires ongoing education and support, as well as optimisation of environmental contexts and resources to enable best practice. We regard an appropriate increase in ICU utilisation as an ongoing success and continue to investigate whether the wider impacts of the programme included effects on mortality and hospital length of stay.