Kallen and Prins: A Systematic Review of Quality Indicators for Appropriate Antibiotic Use in Hospitalized Adult Patients

A Systematic Review of Quality Indicators for Appropriate Antibiotic Use in Hospitalized Adult Patients

Abstract

Many quality indicators for appropriate antibiotic use have been developed. We aimed to make a systematic inventory, including the development methodology and validation procedures, of currently available quality indicators (QIs) for appropriate antibiotic use in hospitalized adult patients. We performed a literature search in the Pubmed interface. From the included articles we abstracted i) the indicators developed ii) the type of infection the QIs applied to iii) study design used for the development of the QIs iv) relation of the QIs to outcome measures v) whether the QIs were validated and vi) the characteristics of the validation cohort. Fourteen studies were included, in which 200 QIs were developed. The most frequently mentioned indicators concerned empirical antibiotic therapy according to the guideline (71% of studies), followed by switch from IV to oral therapy (64% of studies), followed by drawing at least two sets of blood cultures and change to pathogen-directed therapy based on culture results (57% of studies). Most QIs were specifically developed for lower respiratory tract infection, urinary tract infection or sepsis. A RAND-modified Delphi procedure was used in the majority of studies (57%). Six studies took outcome measures into consideration during the procedure. Five out of fourteen studies (36%) tested the clinimetric properties of the QIs and 65% of the tested QIs were considered valid. Many studies report the development of quality indicators for appropriate antibiotic use in hospitalized adult patients. However, only a small number of studies validated the developed QIs. Future validation of QIs is needed if we want to implement them in daily practice.




Competing interest statement

Conflict of interest: the authors declare no potential conflict of interest.

Background

Today antibiotics are indispensable in practically all health care systems.1 However, the extensive use of antibiotics is also the main driving force in the emergence of resistant microorganisms.2 Worldwide, antibiotic consumption and antibiotic resistance are still on the rise, which, together with the decline in the discovery of new antibiotics, creates one of the greatest current threats to human health.2-5

To curb the rise of antibiotic resistance of medically important bacteria, better use of current agents is warranted and a decrease of inappropriate antibiotic use is imperative.3 Antibiotic stewardship programs are developed to optimize the appropriateness of antibiotic use, in order to maximize the chance of clinical cure or prevention of infection.6 At the same time, they aim to limit the unintended consequences of antibiotic use, such as the emergence of resistance, adverse drug events, and costs.6 Antibiotic stewardship programs (APSs) have shown to be effective and financially self-supporting.7-9 Multidisciplinary local stewardship teams are now established across the world, with the task to design programs in their own hospitals.

A requirement for an effective stewardship program is the ability to measure the appropriateness of antibiotic use in individual patients. Quality indicators (QIs) are measurable elements of practice performance for which there is evidence or consensus that they can be used to assess the quality of antibiotic care provided.10 A well-known classification to categorize QIs is: structure-, process- and outcome indicators.11

For an optimal and reliable use of the developed QIs, their clinimetric properties have first to be tested in clinical practice. Registration of data is different in every country and varies over time, which has an effect on validity and reliability of data collection. It is mandatory to locally test the clinimetric properties of the QIs, in order to discriminate between indicators that are feasible, valid and reliable in a specific setting and those that are not. There are several criteria to consider when assessing the QIs, including measurability,12-14 applicability (the indicator should be applicable to a substantial proportion of the reviewed patient records),14,15 inter-observer reliability,12-14,16-18 room for improvement,10,12-14 and case mix stability.12-14,17

During the past decade many quality indicators for appropriate antibiotic use have been developed.19 Providing information on the development and validation processes of available QIs can support healthcare professionals to select QIs that are considered reliable in their healthcare setting. In this systematic review we aimed to make an inventory, including the development methodology and validation procedures, of currently available quality indicators for appropriate antibiotic use in hospitalized adult patients.

Materials and Methods

To create an overview of the existing QIs for appropriate antibiotic use in hospitalized adult patients, two authors (MCK and JMP) performed a systematic literature search in the Pubmed interface. The search strategy is provided in Figure 1. Antibiotics were defined as antibacterial agents. Quality indicators were defined as quality measures, metrics or criteria. Limitations of the search included humans and English language. We screened title and abstract in order to identify studies reporting QIs for antibiotic use. Articles were excluded if they did not concern antibiotic use, did not concern quality measures, did not apply to adults, or concerned the outpatient setting. Duplicate studies were removed. We reviewed potentially relevant articles in full-text format. Articles were excluded if no full text was available or if the minority (<33%) of developed QIs in an article were related to antibiotic use, i.e. QIs as part of a general quality of care set. Finally, we selected the articles describing the consensus or evidence-based development of QIs for appropriate antibiotic use in hospitalized adult patients. The Drive-AB report, a recently performed but not yet published modified Delphi study to identify QIs for antibiotic use, was added to the final list with selected articles.

From the included articles we abstracted i) the indicators developed, ii) type of infection the QIs applied to iii) the study design used for the development of the QIs iv) relation of the QIs to outcome measures v) whether the QIs were validated and vi) the characteristics of the validation cohort. See Table 1 for an overview.19-30

Results

The systematic literature search resulted in 606 potentially relevant articles. After screening of titles and abstracts, 58 potentially relevant articles were selected for full-text screening. After the full text review another 47 articles were excluded, based on the previously mentioned criteria. Two additional articles were selected from literature references and added to the final list, together with the Drive AB report. Finally, fourteen articles describing the development of QIs regarding appropriate antibiotic use in hospitalized adult patients were included (Figure 2). Details of these studies are given in Table 1.19-30

The fourteen included articles described 200 QIs: 17 structure and 183 process indicators. Most QIs were specifically developed for lower respiratory tract infection, urinary tract infection or sepsis. See Appendix 1 for the complete list of indicators. The most frequently mentioned indicators concerned empirical antibiotic therapy according to the guideline (71% of studies), followed by switch from IV to oral therapy (64% of studies), followed by drawing at least two sets of blood cultures and change to pathogen-directed therapy based on culture results (57% of studies) (Table 2).

A (RAND)-modified Delphi consensus procedure was used in the majority of studies (57%). All Delphi studies were performed with a multidisciplinary team of experts, working in different hospitals. Two studies, van den Bosch et al.14,29 and Drive AB,30 performed the consensus procedure with an international team of experts. The panel size varied from 11 to 51 experts.

Six studies took outcome measures, like mortality, morbidity, length of hospitalization, or cost-effectiveness, into consideration during the procedure. For two sets of indicators the relation between adherence to the QIs, and length of hospital stay was investigated in a subsequent observational multicenter trial.14,31

Five out of fourteen studies (36%) tested the clinimetric properties of the QIs. 41 of 63 tested QIs (65%) were considered valid for use in the clinical setting. The most common reasons why QIs were not considered valid were low feasibility of data abstraction from the patient files and lack of room for improvement.

Discussion and Conclusions

In this systematic review we provided an overview, including the development methodology and validation procedures, of all reported quality indicators for appropriate antibiotic use in hospitalized adult patients. Fourteen studies described 200 QIs: 17 structure and 183 process indicators. Five studies (36%) tested the clinimetric properties of the QIs. 41 of 63 tested QIs (65%) were considered valid for use in the clinical setting.

We performed a literature search in order to include all available studies describing the development of quality indicators in adults. Furthermore we extracted from the included studies the methods used to develop the quality indicators, the relation of the QIs with outcome measures, and the validation process of the QIs. To our knowledge such a detailed inventory of QIs for inpatient antibiotic use, including the development methodology and validation procedures, has not been done before. Recently, a systematic review on QIs for diagnosis and antibiotic treatment in primary care was reported.32 The majority (72%) of the 130 QIs focused on choice of antibiotics, 22% concerned the decision to prescribe antibiotics, and few (6%) concerned the diagnostic process. Most QIs were either related to respiratory tract infections or not related to any type of infection. Therefore, the QIs developed for this setting clearly differ from those developed for the inpatient setting.

Our study has several limitations. First, for pragmatic reasons the search was performed only in the Pubmed interface. There is a possibility that articles were overlooked for this reason. However we used a wide range of search terms regarding quality measurements and antibiotics and therefore we assume that the terms should identify those studies reporting the development of quality indicators. Second, we included only the articles that developed QIs exclusively related to antibiotic use. If the minority of QIs (<33%) in an article referred to antibiotic use, we did not select the article. Therefore, we might have missed QIs on antibiotic use. Finally, we excluded articles concerning QIs for pediatric care.

The question remains what the implications of these data are. To develop QIs, a systematic or non-systemic method can be used. Systematic methods rely on available scientific evidence complemented when necessary with expert opinion.33 A Delphi consensus procedure is a systematic process where QIs are developed based on scientific evidence combined with expert opinions.10,12-14,26,27 and at this moment the most rigorous way to develop QIs. In our study, the most frequently applied method indeed was a modified consensus Delphi procedure (57%). All Delphi panels consisted of a multidisciplinary team of (international) experts, usually working in different hospitals. Non-systematic methods, i.e. a point prevalence survey, were also used to develop the QIs.23,34

In a minority of studies (43%) the relation of quality indicators with an outcome measure, like mortality, length of stay or costs, was taken into account during the development process, and for only two sets of indicators the relation between adherence to the QIs, and length of hospital stay was investigated in a subsequent observational multicenter trial. Most of the developed QIs applied only to a specific patient group, for example patients with UTI, CAP or sepsis. Van den Bosch et al. is the only study so far to have developed a generic set of QIs of which the relation to outcome measures was assessed during a Delphi study and in an additional trial.14 As was stated before, a requirement for an effective stewardship program is the ability to measure the appropriateness of antibiotic use using QIs. For an optimal use of the developed QIs it is mandatory to test their clinimetric properties in daily practice, in order to discriminate between indicators that are feasible, valid and reliable in a specific setting and those that are not. In this study we showed that only 65% of the tested QIs were considered valid, implicating that one third of all developed QIs is possibly not eligible for use in clinical practice. Therefore, we recommend to locally test the applicability of these QIs in a pilot or controlled trial before implementing them. Only few study groups have tested the clinimetric properties of their QIs in daily clinical practice.

In our opinion, the set of QIs developed by the Drive AB group,30 is the most comprehensive set of QIs, as it is developed in a high-quality consensus procedure and is based on the most recent literature covering antibiotic use in the inpatient and outpatient setting. However, validation of these QIs is necessary in order to implement them in daily practice. So far, only van den Bosch et al.14,29 managed to develop a high quality, generic and valid set of QIs, and we recommend in comparable settings to apply this set of QIs in stewardship programs.

References

1. 

A Heddini, R. Nugent The path of least resistance. 2014. Available from: http://www.reactgroup.org/uploads/publications/react-publications/the-path-of-least-resistance.pdf.

2. 

Centers for Disease Control and Prevention. Report 2011 (revision): A public health action plan to combat antimicrobial resistance. Centers for Disease Control and Prevention 2011. Available from: http://www.cdc.gov/drugresistance/pdf/public-health-action-plan-combat-antimicrobial-resistance.pdf.

3. 

WHO. Antimicrobial resistance: global report on surveillance 2014. Available from: http://www.who.int/drugresistance/documents/surveillancereport/en

4. 

NethMap. SWAB. Consumption of antimicrobial agents and antimicrobial resistance among medically important bacteria in the Netherlands. 2015. Available from: http://www.swab.nl.

5. 

B Spellberg, M Blaser, RJ Guidos. Combating antimicrobial resistance: policy recommendations to save lives. Clin Infect Dis 2011;52:397-428.

6. 

TH Dellit, RC Owens, JE McGowan. Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America guidelines for developing an institutional program to enhance antimicrobial stewardship. Clin Infect Dis 2007;44:159-77.

7. 

P Carling, T Fung, A Killion. Favorable impact of a multidisciplinary antibiotic management program conducted during 7 years. Infect Control Hosp Epidemiol 2003;24:699-706.

8. 

M Lutters, S Harbarth, JP Janssens. Effect of a comprehensive, multidisciplinary, educational program on the use of antibiotics in a geriatric university hospital. J Am Geriatr Soc 2004;52:112-6.

9. 

S Ruttimann, B Keck, C Hartmeier. Long-term antibiotic cost savings from a comprehensive intervention program in a medical department of a university-affiliated teaching hospital. Clin Infect Dis 2004;38:348-56.

10. 

SM Campbell, J Braspenning, A Hutchinson. Research methods used in developing and applying quality indicators in primary care. BMJ 2003;326:816-9.

11. 

A Donabedian. The quality of care. How can it be assessed? JAMA 1988;260:1743-8.

12. 

HS Hermanides, ME Hulscher, JA Schouten. Development of quality indicators for the antibiotic treatment of complicated urinary tract infections: a first step to measure and improve care. Clin Infect Dis 2008;46:703-11.

13. 

JA Schouten, ME Hulscher, H Wollersheim. Quality of antibiotic use for lower respiratory tract infections at hospitals: (how) can we measure it? Clin Infect Dis 2005;41:450-60.

14. 

CM van den Bosch, SE Geerlings, S Natsch. Quality indicators to measure appropriate antibiotic use in hospitalized adults. Clin Infect Dis 2015;60:281-91.

15. 

R Grol, R Baker, F. Moss Quality improvement research: understanding the science of change in health care. Qual Saf Health Care 2002;11:110-1.

16. 

JR Landis, GG Koch. An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics 1977;33:363-74.

17. 

HR Rubin, P Pronovost, GB Diette. From a process of care to a measure: the development and testing of a quality indicator. Int J Qual Health Care 2001; 13:489-96.

18. 

C Pulcini, S Defres, I Aggarwal. Design of a day 3 bundle to improve the reassessment of inpatient empirical antibiotic prescriptions. J Antimicrob Chemother 2008;61:1384-8.

19. 

T Kotter, E Blozik, M. Scherer Methods for the guideline-based development of quality indicators—a systematic review. Implement Sci 2012;7:21.

20. 

D Nathwani, F Williams, J Winter. Use of indicators to evaluate the quality of community-acquired pneumonia management. Clin Infect Dis 2002;34:318-23.

21. 

SM Berenholtz, PJ Pronovost, K Ngo. Developing quality measures for sepsis care in the ICU. Jt Comm J Qual Patient Saf 2007;33:559-68.

22. 

DE Fry. Surgical site infections and the surgical care improvement project (SCIP): evolution of national quality measures. Surg Infect 2008;9:579-84.

23. 

P Zarb, B Amadeo, A Muller. Identification of targets for quality improvement in antimicrobial prescribing: the web-based ESAC Point Prevalence Survey 2009. J Antimicrob Chemother 2011;66:443-9.

24. 

P Zarb, F Ansari, A Muller. Drug utilization 75% (DU75%) in 17 European hospitals (2000-2005): results from the ESAC-2 Hospital Care Sub Project. Curr Clin Pharmacol 2011;6:62-70.

25. 

A Coll, M Kinnear, A. Kinnear Design of antimicrobial stewardship care bundles on the high dependency unit. Int J Clin Pharm 2012;34:845-54.

26. 

J Thern, K de With, R Strauss. Selection of hospital antimicrobial prescribing quality indicators: a consensus among German antibiotic stewardship (ABS) networkers. Infection 2014;42:351-62.

27. 

CM van den Bosch, ME Hulscher, S Natsch. Development of quality indicators for antimicrobial treatment in adults with sepsis. BMC Infect Dis 2014;14:345.

28. 

H Farida, A Rondags, MH Gasem. Development of quality indicators to evaluate antibiotic treatment of patients with community-acquired pneumonia in Indonesia. Trop Med Int Health 2015;20:501-9.

29. 

CM van den Bosch, ME Hulscher, S Natsch. Applicability of generic quality indicators for appropriate antibiotic use in daily hospital practice: a cross-sectional point-prevalence multicenter study. Clin Microbiol Infect Dis 2016;22:888.e9.

30. 

DRIVE-AB. WP1A quality indicators and quantity metrics of antibiotic use. 2016. Available from: http://driveab.eu/wp-content/uploads/2014/09/WP1A_Final-QMs-QIs_final.pdf.

31. 

V Spoorenberg, ME Hulscher, RP Akkermans. Appropriate antibiotic use for patients with urinary tract infections reduces length of hospital stay. Clin Infect Dis 2014;58:164-9.

32. 

LT Saust, RN Monrad, MP Hansen. Quality assessment of diagnosis and antibiotic treatment of infectious diseases in primary care: a systematic review of quality indicators. Scand J Prim Health Care 2016:1-9.

33. 

R Boulkedid, H Abdoul, M Loustau. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PloS One 2011;6:20476.

34. 

F Ansari, M Erntell, H Goossens. The European surveillance of antimicrobial consumption (ESAC) point-prevalence survey of antibacterial use in 20 European hospitals in 2006. Clin Infect Dis 2009;49:1496-504.

Figure 1.

Search strategy in Pubmed. Limits: humans, English.

idr-2017-1-6821-g001.jpg
Figure 2.

Procedure of article selection

idr-2017-1-6821-g002.jpg
Table 1.

Overview of studies reporting quality indicators for appropriate antibiotic use.

Author Year Study setting No. QIs Target infection Study design Proven relation with outcome? Validated? Validation cohort
Nathwani et al.20 2002 EU 6 Pneumonia Literature review Yes, as evidenced by a literature and guideline review No -
Schouten et a1.13 2005 EU 19 LRT1 Four step RAND-modified Delphi procedure: national, multidisciplinary panel of 11 experts from multiple hospitals Yes, assessed in Delphi study Yes. 15 QIs validated: 9 out of 12 QIs validated for CAP and 6 out of 7 QIs validated for AECB 899 hospitalized patients with LRT1 in the Netherlands
Berenholtz et al.21 2007 US 10 Sepsis Two step modified Delphi procedure: national, multidisciplinary panel of experts from multiple hospitals Yes, assessed in Delphi study No
Hermanides et al.12 2008 EU 13 UT1 Three step RAND-modified Delphi procedure: national, multidisciplinary panel of 13 experts from multiple hospitals Yes, assessed in Delphi study and additional trial Yes. 9 QIs validated for internal medicine dept. and 4 QIs validated for urology dept. 341 patients with complicated UTIs in the Netherlands
Pulcini et al.18 2008 EU 5 NS Literature review No Partially. 4 QIs showed high or moderate inter-rater reliability Forty medical notes of patients admitted to an ID ward in the UK
Fry et al.22 2008 US 3 SSI Literature review No No -
Zarb et al.23 2011 EU 9 NS Based on results of a PPS No No -
Zarb et al.24 2011 EU 2 NS Based on a retrospective audit No No -
Coll et al.25 2012 EU 30 NS Based on agreement of a multidisciplinary team in one hospital, with reference to the evidence base, national strategy and local policy No No
Thern et al.26 2014 EU 21 CAP,HAP, UTI,BSI, all infections Three step RAND/UCLA- modified Delphi procedure: national, multidisciplinary panel of experts from multiple hospitals No No
van den Bosch et al.27 2014 EU 5 Sepsis Five step RAND-modified Delphi procedure: national, multidisciplinary panel of 14 experts from multiple hospitals No No
Farida et al.28 2015 Asia 15 CAP Two step Delphi procedure: national, multidisciplinary panel of 18 experts from multiple hospitals Yes, assessed in Delphi study Yes. 6 QIs validated 128 patients hospitalized with CAP in Indonesia
van den Bosch et al.14,29 2015 EU 11 NS Four step RAND-modified Delphi procedure: international, multidisciplinary panel of 17 experts Yes, assessed in Delphi study and additional trial Yes. 7 QIs validated in the Netherlands 1890 patients from 22 hospitals
DRIVE-AB30 2016 EU 51 NS Four step RAND-modified Delphi procedure: international, multidisciplinary panel of 51 experts No No -

[i] ABS, antibiotic stewardship; LRTI, lower respiratory tract infection; UTI, urinary tract infection; CAP, community- acquired pneumonia; HAP, hospital- acquired pneumonia; BSI, blood stream infection; SSI, surgical site infection; PPS, point prevalence survey; HDU, High Dependency Unit; EU, Europe; US, United States of America, NS, not specified.

Table 2.

Description of the top 10 retrieved quality indicators.

Developed indicators Number of articles mentioning the indicator / total number of articles Percentage of articles mentioning the indicator
Prescribe empirical antibiotic therapy according to (local or national) guidelines 10/14 71
Switch from intravenous to oral therapy 9/14 64
Perform at least two sets of blood cultures 8/14 57
Change to pathogen-directed therapy when culture results become available 8/14 57
Timely initiation of antibiotic therapy 7/14 50
Adapt dose and dosing interval of antibiotics to renal function 7/14 50
Documentation of antibiotic plan in medical record 7/14 50
Perform a site culture 6/14 43
Discontinue antibiotic therapy if infection not confirmed 6/14 43
Duration of antibiotic therapy 6/14 43
Abstract views:
631

Views:
PDF
379
APPENDIX
157
HTML
341

Article Metrics

Metrics Loading ...

Metrics powered by PLOS ALM


Copyright (c) 2017 Marlot C. Kallen, Jan M. Prins

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
 
© PAGEPress 2008-2017     -     PAGEPress is a registered trademark property of PAGEPress srl, Italy.     -     VAT: IT02125780185