Competing interest statement
Conflict of interest: the authors declare no potential conflict of interest.
Today antibiotics are indispensable in practically all health care systems.1 However, the extensive use of antibiotics is also the main driving force in the emergence of resistant microorganisms.2 Worldwide, antibiotic consumption and antibiotic resistance are still on the rise, which, together with the decline in the discovery of new antibiotics, creates one of the greatest current threats to human health.2-5
To curb the rise of antibiotic resistance of medically important bacteria, better use of current agents is warranted and a decrease of inappropriate antibiotic use is imperative.3 Antibiotic stewardship programs are developed to optimize the appropriateness of antibiotic use, in order to maximize the chance of clinical cure or prevention of infection.6 At the same time, they aim to limit the unintended consequences of antibiotic use, such as the emergence of resistance, adverse drug events, and costs.6 Antibiotic stewardship programs (APSs) have shown to be effective and financially self-supporting.7-9 Multidisciplinary local stewardship teams are now established across the world, with the task to design programs in their own hospitals.
A requirement for an effective stewardship program is the ability to measure the appropriateness of antibiotic use in individual patients. Quality indicators (QIs) are measurable elements of practice performance for which there is evidence or consensus that they can be used to assess the quality of antibiotic care provided.10 A well-known classification to categorize QIs is: structure-, process- and outcome indicators.11
For an optimal and reliable use of the developed QIs, their clinimetric properties have first to be tested in clinical practice. Registration of data is different in every country and varies over time, which has an effect on validity and reliability of data collection. It is mandatory to locally test the clinimetric properties of the QIs, in order to discriminate between indicators that are feasible, valid and reliable in a specific setting and those that are not. There are several criteria to consider when assessing the QIs, including measurability,12-14 applicability (the indicator should be applicable to a substantial proportion of the reviewed patient records),14,15 inter-observer reliability,12-14,16-18 room for improvement,10,12-14 and case mix stability.12-14,17
During the past decade many quality indicators for appropriate antibiotic use have been developed.19 Providing information on the development and validation processes of available QIs can support healthcare professionals to select QIs that are considered reliable in their healthcare setting. In this systematic review we aimed to make an inventory, including the development methodology and validation procedures, of currently available quality indicators for appropriate antibiotic use in hospitalized adult patients.
Materials and Methods
To create an overview of the existing QIs for appropriate antibiotic use in hospitalized adult patients, two authors (MCK and JMP) performed a systematic literature search in the Pubmed interface. The search strategy is provided in Figure 1. Antibiotics were defined as antibacterial agents. Quality indicators were defined as quality measures, metrics or criteria. Limitations of the search included humans and English language. We screened title and abstract in order to identify studies reporting QIs for antibiotic use. Articles were excluded if they did not concern antibiotic use, did not concern quality measures, did not apply to adults, or concerned the outpatient setting. Duplicate studies were removed. We reviewed potentially relevant articles in full-text format. Articles were excluded if no full text was available or if the minority (<33%) of developed QIs in an article were related to antibiotic use, i.e. QIs as part of a general quality of care set. Finally, we selected the articles describing the consensus or evidence-based development of QIs for appropriate antibiotic use in hospitalized adult patients. The Drive-AB report, a recently performed but not yet published modified Delphi study to identify QIs for antibiotic use, was added to the final list with selected articles.
From the included articles we abstracted i) the indicators developed, ii) type of infection the QIs applied to iii) the study design used for the development of the QIs iv) relation of the QIs to outcome measures v) whether the QIs were validated and vi) the characteristics of the validation cohort. See Table 1 for an overview.19-30
The systematic literature search resulted in 606 potentially relevant articles. After screening of titles and abstracts, 58 potentially relevant articles were selected for full-text screening. After the full text review another 47 articles were excluded, based on the previously mentioned criteria. Two additional articles were selected from literature references and added to the final list, together with the Drive AB report. Finally, fourteen articles describing the development of QIs regarding appropriate antibiotic use in hospitalized adult patients were included (Figure 2). Details of these studies are given in Table 1.19-30
The fourteen included articles described 200 QIs: 17 structure and 183 process indicators. Most QIs were specifically developed for lower respiratory tract infection, urinary tract infection or sepsis. See Appendix 1 for the complete list of indicators. The most frequently mentioned indicators concerned empirical antibiotic therapy according to the guideline (71% of studies), followed by switch from IV to oral therapy (64% of studies), followed by drawing at least two sets of blood cultures and change to pathogen-directed therapy based on culture results (57% of studies) (Table 2).
A (RAND)-modified Delphi consensus procedure was used in the majority of studies (57%). All Delphi studies were performed with a multidisciplinary team of experts, working in different hospitals. Two studies, van den Bosch et al.14,29 and Drive AB,30 performed the consensus procedure with an international team of experts. The panel size varied from 11 to 51 experts.
Six studies took outcome measures, like mortality, morbidity, length of hospitalization, or cost-effectiveness, into consideration during the procedure. For two sets of indicators the relation between adherence to the QIs, and length of hospital stay was investigated in a subsequent observational multicenter trial.14,31
Five out of fourteen studies (36%) tested the clinimetric properties of the QIs. 41 of 63 tested QIs (65%) were considered valid for use in the clinical setting. The most common reasons why QIs were not considered valid were low feasibility of data abstraction from the patient files and lack of room for improvement.
Discussion and Conclusions
In this systematic review we provided an overview, including the development methodology and validation procedures, of all reported quality indicators for appropriate antibiotic use in hospitalized adult patients. Fourteen studies described 200 QIs: 17 structure and 183 process indicators. Five studies (36%) tested the clinimetric properties of the QIs. 41 of 63 tested QIs (65%) were considered valid for use in the clinical setting.
We performed a literature search in order to include all available studies describing the development of quality indicators in adults. Furthermore we extracted from the included studies the methods used to develop the quality indicators, the relation of the QIs with outcome measures, and the validation process of the QIs. To our knowledge such a detailed inventory of QIs for inpatient antibiotic use, including the development methodology and validation procedures, has not been done before. Recently, a systematic review on QIs for diagnosis and antibiotic treatment in primary care was reported.32 The majority (72%) of the 130 QIs focused on choice of antibiotics, 22% concerned the decision to prescribe antibiotics, and few (6%) concerned the diagnostic process. Most QIs were either related to respiratory tract infections or not related to any type of infection. Therefore, the QIs developed for this setting clearly differ from those developed for the inpatient setting.
Our study has several limitations. First, for pragmatic reasons the search was performed only in the Pubmed interface. There is a possibility that articles were overlooked for this reason. However we used a wide range of search terms regarding quality measurements and antibiotics and therefore we assume that the terms should identify those studies reporting the development of quality indicators. Second, we included only the articles that developed QIs exclusively related to antibiotic use. If the minority of QIs (<33%) in an article referred to antibiotic use, we did not select the article. Therefore, we might have missed QIs on antibiotic use. Finally, we excluded articles concerning QIs for pediatric care.
The question remains what the implications of these data are. To develop QIs, a systematic or non-systemic method can be used. Systematic methods rely on available scientific evidence complemented when necessary with expert opinion.33 A Delphi consensus procedure is a systematic process where QIs are developed based on scientific evidence combined with expert opinions.10,12-14,26,27 and at this moment the most rigorous way to develop QIs. In our study, the most frequently applied method indeed was a modified consensus Delphi procedure (57%). All Delphi panels consisted of a multidisciplinary team of (international) experts, usually working in different hospitals. Non-systematic methods, i.e. a point prevalence survey, were also used to develop the QIs.23,34
In a minority of studies (43%) the relation of quality indicators with an outcome measure, like mortality, length of stay or costs, was taken into account during the development process, and for only two sets of indicators the relation between adherence to the QIs, and length of hospital stay was investigated in a subsequent observational multicenter trial. Most of the developed QIs applied only to a specific patient group, for example patients with UTI, CAP or sepsis. Van den Bosch et al. is the only study so far to have developed a generic set of QIs of which the relation to outcome measures was assessed during a Delphi study and in an additional trial.14 As was stated before, a requirement for an effective stewardship program is the ability to measure the appropriateness of antibiotic use using QIs. For an optimal use of the developed QIs it is mandatory to test their clinimetric properties in daily practice, in order to discriminate between indicators that are feasible, valid and reliable in a specific setting and those that are not. In this study we showed that only 65% of the tested QIs were considered valid, implicating that one third of all developed QIs is possibly not eligible for use in clinical practice. Therefore, we recommend to locally test the applicability of these QIs in a pilot or controlled trial before implementing them. Only few study groups have tested the clinimetric properties of their QIs in daily clinical practice.
In our opinion, the set of QIs developed by the Drive AB group,30 is the most comprehensive set of QIs, as it is developed in a high-quality consensus procedure and is based on the most recent literature covering antibiotic use in the inpatient and outpatient setting. However, validation of these QIs is necessary in order to implement them in daily practice. So far, only van den Bosch et al.14,29 managed to develop a high quality, generic and valid set of QIs, and we recommend in comparable settings to apply this set of QIs in stewardship programs.