politics
E568 CMAJ, October 21, 2014, 186(15) © 2014 Canadian Medical Association or its licensors F requent users of health care services rep - resent a relatively small group of patients who account for a disproportionately large amount of health care utilization, including emergency department visits, 1,2 hospital admis - sions and clinic visits. These patients are often of low socioeconomic status, 3,4 have multiple medi - cal, psychiatric and social disorders 5,6 and have a high mortality. 7 Frequent use of the health care system contributes to longer wait times and affects the quality of care. 4,8 Disproportionate use of health care services by a segment of the population has been identified as a challenge in many countries, including Can - ada. 9–12 To encourage less resource-intensive care for frequent users, many efforts have been imple - mented. Some of these interventions, for example, have been designed specifically to transition health care utilization away from the hospital to other set - tings, such as community-based clinics. 13 Much of the literature has focused on frequent users of emergency departments, with less focus on their use of the health care system in general.
One systematic review identified a number of studies that assessed the effect of various interven - tions, including care coordination. 14 The authors concluded that case management and multidisci - plinary teams were likely effective interventions to reduce emergency department visits. Emergency department visits typically represent only a fraction of the cost burden on the health care system. There is a need to understand the impact of interventions aimed at reducing overall health care utilization, including hospital admissions. We con - ducted a systematic review and meta-analysis of the effectiveness of quality improvement strategies Effectiveness of quality improvement strategies for coordination of care to reduce use of health care services: a systematic review and meta-analysis Andrea C. Tricco PhD, Jesmin Antony MSc, Noah M. Ivers MD PhD, Huda M. A\ shoor BSc, Paul A. Khan PhD, Erik Blondal BSc, Marco Ghassemi MSc, Heather MacDonald MLIS, Maggie H. Chen PhD, Lia\ nne Kark Ezer MSc, Sharon E. Straus MD MSc Competing interests: None declared.
Disclaimer: Sharon Straus is an associate editor with CMAJ and was not involved in the editorial decision-making process for this article.
This article has been peer reviewed.
Correspondence to: Sharon Straus, sharon.straus @utoronto.ca CMAJ 2014. DOI:10.1503 /cmaj.140289 Background: Frequent users of health care ser - vices are a relatively small group of patients who account for a disproportionately large amount of health care utilization. We conducted a meta- analysis of the effectiveness of interventions to improve the coordination of care to reduce health care utilization in this patient group.
Methods: We searched MEDLINE, Embase and the Cochrane Library from inception until May 2014 for randomized clinical trials (RCTs) assess - ing quality improvement strategies for the coordination of care of frequent users of the health care system. Articles were screened, and data abstracted and appraised for quality by 2 reviewers, independently. Random effects meta-analyses were conducted.
Results: We identified 36 RCTs and 14 compan - ion reports (total 7494 patients). Significantly fewer patients in the intervention group than in the control group were admitted to hospital (relative risk [RR] 0.81, 95% confidence inter - val [CI] 0.72–0.91). In subgroup analyses, a sim - ilar effect was observed among patients with chronic medical conditions other than mental illness, but not among patients with mental ill - ness. In addition, significantly fewer patients 65 years and older in the intervention group than in the control group visited emergency departments (RR 0.69, 95% CI 0.54–0.89).
Interpretation: We found that quality improve - ment strategies for coordination of care reduced hospital admissions among patients with chronic conditions other than mental ill - ness and reduced emergency department visits among older patients. Our results may help cli - nicians and policy-makers reduce utilization through the use of strategies that target the system (team changes, case management) and the patient (promotion of self-management). Abstract See related commentary, www.cmaj.ca/lookup/doi/10.1503/cmaj.141050 fifi fi CMAJ, October 21, 2014, 186(15) E569 for care coordination for patients who are frequent users of the health care system.
Methods We developed our protocol according to the PRISMA-P (Preferred Reporting Items for Sys - tematic review and Meta-analysis Protocols) statement (available from the authors).
Literature search The search strategies were developed by an experi - enced librarian and were reviewed by a second librarian using the Peer Review of Electronic Search Strategies checklist. 15 A comprehensive search of MEDLINE, Embase and the Cochrane Library was conducted from inception until May 5, 2014, and was limited to adults and humans.
The MEDLINE search strategy, outlined in Appendix 1 (available at www.cmaj.ca/lookup /suppl /doi:10.1503 /cmaj.140289/-/DC1), was modified for the Embase and Cochrane Library searches with the use of appropriate medical sub - ject headings (available upon request). We also searched trial registries and conference abstracts, scanned the reference lists of included studies and relevant reviews, contacted authors to request other potentially relevant studies, searched the 10 most related citations in PubMed for each included study and searched studies that referenced the included studies in Web of Science (i.e., forward citation searching).
Study selection Before screening began, a calibration exercise was conducted to ensure high reliability in cor - rectly selecting articles for inclusion. This exer - cise entailed screening a random sample of 75 citations (titles and abstracts) using Synthesi.SR (a proprietary online systematic review tool developed by the Joint Program in Knowledge Translation at St. Michael’s Hospital, Toronto). The percentage agreement among these review - ers was quantified. After high agreement was achieved, each citation was screened by 2 authors using the predefined relevance criteria form. Discrepancies were resolved by discussion or the involvement of a third reviewer. The same process was followed for full-text review of potentially relevant articles identified through citation screening. When eligibility of a particu - lar study was unclear, the study’s authors were contacted for additional information. Eligible studies were randomized clinical tri - als (RCTs) that assessed at least 1 of 5 pre - defined quality improvement strategies targeting adult patients (age ≥ 18 yr) who were frequent users of the health care system. The quality Box 1: Description of quality improvement strategies 17 Care coordination Care coordination is the deliberate organization of patient care activit\ ies between 2 or more participants (including the patient) involved in a patient’s care to facilitate the appropriate delivery of health care \ services. Organizing care involves the marshalling of personnel and other resource\ s needed to carry out all required patient care activities; it is often ma\ naged by the exchange of information among participants responsible for different aspects of care. 15 • Case management: The coordination of patient care, including diagnosis, treatment and ongoing patient management (e.g., arranging referrals, follow-up of test results, patient education, patient reminders) by an individual other than the primary care clinician. 18 • Team changes: Changes to the primary health care team and how it functions, including routine patient visits with personnel other than th\ e primary care physician, use of multidisciplinary teams and the expansion\ or revision of team members’ professional roles. 18 • Promotion of self-management: Providing equipment (e.g., home glucometers for patients with diabetes) or access to resources (e.g., electronic systems for transferring glucose measurements for patients with diabetes) and establishing joint goals to empower patients to manage their disease on their own. 18 • Decision support: Operational process of adjustment for a system that generates regular feedback (from registry data) to clinical teams on guideline compliance or organizational support to facilitate other mechanisms for coordinating care. 19 • Clinical information system: A quality improvement strategy encompassing numerous systems performing a wide variety of functions; distinguished from administrative information systems by the requirement for data entry or data retrieval by clinicians at the point \ of care. 20 Additional components • Patient navigator: “Guide people through the health care maze, connecting them with the right doctors and helping them gain access to available therapies.” 21 • Outreach activities: Assessment, education or follow-up conducted outside the clinic or hospital, in or near the patient’s home. Other quality improvement strategies • Patient education: Educating patients about their disease, including prevention and treatment strategies. 18 • Patient reminder systems: Reminding patients about upcoming appointments or important aspects of self-care (e.g., glucose monitorin\ g for patients with diabetes). 18 • Clinician education: Educating clinicians about a particular condition or illness that their patients might face, including strategies for prevent\ ion and treatment (e.g., based on clinical practice guidelines); may be conducted through conferences, workshops, distribution of educational materials and one-on-one educational outreach meetings (or academic detailing). 18 • Clinician reminders: Reminding clinicians to look up patients’ clinical information or to conduct specific tasks. 18 • Audit and feedback: Generating summaries of clinic’s or individual clinician’s performance, which are transmitted back to the clinician.\ 18 • Financial incentives: Providing clinicians with financial incentives for reaching pre-established goals or achievements; may also include incentives for patients or system-wide changes in reimbursement. 18 • Continuous quality improvement: Using specific processes to identify quality problems, developing solutions, and implementing and evaluating changes; may include interventions, such as total quality management or plan–do–study–act. 18 • Facilitated relay of information to clinicians: Transmitting clinical information from patients to clinicians by means other than the existing\ medical record. 18 fifi E570 CMAJ, October 21, 2014, 186(15) improvement interventions of interest, chosen to fill gaps in the “expanded chronic care model” 16 and described in Box 1, 15,17–21 are closely related to care coordination: case management, team changes, promotion of self-management, deci - sion support, and clinical information systems.
We also considered the effects of 2 additional components to an intervention: patient naviga - tors and outreach activities. Quality improvement strategies were com - pared with usual care, no intervention or other quality improvement strategies, as listed in Box 1. When more than one control arm was available in the studies, we chose the usual-care arm for inclusion in the analysis. Included stud - ies had to report at least one of the eligible health utilization outcomes, specifically emer - gency department visits, hospital admissions or clinic visits; the proportion of patients was the primary outcome of interest. Studies written in any language, whether published or unpub - lished, and conducted at any point in time were eligible for inclusion.
Data collection A data abstraction form was drafted and pilot- tested by 8 of us (A.C.T., N.M.I., H.M.A., P.A.K., E.B., M.G., H.M. and L.K.E.) working indepen - dently on a random sample of 5 articles. Data items we recorded were study characteristics (e.g., setting, type of study design), patient characteris - tics (e.g., population examined, mean age), quality improvement strategies examined and utilization outcomes examined. Two reviewers (A.C.T., N.M.I., H.M.A., P.A.K., E.B., M.G., H.M. or L.K.E.) independently read each article and abstracted the relevant data. Differences in abstraction were resolved by team discussion.
Because it is often difficult to classify quality improvement strategies, classification of strategies was performed independently by a systematic review methodologist and a clinician. Conflicts were resolved through discussion. Attempts were made to identify related publications (referred to as companion reports). Study authors were con - tacted via email for clarification of data if neces - sary (e.g., unreported standard deviations for con - tinuous data, mean age of included patients).
Appraisal of risk of bias We used the Cochrane Effective Practice and Organisation of Care Risk-of-Bias Tool to assess risk of bias. 22 Each included article was indepen - dently appraised by 2 reviewers (A.C.T., N.M.I., H.M.A., P.A.K., E.B., M.G., H.M. or L.K.E.).
Conflicts were resolved by discussion or the involvement of a third reviewer (A.C.T. or S.E.S.).
Data synthesis We used a random-effects meta-analysis to com - bine data for outcomes reported in at least 2 RCTs. 16 Mean differences were calculated for studies reporting the average number of visits per patient per month (i.e., continuous outcomes), and relative risks (RRs) were calculated for stud - ies reporting the proportion of patients with visits (i.e., dichotomous outcomes). Funnel plots were created to identify potential publication bias. 23 Before conducting the meta-analysis, we examined 3 types of heterogeneity: clinical (e.g., type of patient population, setting), methodologic (e.g., quality improvement strategy examined) and statistical (e.g., I2 statistic). 24 Our approach for dealing with significant heterogeneity was to conduct appropriate subgroup analyses. We con - ducted post hoc subgroup analyses to determine the influence of the following factors: type of patient (primarily those with mental illness v.
those with chronic medical conditions other than mental illness; and age ≥ 65 yr v. < 65 yr), and type of frequent user based on the RCT eligibility criteria (at risk of being a frequent user = having a history of inpatient care with other predisposing factors, such as multiple comorbidities or psycho - social morbidity; low utilization = “frequent use” defined as 1 to 2 contacts with the health care system in the past year among patients with mul - tiple comorbidities or psychosocial morbidity; moderate utilization = 3 to 4 contacts with the health care system in the past year; and most fre - quent/severe utilization = ≥ 5 contacts with the health care system in the past year). Potentially eligib\ule reports identif\ued through literature \usear\fh n = 11 10\b Ex\fluded n = 10 444 Study design not\u relevant n = 9 92 0 Not adult patien\uts n = 443 Not a quality im\uprovement strategy\u n = 41 Trial proto\fol, \u\fonferen\fe abstra\ft,\u systemati\f review , letter to the edi\utor n = 40 Ex\fluded n= 61 3 Not adult patien\uts n = 322 Study design not\u relevant n = 154 Trial p roto\fol, \fonferen\fe \uabstra \ft, systemati\f review , letter to the ed\uito r n = 62 No relevant/ab stra\ftable out\fomes \u n = 3\b Not a quality im\uprovement strateg y n = 36 Arti\fle not retr\uievable n = 2 In\fluded in meta-an\ualysis n = 50 (36 RCTs, 14 \u\fompanion reports) \u Reports retrieved \uin full n = 663 Figure 1: Selection of articles for the meta-analysis. RCT = randomized \ clinical trial.
fifi CMAJ, October 21, 2014, 186(15) E571 Results Search results and study characteristics Of the 11 107 citations identified through the lit - erature search, 663 full-text articles were reviewed. After exclusion of 613 articles for var - ious reasons (Figure 1), we included 36 RCTs (total 7 494 patients) 25–60 plus an additional 14 companion reports. 61–74 The studies were published between 1987 and 2014 by researchers in North America ( n = 24), Europe ( n = 8), Australia ( n = 2), Israel ( n = 1) and South Africa ( n = 1) (Table 1). One study was a cluster RCT. The duration of follow-up ranged from 1 to 36 months. The definition of a frequent user of health care services varied across the studies. Some studies included patients who were at risk of being fre - quent users ( n = 11 studies), whereas others included patients with low utilization ( n = 8 stud - ies), moderate utilization ( n = 2 studies) or the most frequent/severe utilization ( n = 15 studies). (Additional study and patient characteristics are shown in Appendix 2, available at www.cmaj .
ca /lookup/suppl/doi:10.1503/cmaj.140289 /-/DC1).
Most of the studies included patients with a pri - mary diagnosis of mental illness; 14 studies included patients with a chronic medical condition other than mental illness (Table 1). Twelve stud - ies included patients with severe mental health conditions, such as schizophrenia and substance abuse disorders, and 12 studies included patients who were homeless. The mean age of participants ranged from 28.1 to 81.6 years. The studies included from 25% to 77% women (Appendix 2).
Care coordination strategies The following strategies were used to improve care coordination: case management ( n = 29 stud - ies), team changes ( n = 21), self- management ( n = 19) and clinical information systems ( n = 1) (de - tails about the strategies are included in Appendi - ces 3 and 4, available at www.cmaj.ca/lookup /suppl/doi:10.1503/cmaj.140289/-/DC1). The number of quality improvement strategies exam - ined per study ranged from 1 to 5 (median 2.5).
The intervention included outreach activities in 23 studies and patient navigators in 6 studies. The comparator group received patient education in 1 study or low-intensity case management in 11 studies involving patients with mental illness.
Risk of bias results The risk of bias varied widely across the studies (Table 2; Appendix 5, available at www.cmaj.ca /lookup/suppl/doi:10.1503/cmaj.140289 /-/DC1).
One study had a high risk of bias on 4 criteria, another had a high risk of bias on 3 criteria, 3 stud - ies had a high risk of bias on 2 criteria, 18 had a high risk of bias on 1 criterion, and the rest of the studies did not have a high risk of bias on any of the criteria. The risk of bias was unclear across many of the criteria. Funnel plots did not reveal evidence of publication bias (data not shown).
Effect on emergency department visits After a median duration of 9 months of follow- up, the proportion of patients who visited emer - gency departments did not differ significantly between the intervention and control groups (RR 1.11, 95% confidence interval [CI] 0.65 to 1.90; 6 studies; I2 = 0.85%) (Figure 2; Appendix 6, avail - able at www.cmaj.ca/lookup/suppl/doi:10.1503 /cmaj.140289/-/DC1). The effect was significant only among older patients, with fewer in the intervention group than in the control group visit - ing emergency departments (RR 0.69, 95% CI 0.54 to 0.89; 2 studies; I2 = 0%). In the analysis of studies that reported the mean number of emergency department visits per patient per month, no difference was found between the intervention and control groups after a median duration of 12 months of follow-up (mean difference −0.02, 95% CI −0.06 to 0.03; 7 studies; I2 = 0%) (Appendices 6 and 7, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj .140289/-/DC1). None of the subgroup analyses was statistically significant.
Effect on hospital admissions After a median duration of 12 months of follow- up, significantly fewer patients in the intervention group than in the control group were admitted to hospital (RR 0.81, 95% CI, 0.72 to 0.91; 18 stud - ies; I2 = 58%) (Figure 3; Appendix 6). Specific quality improvement strategies that significantly reduced the number of admissions were case management, team changes, promotion of self- management and patient education. Among patients with chronic conditions other than mental illness, significantly fewer patients in the interven - tion group than in the control group were admitted to hospital. No difference was found between the intervention and control groups among patients with mental illness or severe mental illness (e.g., schizophrenia and severe bipolar disorder). Inter - ventions that had a significant effect were those with an outreach component and those aimed at patients with the most frequent/severe utilization rate and those at risk of frequent use. Statistically significant results were not observed with inter - ventions that used patient navigators or those aimed at patients with low utilization rates. In the analysis of studies that reported the mean number of hospital admissions per patient per month, no difference was found between the inter - fifi E572 CMAJ, October 21, 2014, 186(15) vention and control groups after a median duration of 18 months of follow-up (mean difference 0.00, 95% CI −0.01 to 0.01; 12 studies; I2 = 0%) (Appen - dices 6 and 8, available at www.cmaj.ca /lookup /suppl/doi:10.1503/cmaj.140289/-/DC1). None of the subgroup analyses was statistically significant. Table 1: Study and patient characteristics Study* Country Quality improvement strategy Patients with mental illness Homeless patients Age, yr, mean ± SD Duration of follow up, mo Botha et al., 2014 25 [61] South Africa CM, TC Yes‡ Ye s 32.3 ± 9.9 36 Burns et al., 2014 26 United States CM, SM, PE No No NR 1 Gellis et al., 2014 27 [62] United States FR, CM, SM, PE, CE Ye s No 79.2 ± 7.4 12 Ruchlewska et al., 2014 28 Europe SM Yes‡ Ye s 40.0 ± 11.6 18 Puschner et al., 2011 29 Europe TC, SM Yes‡ Ye s 41.3 ± 11.2 18 Courtney et al., 2009 30 Australia CM, TC, SM, PE Ye s No 78.8 ± 6.9 6 Killaspy et al., 2009 31 [63] Europe CM, TC Ye s No 39.0 ± 11.0 36 Koehler et al., 2009 32 United States TC, CM, PE, SM, CIS No No 78.5 ± 5.5 2 Bellon et al., 2008 33 Europe SM, CQI, CE Yes§ No 48.4 ± NR 15 Lichtenberg et al., 2008 34 Israel CM, TC, SM Ye s No 28.1 ± 11.0 12 Shumway et al., 2008 35 United States CM Yes§ No 43.3 ± 9.5 24 Rivera et al., 2007 36 United States CM Yes‡ Ye s 38.3 ± 12.8 12 Schreuders et al., 2007 37 [64,65] Europe CM, SM Ye s No 52.9 ± 14.8 3 Sledge et al., 2006 38 United States CM, TC, SM No No 51.0 ± 52.8 12 Scott et al., 2004 39 [66] United States TC, PE No No 74.2 ± 7.5 24 Castro et al., 2003 40 United States CM, PE, SM No No 36.4 ± 11.5 12 Laramee et al., 2003 41 United States CM, TC, PE, SM No No 70.7 ± 11.8 2 Harrison-Read et al., 2002 42 Europe CM, TC, SM Yes‡ Ye s 39.2 ± 39.2 24 Kasper et al., 2002 43 United States CM, TC, PE, SM, FI No No 61.9 ± 13.4 6 Katzelnick et al., 2000 44 [67] United States CM, PE, CE Ye s No 45.5 ± NR 12 Burns 1999 45 [68–71] Europe CM, TC, PE Yes‡ Ye s 38.3 ± 11.7 24 Coleman et al., 1999 46† United States CM No No 77.3 ± NR 24 Gagnon et al., 1999 47 Canada TC, SM, CE No No 81.6 ± 6.5 10 Salkever et al., 1999 48 United States CM Yes‡ Ye s 35.7 ± NR 18 Essock et al., 1998 49 [72] United States CM, TC Yes‡ Ye s 41.0 ± NR 18 Stewart et al., 1998 50 Australia TC, CM, PE, SM No No 75.0 ± 10.5 6 Beck et al., 1997 51 United States TC, PE, FR No NA 73.5 ± NR 12 Spillane et al., 1997 52 United States TC No No 38.5 ± 48.2 12 Lafave et al., 1996 53 Canada CM, TC, SM Ye s No 35.8 ± 2.0 12 Quinlivan et al., 1995 54 United States CM Yes‡ Ye s NR 24 Rich et al., 1995 55 [73] United States CM, TC, PE, SM No No 79.2 ± 6.0 3 Rosenheck et al., 1995 56 [74] United States CM, TC Yes‡ Ye s NR 24 Muijen et al., 1994 57 Europe CM Yes‡ Ye s 37.0 ± 11.0 18 Rich et al., 1993 58 United States TC, CM, PE, SM No No 79.0 ± 6.2 3 Bond et al., 1988 59 United States CM Ye s No 34.5 ± NR 6 Franklin et al., 1987 60 United States CM Yes‡ Ye s NR 12 Note: CE = clinician education, CIS = clinical information system, CM = \ case management, CQI = continuous quality improvement, FI = financial incentives, FR = facilitated relay of clinical information, NR = not reported, PE = patie\ nt education, SD = standard deviation, SM = self-management, TC = team changes. *Reference numbers in square brackets indicate companion reports. †Cluster randomized clinical trial. ‡Included patients with severe mental health conditions, such as schi\ zophrenia and substance abuse disorders. §Mental illness was primary diagnosis, but patients may have had othe\ r comorbidities.
fifi CMAJ, October 21, 2014, 186(15) E573 Table 2:
Appraisal of risk of bias of the included studies* Study Random sequence generation Allocation concealment Similar baseline outcome measures Similar baseline characteristics Incomplete outcome data Blinding Contamination Selective outcome reporting Other bias Botha et al.
25 Low Unclear Low Low Low Low High Unclear Low Burns et al.
26 Unclear Unclear Low Low High Low Low Unclear Low Gellis et al.
27 Low Unclear Low Low High Low Low Unclear Low Ruchlewska et al.
28 Low Unclear Low Low Low Low Low Low Low Puschner et al.
29 Unclear Low Low Low High Low Low Unclear Low Courtney et al.
30 Low Unclear Low Low High Low Low Unclear Low Killaspy et al.
31 Low Low Low Low Low Low Low Low Low Koehler et al.
32 Low Low Unclear Low Unclear Low Low Unclear Low Bellon et al.
33 Low Unclear High Low High Low Low Low Unclear Lichtenberg et al.
34 Unclear Unclear Low Low Unclear Low Low Unclear Unclear Shumway et al.
35 Unclear Unclear Low Low High Low Low Unclear Low Rivera et al.
36 Unclear Unclear Unclear Low Low Unclear Low Unclear Low Schreuders et al.
37 Low Low Low Low High Low Low Low Low Sledge et al.
38 Unclear Unclear Low Unclear High Low Low Unclear Low Scott et al.
39 Low Unclear Low Low High Low Low Unclear Low Castro et al.
40 Unclear Low Low Low Low Low Low Unclear Low Laramee et al.
41 Low Unclear Low Low High Low Low Unclear Low Harrison-Read et al.
42 Low Unclear Low Low High Low Low Unclear Low Kasper et al.
43 Unclear Unclear Unclear Low Low Low Low Unclear Low Katzelnick et al.
44 Low Low Low Low High Low Low Unclear Unclear Burns et al.
45 Low Low Low Low Low Low Low Low Unclear Coleman et al.
46 Unclear Unclear Low Low High Low Low Unclear Low Gagnon et al.
47 Low Low Low Low Low Low Unclear Unclear Unclear Salkever et al.
48 Low Low Low Low High High Low Unclear Low Essock et al.
49 Unclear Unclear Unclear High Unclear Low Low Unclear Low Stewart et al.
50 Unclear Unclear Low Low Unclear Low Unclear Unclear Low Beck et al.
51 Low High Unclear High High Low Low Unclear Low Spillane et al.
52 Low Low Low Low High Low Low Unclear Unclear Lafave et al.
53 Unclear Unclear Unclear Low High Low Low Unclear Low Quinlivan et al.
54 Unclear Unclear Unclear Low Low Low Low Unclear Low Rich et al.
55 High High Unclear High High Low Low Unclear Low Rosenheck et al.
56 Low Unclear Low Low Unclear Low Low Unclear Unclear Muijen et al.
57 Unclear Unclear Low Low High Unclear Unclear Unclear Low Rich et al.
58 Low Unclear Unclear Low Low Low Low Unclear Low Bond et al.
59 Unclear Unclear Unclear Unclear High Low Low Unclear Low Franklin et al.
60 Low High Low Low High Low Low Unclear Low Note: High = high risk, low = low risk, unclear = unclear risk.
*The risk of bias was assessed with the Cochrane risk-of-bias tool; only\ the main publications were assessed, not the companion reports.
fifi E574 CMAJ, October 21, 2014, 186(15) Effect on clinic visits After a median duration of 12 months of follow- up, the proportion of patients who made clinic visits did not differ significantly between the intervention and control groups (RR 0.86, 95% CI 0.58 to 1.27; 5 studies; I2 = 91%) (Appendix 6). None of the subgroup analyses was statisti - cally significant. There was also no difference in the mean num - ber of clinic visits per patient per month between the 2 groups after a median of 12 months of follow- up (mean difference −0.08, 95% CI −0.23 to 0.06; 11 studies; I2 = 65%) (Appendices 6 and 9, avail - able at www.cmaj.ca /lookup /suppl/doi:10.1503 /cmaj.140289/-/DC1). None of the subgroup analy - ses was statistically significant.
Effect on length of stay After a median duration of 12 months of follow- up, the mean number of days in hospital per patient per month did not differ significantly between the intervention and control groups (mean difference −0.09, 95% CI −0.26 to 0.09; 19 studies; I2 = 0%) (Appendices 6 and 10, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj .140289/-/DC1). None of the subgroup analyses was statistically significant.
Interpretation We found that quality improvement strategies focused on the coordination of care reduced hos - pital admissions among patients with chronic conditions other than mental illness and reduced emergency department visits among older patients. The strategies were not effective in reducing the use of health care services among patients with mental illness. This lack of effect may have been because 7 of the 11 studies involving patients with mental illness had a care coordination strategy (a form of case manage - ment) as part of their control intervention. Of the interventions examined, team changes, case management and promotion of self- management had significant effects on reducing hospital admissions. Patient education, which is not one of the care-coordination quality improve - ment strategies based on Wagner’s model, 16 also significantly reduced hospital admissions. Patient education and promotion of self- management are likely less resource intensive than case manage - ment interventions are, 17 which suggests that qual - ity improvement strategies targeting patients (as opposed to clinicians) might be an efficient use of resources. Indeed, in other systematic reviews, pa - tient education and promotion of self-management were found to be highly effective in improving diabetes care. 75,76 A previous systematic review assessed the effect of various interventions on frequent users and found that case management and multidisci - plinary teams were likely effective in reducing emergency department visits. 14 The authors did not conduct a meta-analysis or examine utiliza - tion beyond the emergency department. We observed statistically significant reductions in emergency department visits among older patients, but not specifically for interventions involving case management or team changes.
Limitations We identified several limitations in the literature included in our analysis. First, similar to other studies of complex interventions, 77 studies in - cluded in our meta-analysis reported few details about the intensity and “dose” of quality im - Study Treatment n/N Contro l n /N RR (95%CI) Scott et al. 39 51/146 78/149 0.67 (0.51 to 0.87) Courtney et al. 30 14/49 20/58 0.83 (0.47 to 1.46) Burns et al. 26 20/110 42/313 1.35 (0.83 to 2.20) Koehler et al.32 6/20 9/21 0.70 (0.30 to 1.61) Ruchlewska et al .
28 22/70 25/73 0.92 (0.57 to 1.47) Franklin et al. 60 35/213 7/204 4.79 (2.18 to 10.54) Overall Heterogeneity: I 2 = 85% 1.11 (0.65 to 1.90) 0.251.0 4.0 RR (95% CI) Decreased risk Increased risk Figure 2: Effect of quality improvement strategies for coordination of c\ are on emergency department visits. Relative risks less than 1.0 indicate a decreased risk of an emergency department visit. CI = confi\ dence interval, RR = relative risk.
fifi CMAJ, October 21, 2014, 186(15) E575 provement strategies, as well as further details re - garding delivery. The Standards for Quality Im - provement Reporting Excellence (SQUIRE) guidelines have been developed to improve the reporting of quality improvement strategies, 78 which will be of benefit to future meta-analyses such as ours. Second, in some studies, the dura - tion of intervention may have been too short (e.g., 1 mo) to show any significant impact.
Third, the duration of follow-up (as little as 3 mo) was also short in some studies. Fourth, the defini - tion of a frequent user was inconsistent across the studies. Finally, most of the included studies had unclear or inadequate concealment of the alloca - tion sequence and a high risk of bias owing to in - complete outcome data. Our systematic review process also had some limitations. First, although we searched for unpublished studies, none was identified. How - ever, the funnel plots compiled for the meta- analyses of more than 10 RCTs showed no evi - dence of publication bias. Second, this was a challenging area to search, and many of the included studies did not use adequate search terms to allow their identifi - cation. We conducted supplementary searches to surmount this issue (e.g., forward citation searches, manual searches of related articles), but we may have missed relevant studies. Third, our analysis was limited because the quality improvement strategies were complex and difficult to classify consistently. For exam - ple, some of the strategies were interconnected, such as patient education and promotion of self- management, or case management and team changes. However, we conducted a sensitivity analysis of our classification of the strategies, and our results did not change. Fourth, because of the dearth of data, we were unable to perform more sophisticated analyses, such as meta-regression analysis. As such, we did not control for all potential confounding fac - tors or effect modifiers. Also, there was a small number of studies included for some outcomes (e.g., emergency department visits, clinic visits), which may have led to the nonsignificant effect.
As well, the results of the subgroup analyses should be interpreted with caution because of the risk of type 2 statistical error owing to the small number of studies included. Study Treatment n/N Control n/N RR (95%CI) Beck et al. 51 35/160 47/161 0.75 (0.51 to 1.09) Botha et al. 25 13/32 18/24 0.54 (0.34 to 0.87) Burns et al.45 210/353 228/355 0.93 (0.82 to 1.04) Franklin et al.60 62/213 38/204 1.56 (1.10 to 2.23) Lafave et al. 53 13/24 37/41 0.60 (0.41 to 0.88) Puschner et al.29 108/241 103/250 1.09 (0.89 to 1.33) Rich et al . 55 41/142 59/140 0.69 (0.50 to 0.95) Salkever et al. 48 27/91 25/53 0.63 (0.41 to 0.96) Rich et al . 58 21/63 16/35 0.73 (0.44 to 1.20) Kasper et al.43 47/102 55/98 0.82 (0.62 to 1.08) Courtney et al .30 13/49 27/58 0.57 (0.33 to 0.98) Castro et al. 40 20/50 25/46 0.74 (0.48 to 1.13) Burns et al.26 17/110 56/313 0.86 (0.53 to 1.42) Koehler et al.32 6/20 9/21 0.70 (0.30 to 1.61) Ruchlewska et al .28 24/70 33/73 0.76 (0.50 to 1.14) Laramee et al. 41 49/131 46/125 1.02 (0.74 to 1.40) Stewart et al.50 24/49 31/48 0.76 (0.53 to 1.08) Lichtenberg et al . 34 71/122 74/95 0.75 (0.62 to 0.90) Overall Heterogeneity: I 2 = 58% 0.81 (0.72 to 0.91) 0.251.0 4.0 RR (95% CI) Decreased risk Increased risk Figure 3: Effect of quality improvement strategies for coordination of c\ are on hospital admissions. Relative risks less than 1.0 indicate a decreased risk of admission to hospital. CI = confidence interval, RR \ = relative risk.
fifi E576 CMAJ, October 21, 2014, 186(15) Fifth, many of the meta-analyses had substan - tial heterogeneity, which was to be expected given the number of quality improvement strate - gies assessed, the variety of patient populations examined and the inconsistent definitions of usual care used across the studies. The high het - erogeneity may indicate that the results should be interpreted with caution; however, heteroge - neity was substantially lower in most of the sub - group analyses (e.g., by type of quality improve - ment strategy). Sixth, we did not examine patient-centred outcomes, such as patient experience and quality of life, because the target for our research was health system outcomes. Seventh, we were unable to examine contex - tual factors that would have been relevant to our objective, such as socioeconomic status, appro - priateness of care and access to a primary care physician, because they were not measured con - sistently across the studies. Finally, we abstracted some data on costs but were unable to summarize this in a meaningful manner, because this information varied widely by context.
Conclusion We found that quality improvement strategies focused on the coordination of care reduced hos - pital admissions among patients with chronic conditions other than mental illness and reduced emergency department visits among older pa - tients. Novel strategies are required for patients with mental health conditions. Researchers who are developing and implementing interventions targeted to frequent users should consider spe - cific strategies, such as team changes, case man - agement and promotion of self- management, be - cause these approaches appear to be more effective than other quality improvement strate - gies in reducing health care utilization. Further research is needed to determine how to optimize care coordination strategies for specific patient subgroups and settings.
References 1. Mandelberg JH, Kuhn RE, Kohn MA. Epidemiologic analysis of an urban, public emergency department’s frequent users. Acad Emerg Med 2000;7:637-46. 2. Hansagi H, Olsson M, Sjoberg S, et al. Frequent use of the hos - pital emergency department is indicative of high use of other health care services. Ann Emerg Med 2001;37:561-7. 3. Gordon JA, Chudnofsky CR, Hayward RA. Where health and welfare meet: social deprivation among patients in the emer - gency department. J Urban Health 2001;78:104-11. 4. Hunt KA, Weber EJ, Showstack JA, et al. Characteristics of fre - quent users of emergency departments. Ann Emerg Med 2006; 48:1-8. 5. Williams ER, Guthrie E, Mackway-Jones K, et al. Psychiatric status, somatisation, and health care utilization of frequent attenders at the emergency department: a comparison with rou - tine attenders. J Psychosom Res 2001;50:161-7. 6. Sun BC, Burstin HR, Brennan TA. Predictors and outcomes of frequent emergency department users. Acad Emerg Med 2003; 10:320-8. 7. Gunnarsdottir OS, Rafnsson V. Mortality of the users of a hos - pital emergency department. Emerg Med J 2006;23:269-73. 8. Derlet RW, Richards JR. Overcrowding in the nation’s emer - gency departments: complex causes and disturbing effects. Ann Emerg Med 2000;35:63-8. 9. Chambers C, Chiu S, Katic M, et al. High utilizers of emer - gency health services in a population-based cohort of homeless adults. Am J Public Health 2013;103(Suppl2):S302-10. 10. Moe J, Bailey AL, Oland R, et al. Defining, quantifying, and characterizing adult frequent users of a suburban Canadian emergency department. CJEM 2013;15:1-13. 11. Fisher S, Stevens RF. Subgroups of frequent users of an inpa - tient mental health program at a community hospital in Can - ada. Psychiatr Serv 1999;50:244-7. 12. Demers M. Frequent users of ambulatory health care in Quebec: the case of doctor-shoppers. CMAJ 1995;153:37-42. 13. Pines JM, Asplin BR, Kaji AH, et al. Frequent users of emer - gency department services: gaps in knowledge and a proposed research agenda. Acad Emerg Med 2011;18:e64-9. 14. Althaus F, Paroz S, Hugli O, et al. Effectiveness of interven - tions targeting frequent users of emergency departments: a sys - tematic review. Ann Emerg Med 2011;58:41-52 e42. 15. Sampson M, McGowan J, Cogo E, et al. An evidence-based practice guideline for the peer review of electronic search strat - egies. J Clin Epidemiol 2009;62:944-52. 16. Barr VJ, Robinson S, Marin-Link B, et al. The expanded Chronic Care Model: an integration of concepts and strategies from population health promotion and the Chronic Care Model. Hosp Q 2003;7:73-82. 17. Tricco AC, Ivers NM, Grimshaw JM, et al. Effectiveness of quality improvement strategies on the management of diabetes: a system - atic review and meta-analysis. Lancet 2012;379:2252-61. 18. Shojania KG, Ranji SR, McDonald KM, et al. Effects of qual - ity improvement strategies for type 2 diabetes on glycemic control: a meta-regression analysis. JAMA 2006;296:427-40. 19. McDonald KM, Sundaram V, Bravata DM, et al. Closing the quality gap: a critical analysis of quality improvement strategies (Vol 7: Care coordination) . Rockville (MD): Agency for Healthcare Research and Quality; 2007. 20. Shojania KG, Ranji SR, Shaw LK, et al. Closing the quality gap: a critical analysis of quality improvement strategies (Vol 2: Diabetes care) . Rockville (MD): Agency for Healthcare Research and Quality; 2004. 21. Walkinshaw E. Patient navigators becoming the norm in Canada. CMAJ 2011;183:E1109-10. 22. Suggested risk of bias criteria for EPOC reviews . Ottawa: Cochrane Effective Practice and Organisation of Care Group; 2012. Available: http://epoc.cochrane.org/sites/epoc.cochrane .org /files/uploads/Suggested%20risk%20of%20bias%20criteria % 20for%20EPOC%20reviews.pdf (accessed 2014 Aug 21). 23. Egger M, Davey Smith G, Schneider M, et al. Bias in meta- analysis detected by a simple, graphical test. BMJ 1997; 315: 629-34. 24. Higgins JP, Thompson SG. Quantifying heterogeneity in a meta- analysis. Stat Med 2002;21:1539-58. 25. Botha UA, Koen L, Galal U, et al. The rise of assertive commu - nity interventions in South Africa: a randomized control trial assessing the impact of a modified assertive intervention on read - mission rates; a three year follow-up. BMC Psychiatry 2014; 14:56. 26. Burns ME, Galbraith AA, Ross-Degnan D, et al. Feasibility and evaluation of a pilot community health worker intervention to reduce hospital readmissions . Int J Qual Health Care 2014; 26:358-65. 27. Gellis ZD, Kenaley BL, Ten Have T. Integrated telehealth care for chronic illness and depression in geriatric home care patients: the Integrated Telehealth Education and Activation of Mood (I-TEAM) study. J Am Geriatr Soc 2014;62:889-95. 28. Ruchlewska A, Wierdsma AI, Kamperman AM, et al. Effect of crisis plans on admissions and emergency visits: a randomized controlled trial. PLoS ONE 2014;9:e91882. 29. Puschner B, Steffen S, Volker KA, et al. Needs-oriented discharge planning for high utilisers of psychiatric services: multicentre ran - domised controlled trial. Epidemiol Psychiatr Sci 2011; 20:181-92. 30. Courtney M, Edwards H, Chang A, et al. Fewer emergency readmissions and better quality of life for older adults at risk of hospital readmission: a randomized controlled trial to determine the effectiveness of a 24-week exercise and telephone follow-up program. J Am Geriatr Soc 2009;57:395-402. 31. Killaspy H, Kingett S, Bebbington P, et al. Randomised evalua - tion of assertive community treatment: 3-year outcomes. Br J Psychiatry 2009;195:81-2.
fifi CMAJ, October 21, 2014, 186(15) E577 32. Koehler BE, Richter KM, Youngblood L, et al. Reduction of 30-day postdischarge hospital readmission or emergency depart - ment (ED) visit rates in high-risk elderly medical patients through delivery of a targeted care bundle. J Hosp Med 2009; 4: 211-8. 33. Bellón JA, Rodriguez-Bayon A, de Dios Luna J, et al. Success - ful GP intervention with frequent attenders in primary care: ran - domised controlled trial. Br J Gen Pract 2008;58:324-30. 34. Lichtenberg P, Levinson D, Sharshevsky Y, et al. Clinical case management of revolving door patients — a semi-randomized study. Acta Psychiatr Scand 2008;117:449-54. 35. Shumway M, Boccellari A, O’Brien K, et al. Cost- effectiveness of clinical case management for ED frequent users: results of a randomized trial. Am J Emerg Med 2008;26:155-64. 36. Rivera JJ, Sullivan AM, Valenti SS. Adding consumer- providers to intensive case management: Does it improve outcome? Psychiatr Serv 2007;58:802-9. 37. Schreuders B, van Marwijk H, Smit J, et al. Primary care patients with mental health problems: outcome of a randomised clinical trial. Br J Gen Pract 2007;57:886-91. 38. Sledge WH, Brown KE, Levine JM, et al. A randomized trial of primary intensive care to reduce hospital admissions in patients with high utilization of inpatient services. Dis Manage 2006; 9: 328-38. 39. Scott JC, Conner DA, Venohr I, et al. Effectiveness of a group outpatient visit model for chronically ill older health maintenance organization members: a 2-year randomized trial of the coopera - tive health care clinic. J Am Geriatr Soc 2004;52:1463-70. 40. Castro M, Zimmermann NA, Crocker S, et al. Asthma interven - tion program prevents readmissions in high healthcare users. Am J Respir Crit Care Med 2003;168:1095-9. 41. Laramee AS, Levinsky SK, Sargent J, et al. Case management in a heterogeneous congestive heart failure population: a ran - domized controlled trial. Arch Intern Med 2003;163:809-17. 42. Harrison-Read P, Lucas B, Tyrer P, et al. Heavy users of acute psychiatric beds: randomized controlled trial of enhanced com - munity management in an outer London borough. Psychol Med 2002;32:403-16. 43. Kasper EK, Gerstenblith G, Hefter G, et al. A randomized trial of the efficacy of multidisciplinary care in heart failure outpa - tients at high risk of hospital readmission. J Am Coll Cardiol 2002;39:471-80. 44. Katzelnick DJ, Simon GE, Pearson SD, et al. Randomized trial of a depression management program in high utilizers of medi - cal care. Arch Fam Med 2000;9:345-51. 45. Burns T, Creed F, Fahy T, et al. Intensive versus standard case management for severe psychotic illness: a randomised trial. UK 700 Group. Lancet 1999;353:2185-9. 46. Coleman EA, Grothaus LC, Sandhu N, et al. Chronic care clin - ics: a randomized controlled trial of a new model of primary care for frail older adults. J Am Geriatr Soc 1999;47:775-83. 47. Gagnon AJ, Schein C, McVey L, et al. Randomized controlled trial of nurse case management of frail older people. J Am Geriatr Soc 1999;47:1118-24. 48. Salkever D, Domino ME, Burns BJ, et al. Assertive community treatment for people with severe mental illness: the effect on hospital use and costs. Health Serv Res 1999;34:577-601. 49. Essock SM, Frisman LK, Kontos NJ. Cost-effectiveness of assertive community treatment teams. Am J Orthopsychiatry 1998;68:179-90. 50. Stewart S, Pearson S, Horowitz JD. Effects of a home-based inter - vention among patients with congestive heart failure discharged from acute hospital care. Arch Intern Med 1998; 158: 1067-72. 51. Beck A, Scott J, Williams P, et al. A randomized trial of group outpatient visits for chronically ill older HMO members: the Cooperative Health Care Clinic. J Am Geriatr Soc 1997;45:543-9. 52. Spillane LL, Lumb EW, Cobaugh DJ, et al. Frequent users of the emergency department: Can we intervene? Acad Emerg Med 1997; 4:574-80. 53. Lafave HG, de Souza HR, Gerber GJ. Assertive community treat - ment of severe mental illness: a Canadian experience. Psychiatr Serv 1996;47:757-9. 54. Quinlivan R, Hough R, Crowell A, et al. Service utilization and costs of care for severely mentally ill clients in an intensive case management program. Psychiatr Serv 1995;46:365-71. 55. Rich MW, Beckham V, Wittenberg C, et al. A multidisciplinary intervention to prevent the readmission of elderly patients with congestive heart failure. N Engl J Med 1995;333:1190-5. 56. Rosenheck R, Neale M, Leaf P, et al. Multisite experimental cost study of intensive psychiatric community care. Schizophr Bull 1995;21:129-40. 57. Muijen M, Cooney M, Strathdee G, et al. Community psychiat - ric nurse teams: intensive support versus generic care. Br J Psy - chiatry 1994;165:211-7. 58. Rich MW, Vinson JM, Sperry JC, et al. Prevention of readmis - sion in elderly patients with congestive heart failure: results of a prospective, randomized pilot study. J Gen Intern Med 1993; 8: 585-90. 59. Bond GR, Miller LD, Krumwied RD, et al. Assertive case man - agement in three CMHCs: a controlled study. Hosp Community Psychiatry 1988;39:411-8. 60 Franklin JL, Solovitz B, Mason M, et al. An evaluation of case management. Am J Public Health 1987;77:674-8. 61. Botha UA, Koen L, Joska JA, et al. Assessing the efficacy of a modified assertive community-based treatment programme in a developing country. BMC Psychiatry 2010;10:73. 62. Gellis ZD, Kenaley B, McGinty J, et al. Outcomes of a telehealth intervention for homebound older adults with heart or chronic respiratory failure: a randomized controlled trial. Gerontologist 2012;52:541-52. 63. Killaspy H, Bebbington P, Blizard R, et al. The REACT study: randomised evaluation of assertive community treatment in north London. BMJ 2006;332:815-20. 64. Schreuders B, van Oppen P, van Marwijk HW, et al. Frequent attenders in general practice: problem solving treatment pro - vided by nurses. BMC Fam Pract 2005;6:42. 65. Bosmans JE, Schreuders B, van Marwijk HW, et al. Cost- effectiveness of problem-solving treatment in comparison with usual care for primary care patients with mental health prob - lems: a randomized trial. BMC Fam Pract 2012;13:98. 66. Coleman EA, Eilertsen TB, Kramer AM, et al. Reducing emer - gency visits in older adults with chronic illness: a randomized, controlled trial of group visits. Effect Clin Pract 2001;4:49-57. 67. Simon GE, Manning WG, Katzelnick DJ, et al. Cost- effectiveness of systematic depression treatment for high utilizers of general medical care. Arch Gen Psychiatry 2001;58:181-7. 68. Cost-effectiveness of intensive v. standard case management for severe psychotic illness. UK700 case management trial. UK700 Group. Br J Psychiatry 2000;176:537-43. 69. Burns T, Fiander M, Kent A, et al. Effects of case-load size on the process of care of patients with severe psychotic illness. Report from the UK700 trial. Br J Psychiatry 2000;177:427-33. 70. Burns T, White I, Byford S, et al. Exposure to case manage - ment: relationships to patient characteristics and outcome. Report from the UK700 trial. Br J Psychiatry 2002;181:236-41. 71. Hassiotis A, Ukoumunne OC, Byford S, et al. Intellectual func - tioning and outcome of patients with severe psychotic illness randomised to intensive case management. Report from the UK700 trial. Br J Psychiatry 2001;178:166-71. 72. Essock SM, Mueser KT, Drake RE, et al. Comparison of ACT and standard case management for delivering integrated treat - ment for co-occurring disorders. Psychiatr Serv 2006;57:185-96. 73. Rich MW, Gray DB, Beckham V, et al. Effect of a multidisci - plinary intervention on medication compliance in elderly patients with congestive heart failure. Am J Med 1996;101:270-6. 74. Rosenheck RA, Neale MS. Cost-effectiveness of intensive psy - chiatric community care for high users of inpatient services. Arch Gen Psychiatry 1998;55:459-66. 75. McKay MP, Vaca FE, Field C, et al. Public health in the emer - gency department: overcoming barriers to implementation and dissemination. Acad Emerg Med 2009;16:1132-7. 76. Prior M, Guerin M, Grimmer-Somers K. The effectiveness of clinical guideline implementation strategies — a synthesis of systematic review findings. J Eval Clin Pract 2008;14:888-97. 77. Shepperd S, Lewin S, Straus S, et al. Can we systematically review studies that evaluate complex interventions? PLoS Med 2009;6:e1000086. 78. Davidoff F, Batalden PB, Stevens DP, et al. Development of the SQUIRE publication guidelines: evolution of the SQUIRE proj - ect. Jt Comm J Qual Patient Saf 2008;34:681-7. Affiliations: Li Ka Shing Knowledge Institute (Tricco, Ant - ony, Ashoor, Khan, Blondal, Ghassemi, MacDonald, Chen, Ezer, Straus), St. Michael’s Hospital, Toronto, Ont.; Division of Epidemiology (Tricco), Dalla Lana School of Public Health, University of Toronto, Toronto, Ont.; Women’s Col - lege Hospital (Ivers), Toronto, Ont.; Departments of Family and Community Medicine (Ivers) and of Geriatric Medicine (Straus), University of Toronto, Toronto, Ont.
Contributors: Andrea Tricco contributed to the study con - cept and design, helped obtain funding for the study, screened citations and full-text articles, abstracted data, helped analyze the data and interpreted the results. Jesmin Antony screened full-text articles, abstracted data and appraised study quality. Noah Ivers abstracted data, fifi E578 CMAJ, October 21, 2014, 186(15) appraised study quality and interpreted the results. Huda Ashoor abstracted data and appraised study quality. Paul Khan, Erik Blondal and Marco Ghassemi screened citations, abstracted data and appraised study quality. Lianne Ezer screened citations and abstracted data. Heather MacDonald screened citations and full-text articles and abstracted data. Sharon Straus contributed to the study concept and design, helped obtain the funding and interpreted the results. Andrea Tricco and Sharon Straus drafted the manuscript, Lianne Ezer helped write the background section and introduction, and all of the authors critically revised the manuscript and accepted the final version submitted for publication. All of authors had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the analyses.
Funding: This project was funded by the Building Bridges to Integrate Care (BRIDGES) initiative, through the Ontario Min - istry of Health and Long-Term Care. The funding group had no role in the study design, the collection and analysis of data, the decision to publish or the preparation of the manuscript. Andrea Tricco is funded by a Canadian Institutes of Health Research/Drug Safety and Effectiveness Network (CIHR/DSEN) New Investigator Award on Knowledge Syn - thesis Methodology. Noah Ivers holds fellowship awards from the CIHR and from the Department of Family and Community Medicine, University of Toronto. Sharon Straus is funded by a CIHR Tier 1 Research Chair in Knowledge Translation.
Acknowledgements: The authors thank Laure Perrier for conducting, and Becky Skidmore for peer reviewing, the liter - ature search; Mariam Tashkandi, Erin Lillie and Charlene Soobiah for screening some of the citations; Jennifer D’Souza for locating full-text articles; and Wing Hui and Judy Tran for formatting the tables and text in the submitted manuscript.
fifi
