nif:isString
|
-
Systematic reviews of reviews have been particularly advocated to inform policy, clinical, and research deliberations by providing an evidence-based summary of inter-related technologies [9]. Our approach involved drawing on established systematic review methodology (i.e., those developed by The Cochrane Collaboration) to ensure rigour by minimising the risk of bias [10]; we also drew on more novel methods of evidence synthesis (i.e., those developed by the UK National Health Service [NHS] Service Delivery and Organisation Programme) with the aim of producing an overview that we hoped would prove useful to decision makers [11]. We present here a summary of the methods used.
Inherent difficulties associated with systematic reviews of health care organisation and delivery intervention include the considerable effort required at the outset to facilitate their conduct [9]. Accordingly, we began with an in-depth exploration of the fields of health care quality and safety, as well as eHealth functionalities used in health care delivery. This exploration entailed conceptually mapping the fields to understand various processes involved as well as how these relate to each other. For quality and safety considerations, we identified existing taxonomies and frameworks to facilitate this conceptual mapping exercise, which helped to delineate the scope of our work. For the field of eHealth, we drew from existing team members' conceptual and empirical work to aid our construction of a conceptual map for eHealth technologies [12],[13]. This exercise allowed us to categorise interventions with regards to over-arching similarities. We characterised eHealth technologies as having three main overlapping functions: (1) to enable the storage, retrieval, and transmission of data; (2) to support clinical decision making; and (3) to facilitate remote care. Given the strategic focus of the English National Programme for Information Technology (NPfIT) (and other similar large-scale programmes) on electronic record and professional decision support systems [1], the first two functions were prioritised in this initial phase of our work. The current reported work thus concerns the related areas of EHRs, PACS, CPOEs, ePrescribing, and computerised systems for supporting clinical decision making. Remote care and consumer health informatics are the subjects of a subsequent 3-y research enquiry, which is currently in progress.
We drew on established Cochrane-based systematic review principles to search for relevant systematic reviews. An inclusive string of MeSH and free terms (Text S1) was developed to query PubMed/MEDLINE, EMBASE, and the Cochrane Library contents for secondary research reports published from 1997 up to 2007 with no restrictions placed on language. The bibliographies of reports identified as potentially relevant were reviewed as was a catalogue of secondary research amassed through various contributions by team members. Additional searches of key health informatics resources, namely the conference proceedings and publication databases of the American Medical Informatics Association and the Agency of Healthcare Research and Quality, were also undertaken. Finally, the Internet was searched using the Google and Google Scholar search engines. Searches were periodically updated to ensure that the most recent publications were included with the last update occurring at the end of April 2010.
Selection and Critical Appraisal of Systematic Reviews: On the basis of the areas identified for prioritisation, we developed a detailed list of interventions that were to be included/excluded (Text S2). End users of applicable interventions were limited to health care professionals; any findings relating to patient-focused interventions were therefore excluded. Of interest were systematic reviews that focused on the assessment of patient, practitioner, or organisational outcomes. We detailed the following methodological criteria for the identification of systematic reviews: (1) reference to the study as being a systematic review by the authors within the title, abstract, or text; and/or (2) evidence from the description of the methods that systematic review principles had been utilised in searching and appraising the evidence. All systematic reviews having been identified as potentially suitable were assessed for inclusion by two independent reviewers, with arbitration by a third reviewer if necessary. Data from systematic reviews meeting the above criteria, henceforth referred to as “reviews,” were independently critically reviewed by two reviewers, and relevant data were abstracted. Systematic reviews not primarily concerned with assessing impact on patients, professionals, or the organisation, but nonetheless intervention focused, were drawn on to provide additional contextual information. These supplementary systematic reviews (henceforth referred to as “supplementary reviews”) were not subjected to formal critical appraisal. Critical appraisal was undertaken using an adapted version of the Critical Appraisal Skills Programme (CASP) tool for systematic reviews [14]. These modifications were informed by the growing literature regarding both the methodological and reporting issues with primary research in health informatics (Table S1). The details of this process and the tool's associated properties will be the subject of a separate publication in due course.
A standard approach was taken for each of the eHealth technologies of interest. Definitions were first clarified and then the individual use and broader scope for deployment conceptualised. Juxtaposing this with the aforementioned conceptual maps of the fields of eHealth, quality and safety provided a literature-based framework for delineating the principal theorised benefits and risks associated with each intervention. We used this framework to guide synthesis of the empirically demonstrated benefits and risks of implementing eHealth technologies. The body of literature identified was too diverse to allow quantitative synthesis of empirical evidence and we therefore undertook a narrative synthesis. This synthesis involved initially describing the technologies and outcomes studies using the above-described framework for each of the included reviews, which was followed by developing a summary of our assessment of and the key findings from each review (Table S2). We then employed a modified version of the World Health Organization's Health Evidence Network system for appraising public health evidence, which classifies evidence into three main categories, i.e., strong, moderate or weak; this assessment being based on a combination of the overall consistency, quality, and volume of evidence uncovered. These review-derived data were then thematically synthesised in relation to each of the technologies under consideration, drawing on key findings from the additional reviews, as appropriate [8].
|