Description of Evidence Analysis Methods
Judgments about the quality of evidence (see the "Rating Scheme for the Strength of the Evidence" field) require assessing the validity of results for important outcomes in individual studies. Explicit criteria should be used in making these judgments. In the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) Working Group approach, a systematic review of available evidence guides these judgments.
Sequential judgments are made concerning the following factors:
• The quality of evidence across studies for each important outcome
• Which outcomes are critical to a decision
• The overall quality of evidence across these critical outcomes
• The balance between benefits and harms
• The strength of recommendations
Reviewers consider four key elements: study design, study quality, consistency, and directness.
Definitions
Study design refers to the basic study design (broadly, observational studies and randomized trials).
Study quality refers to the detailed study methods and execution. Appropriate criteria are used to assess study quality for each important outcome. For randomized trials, for example, these criteria might include the adequacy of allocation concealment, blinding, and follow up. Reasons for downgrading a quality rating must be explicit (e.g., failure to blind patients and physicians reduced the quality of evidence for an intervention's impact on pain severity, a serious limitation).
Consistency refers to the similarity of effect estimates across studies. If there is important unexplained inconsistency in study results, confidence in the effect estimate for that outcome is reduced.
Directness refers to the extent to which the people, interventions, and outcome measures in the studies are similar to those of interest. For example, the directness of the evidence may be uncertain if the people of interest are older, sicker, or have more comorbidity than those in the studies. To determine whether important uncertainty exists, one can ask whether there is a compelling reason to expect important differences in the effect size. Because many interventions have more or less the same relative effects across most patient groups, reviewers should not use overly stringent criteria in deciding whether evidence is direct.
Criteria
Criteria for decreasing the grade:
• Serious (-1) or very serious (-2) limitation to study quality
• Important inconsistency (-1)
• Some (-1) or major (-2) uncertainty about directness
• Imprecise or sparse data (-1)
• High probability of reporting bias (-1)
Criteria for increasing the grade:
• Strong evidence of association: Significant relative risk greater than 2 (less than 0.5), based on consistent evidence from two or more observational studies, with no plausible confounders (+1)
• Very strong evidence of association: Significant relative risk greater than 5 (less than 0.2), based on direct evidence with no major threats to validity (+2)
• Evidence of a dose-response gradient (+1)
• All plausible confounders would have reduced the effect (+1)
These criteria are cumulative – e.g., if randomized controlled trials (RCTs) have serious limitations and there is uncertainty about the directness of the evidence, the grade of evidence would drop from high to low.
Methods Used to Formulate the Recommendations
Expert Consensus
Description of Methods Used to Formulate the Recommendations
Original guidelines are developed by interdisciplinary workgroups, using a process that combines evidence- and consensus-based approaches. Workgroups include practitioners and others involved in patient care in long term care (LTC) facilities. Beginning with pertinent literature searches for articles and information related to the guideline subject and a draft outline/framework, each group works to develop a concise, usable guideline that is tailored to the LTC setting. Because scientific research in the LTC population is limited, many recommendations are based on findings from research involving community-living older adults. Some recommendations are based on the expert consensus opinion of practitioners and experts in the field of geriatric medicine. The American Medical Directors Association (AMDA) Clinical Practice Guideline Steering Committee directs the guideline development and revision process. Each year the Steering Committee reviews all AMDA clinical practice guidelines that are 3 years old and commissions a thorough literature review to determine whether the content of each guideline remains current. The AMDA Clinical Practice Committee Chair selects the existing guidelines to be revised and new guidelines to be created based on 1) the Steering Committee's recommendations, 2) data collected, and 3) an assessment of the difficulty of development and relevance to the AMDA membership. AMDA's Board of Directors has final approval over this process.