"Evidence-based" is a descriptor that is often used to describe medically-related reference resources. Unfortunately, it is often used indiscriminately and without merit.

For a clinical reference resource to truly be called evidence-based, conclusions must be based on the best available evidence. Conclusions can be based on the best available evidence only if the evidence is consistently and systematically identified, evaluated and selected.

The DynaMed editorial process applies the following strict protocols:

1. Systematically identifying the evidence
2. Systematically selecting the best available evidence from that identified
3. Systematically evaluating the selected evidence (critical appraisal)
4. Objectively reporting the relevant findings and quality of the evidence
5. Synthesizing multiple evidence reports
6. Deriving overall conclusions and recommendations from the evidence synthesis
7. Changing the conclusions when new evidence alters the best available evidence

Every article considered for inclusion in DynaMed is processed using this system, and the system ensures the integrity of the conclusions.

Systematic literature surveillance

• To ensure that DynaMed provides the best available evidence, an extensive set of current literature is monitored daily. Systematic Literature Surveillance is conducted using more than 500 journals directly or indirectly through many journal review services and other sources of systematic evidence evaluation. For a comprehensive list of sources, see DynaMed Content Sources.
• For each source monitored, each issue is reviewed cover-to-cover. All entries are considered, because information in letters to the editor, editorials, and "reporting from the literature" pages may contain reports of new research that would be otherwise unidentified if relying exclusively on abstracts posted with traditional research articles.

Systematic searching

• In selected cases, when adding a new DynaMed topic or critically revising an existing DynaMed topic, PubMed Clinical Queries is used to provide systematic searches for identification of the best available evidence. In addition, numerous sources are searched for evidence-based reviews (such as Cochrane Database of Systematic Reviews), for guidelines (such as National Guideline Clearinghouse), and for traditional reviews.

Each article is assessed for clinical relevance and each relevant article is further assessed for validity relative to existing DynaMed content. The most valid articles are summarized, the summaries are integrated with DynaMed content, and overview statements and outline structure are updated based on the overall evidence synthesis.  Article selection is completed by editors with clinical expertise and training in scientific analysis.

Determining Relevance

Determining clinical relevance is the first consideration in systematically selecting the best available evidence from that identified. The relevance of medical information is different for every user. DynaMed is used in clinical care by practitioners with a wide range of experience and interests, and is also used in medical education. When adding information to DynaMed the following considerations are used for determining relevance:

1. Does this information have a direct bearing on patient-oriented outcomes?
Patient-oriented outcomes are outcomes that affect quality of life without extrapolation. Examples include mortality, incidence of myocardial infarction, and presence and severity of pain. These are also called clinical outcomes. Disease-oriented outcomes are used as surrogate markers for monitoring the effects of interventions ultimately intended to affect patient-oriented outcomes. Examples include cholesterol concentration, blood pressure and bone mineral density. Patients are only interested in these outcomes as a means for affecting clinically significant outcomes such as mortality or fracture incidence. Because DynaMed is primarily a clinical tool for use during patient care, patient-oriented outcomes information is considered relevant and included. Patient-oriented evidence is given priority over disease-oriented evidence, with disease-oriented evidence entered only if it adds substantially new information.

2. In the absence of patient-oriented evidence, might this information be useful in clinical decision-making?
Much of medical knowledge is still lacking in terms of patient-oriented outcomes research. Clinical decisions based on extrapolated disease-oriented evidence are not proven to be appropriate. However, clinicians still need to make decisions in situations where patient-oriented evidence is not yet available. Disease-oriented evidence is considered relevant for DynaMed in situations where patient-oriented evidence is lacking. Individual clinicians will have to determine if this information is considered relevant within their practice. When disease-oriented evidence is not presented as such in the supporting reference, DynaMed commentary will be added and may appear as "patient-oriented outcomes not assessed", "clinical outcomes not assessed", or specific commentary pointing out problems with extrapolating the information to clinical care.

3. Is this information part of a clinical controversy?
In situations where the evidence does not clearly support or refute a clinical fact, opposing views are presented. DynaMed is not designed to resolve clinical controversies and strives to present information with as little "inappropriate" bias as possible. The inherent bias towards patient-oriented outcomes is considered appropriate. Information that questions "standard" approaches and has a potential bearing on patient-oriented outcomes is considered relevant for inclusion in DynaMed.

4. Is this information that is of unique interest due to popularity?
Some medical information is not clinically relevant but widely publicized. Summarization of this type of information (often with commentary) is relevant to DynaMed users if it is likely that clinicians will be asked about it during clinical encounters. It is important for physician and patient education to point out where this type of information is not clinically applicable.

Determining Validity

Clinically relevant articles must be assessed to determine the scientific validity of conclusions and facts presented before consideration for use. Conducting critical appraisal for all articles would be wasteful if these articles would not make a change to the existing DynaMed knowledge base.
Easily identifiable study features (e.g. study method, sample size) are compared with existing studies in DynaMed to determine if new articles potentially represent the best available evidence. Articles that do not provide relevant information with validity that meets or exceeds the existing DynaMed content are excluded at this stage.

Example of importance of Step 2

Consider the question, "Is surgery effective for a painful rotator cuff tear in elderly patients?"
No randomized trials have been conducted. The best available evidence is based on two cohort studies, which are summarized in DynaMed as:

• limited evidence suggests good outcome for elderly patients (level 2 [mid-level] evidence)
• painful massive rotator cuff tears can be repaired successfully in most elderly patients
• 69 patients > 70 years old with surgical repair of massive rotator cuff tears were followed for at least 2 years
• bone-tendon repair considered good in 55 (80%), fair in 5 (7%), poor in 8 (12%) and absent in 1 patient
• 78% patients achieved satisfactory results
• Reference - J Shoulder Elbow Surg 1999 Jan-Feb;8(1):26
• consistent results reported in retrospective cohort study
• retrospective study of 105 consecutive patients > 62 years old with repair of rotator cuff tear, 92 patients with 97 rotator cuff tears were re-examined after at least 2 years
• 5 (5%) had failure of repair, 6 (6%) had complications (including 1 infection and 1 brachial plexus stretch injury)
• 87% patients had good or excellent results
• Reference - J Shoulder Elbow Surg 2001 Mar-Apr;10(2):97

This is considered more useful than simply stating an expert opinion. These articles are relevant and the best available evidence when this clinical question occurs in practice, yet would not be found in most evidence-based systems (which limit their focus to randomized trials) or textbooks (which rely primarily on expert opinion).

Abstracts in research publications often do not accurately reflect the methodologic quality and results found in full-text articles. Article summaries in other publications often do not accurately reflect the methodologic quality and results found in full-text articles.

For DynaMed, full-text evaluation of articles is required for:

1. Any article rated as Level 1 [likely reliable] evidence or Grade A recommendation [consistent high-quality evidence]
2. Any article potentially ratable as Level 1 or Grade A based on abstract-only information; full-text evaluation is necessary to provide lower levels or grades
3. Any article for which definition of absolute magnitude of effect and/or detailed description of interventions or exposures are necessary, regardless of level of evidence
4. Any article which represents the most important guidance for a DynaMed topic, regardless of level of evidence

Reports used for updating DynaMed represent the best available evidence for the specific content under consideration. Evidence may be labeled in one of three levels as outlined under Levels of Evidence. Articles that potentially warrant the highest evidence ratings undergo complete critical appraisal using methods established in the Users' Guides to Evidence-Based Practice from the Evidence-Based Medicine Working Group (available from Centre for Health Evidence). If serious methodological shortcomings are discovered (sufficient to affect clinically relevant results), then the evidence is labeled as mid-level evidence and the shortcomings are described.
Critical appraisal is completed by editors trained in critical appraisal.

Example of importance of Step 3

A meta-analysis of the effects of hormone replacement therapy on mortality reported:
"CONCLUSIONS: Hormone replacement therapy reduced total mortality in trials with mean age of participants under 60 years. No change in mortality was seen in trials with mean age over 60 years."
The DynaMed topic reads:

• HRT not associated with significant effect on overall mortality
• based on systematic review and meta-analysis of 30 randomized trials with 26,708 women
• Reference - J Gen Intern Med 2004 Jul;19(7):791 full-text, commentary can be found in BMJ 2005 Jan 1;330(7481):filler + in ACP J Club 2005 Jan-Feb;142(1):1
• DynaMed commentary -- conclusion of lower mortality in women aged < 60 years in this review not considered valid because
• analysis was based on 4,141 women in trials with mean age < 60 years
• analysis did not include 5,522 women aged 50-59 years in WHI trial (which had a mean age 63 years)
• analysis with WHI trial would find no difference in mortality
• DynaMed commentary -- entire meta-analysis fundamentally flawed by weighing studies based on number of deaths instead of sample size
• for example consider the meta-analysis of trials with mean age < 60 years which included 17 trials and 4,141 women
• 1 trial with high mortality in 130 ovarian cancer patients provided 3% of the overall sample size but was calculated as providing 41% of the weight in this meta-analysis
• DynaMed commentary -- similar conclusions for outcome of coronary heart disease events (reduced risk in women < 60 years old) reported in meta-analysis of 23 trials with 39,049 women conducted by same authors (J Gen Intern Med 2006 Apr;21(4):363) but similar methodologic flaws limit validity of conclusion

This example may be considered fundamentally flawed and inappropriate for use in a clinical reference. However, because this article may be noted in other sources, clinicians may need to know about this information so they can provide an informed response.

When reporting the evidence, DynaMed Editors consider all of the following:

1. Were all relevant outcomes reported in the original article?
2. What are the most relevant outcomes to report in the DynaMed topic?
3. For relevant outcomes, what is the magnitude of effect? This may be represented by absolute rates and number needed to treat (NNT) or harm (NNH) abbreviations, or by absolute differences in continuous variables (e.g. mean decrease in 1.3 points on 0-10 visual analog pain scale).
4. Were the findings clinically significant?
5. In the case of no statistically significant differences, were the findings robust enough to rule out clinically significant difference
6. Are there any methodologic limitations sufficient to alter reliability of clinical conclusions?

DynaMed editors check the data against original study reports, and clinical editors review all summaries for validity and relevance at the point of care. Level of evidence labeling is done by protocol with explicit reasons stated for downgrading levels of evidence.

Example of importance of Step 4

Consider the use of statins for reducing the risk of coronary disease in elderly patients. Consider a 75-year-old woman with total cholesterol 190 mg/dL, blood pressure 150/90 mmHg, and no history of heart disease, smoking or diabetes. Should she be given a statin?
There is one randomized trial reporting clinical outcomes in this population. The original abstract states "Mortality from coronary disease fell by 24% (p=0.043) in the pravastatin group." and concludes "Pravastatin given for 3 years reduced the risk of coronary disease in elderly individuals."
The DynaMed summary reads:

• in elderly patients, pravastatin may reduce coronary mortality but may increase cancer mortality so no difference in overall mortality (level 2 [mid-level] evidence)
• based on randomized trial with results of borderline statistical significance
• 5,804 patients aged 70-82 years with history of or risk factors for vascular disease were randomized to pravastatin 40 mg/day vs. placebo for mean 3.2 years, baseline cholesterol levels 4-9 mmol/L (154.3-347.6 mg/dL)
• comparing pravastatin vs. placebo
• no differences in overall mortality (10.3% vs. 10.5%)
• 3.25% vs. 4.19% death from coronary heart disease (p = 0.043, NNT 107)
• 3.98% vs. 3.12% death from cancer (p = 0.082, NNH 116)
• 14.1% vs. 16.2% combined outcome of coronary heart disease death, myocardial infarction or stroke (p = 0.014, NNT 48)
• new cancer diagnoses reported in 8.5% pravastatin patients vs. 6.8% placebo patients (NNH 59); no significant difference in new cancer diagnoses in meta-analyses incorporating other trials, but these other trials were based on younger patients
• no significant differences in women or in primary prevention in subgroup analyses
• Reference - PROSPER trial (Lancet 2002 Nov 23;360(9346):1623), editorial can be found in Lancet 2002 Nov 23;360(9346):1618, commentary can be found in Lancet 2003 Feb 1;361(9355):427 and in Lancet 2003 Mar 29;361(9363):1135, commentary can be found in ACP J Club 2003 Jul-Aug;139(1):9
• DynaMed commentary – though reduction in coronary mortality is statistically significant at p < 0.05 (95% chance that mortality reduction is less than zero), the findings do not demonstrate clinically relevant mortality reductions throughout the 95% confidence interval which ranges from NNT 57 to NNT 2,387

Attention to absolute risks instead of relative risks, attention to multiple outcomes (not just the "statistically significant" outcome), and attention to subgroup analyses, make this critical trial summary much different when comparing objective reporting with the original abstract.

Evidence-based summarization of articles is necessary, but insufficient for a point-of-care reference. Evaluating individual evidence reports requires synthesizing multiple evidence reports.

Addition, deletion, and organization of information within DynaMed is done with consideration of levels of evidence. When multiple articles are present on the same topic, preference for inclusion and organization is based on the quality of methodology, e.g. preference given to data derived from randomized controlled trials over data from prospective observational studies, which is given preference over retrospective studies, which is given preference over anecdotal reports. When data of lesser quality does not add any substantially new or different information, this data is then deleted from DynaMed.

Clinical editors review all synthesis results for validity and relevance at the point of care.

Example of importance of Step 5

Consider a question regarding the comparative efficacy of steroid injections and surgery for the treatment of carpal tunnel syndrome.
The DynaMed article summary conclusion for an article published in Arthritis Rheum 2005 Feb;52(2):612 is:

• comparing 1-2 steroid injections vs. surgery
• steroid injection improves symptoms and function more than surgery at 3 months (level 1 [likely reliable] evidence)
• no differences at 6 months (level 1 [likely reliable] evidence)
• surgery tended to have better functional improvement at 12 months (level 2 [mid-level] evidence)

But the DynaMed article summary for an article published in Neurology 2005 Jun 28;64(12):2074 is:

surgery may reduce global symptoms at 20 weeks more than single steroid injection (level 1 [likely reliable] evidence)

If these articles were not synthesized, there is a risk of reading only one report and not viewing the whole picture. These two articles are summarized together with a higher-order outline summary of:

2 trials comparing local steroid injection with surgery have varied results

Deriving overall conclusions and recommendations from the evidence synthesis is required for a comprehensive point-of-care reference. In DynaMed, multiple evidence reports of similar quality are organized such that the overall conclusions quickly provide a synthesis of the best available evidence.

Treatment Overviews (the ultimate synthesis of evidence for a clinical topic) are based upon all of the available evidence in the treatment section, and selection of the most important concepts. As new DynaMed topics are created and existing DynaMed topics are critically revised, Treatment Overviews are explicitly linked directly to the supporting evidence synthesis.

Editors confirm that overviews are clinically useful and accurately match supporting data.

Example of importance of Step 6

There are conflicting opinions regarding whether or not to use antibiotics for acute sinusitis, and which antibiotics to use. The clinician needs to be aware of this controversy, including the best available evidence, recommendations from authoritative sources, and sufficient guidance for clinical decision-making (such as drug doses) YET WITHOUT being told how to practice (cookbook medicine) or being misled by a single author's opinion ("Here’s what I do")

This is the "antibiotics" portion of the Treatment Overview for acute sinusitis:

• antibiotics controversial
• benefit is neither proven nor disproved, best evidence summaries find antibiotics statistically superior to placebo but magnitude of benefit small
• recommendations from CDC evidence-based guidelines
• acute bacterial sinusitis does NOT require antibiotic treatment, especially if symptoms are mild or moderate
• antibiotics should generally be used only for
• patients with persistent purulent nasal discharge and facial pain or tenderness who are not improving after 7 days
• patients with severe symptoms of rhinosinusitis, regardless of duration
• narrow-spectrum antibiotics are reasonable first-line agents, since no evidence for superior outcome with other newer agents
• amoxicillin 500 mg PO three times daily
• doxycycline
• trimethoprim/sulfamethoxazole 160/800 mg PO twice daily
• appropriate duration of antibiotics not clear
• 3-5 days of antibiotics are as effective as 10 days
• no trials identified comparing shorter regimens (< 10 days) with longer regimens (> 10 days)

The final step in DynaMed’s evidence-based methodology is changing conclusions when new evidence alters the best available evidence. This step is crucial because new evidence is published every day. Having new evidence summaries handled separately from reviewed content in a manner requiring the clinician to search in two locations to synthesize the entire story would make finding the best available evidence more difficult.

As soon as new evidence is evaluated using the 6 steps governing systematic processing, it is added to the appropriate DynaMed topic(s) in context. This process allows immediate and comprehensive access to the best available evidence as it occurs.

This process occurs EVERY DAY in DynaMed.