"Evidence-based" is a descriptor that is often used to describe medically-related reference resources. Unfortunately, it is often used indiscriminately and without merit.

For a clinical reference resource to truly be called evidence-based, conclusions must be based on the best available evidence. Conclusions can be based on the best available evidence only if the evidence is consistently and systematically identified, evaluated and selected.

The DynaMed editorial process applies the following strict protocols:

1. Systematically identifying the evidence
2. Systematically selecting the best available evidence from that identified
3. Systematically evaluating the selected evidence (critical appraisal)
4. Objectively reporting the relevant findings and quality of the evidence
5. Synthesizing multiple evidence reports
6. Deriving overall conclusions and recommendations from the evidence synthesis
7. Changing the conclusions when new evidence alters the best available evidence

Every article considered for inclusion in DynaMed is processed using this system, and the system ensures the integrity of the conclusions.

Systematic literature surveillance

To ensure that DynaMed provides the best available evidence, an extensive set of current literature is monitored daily. Systematic Literature Surveillance is conducted using many journals, journal review services, systematic review collections, guideline collections and other sources considered relevant to a point-of-care clinical reference. For a comprehensive list of sources, see DynaMed Content Sources.

Example of Importance of Step 1

In traditional medical publishing authors are expected to use their expertise to provide reviews that are current and thorough. Authors may search the literature for information to supplement what they know, but searches may be limited to the information they deem important to consider. New evidence could be clinically important but widely unknown and unconsidered, especially if not marketed to raise awareness.

Here is an example of useful information when caring for wrist fracture patients that would not be an expected consideration if it were not for systematic literature surveillance:

Each article is assessed for clinical relevance and each relevant article is further assessed for validity relative to existing DynaMed content. The most valid articles are summarized, the summaries are integrated with DynaMed content, and overview statements and outline structure are updated based on the overall evidence synthesis.  Article selection is completed by editors with clinical expertise and training in scientific analysis.

Determining Relevance

Determining clinical relevance is the first consideration in systematically selecting the best available evidence from that identified. The relevance of medical information is different for every user. DynaMed is used in clinical care by practitioners with a wide range of experience and interests, and is also used in medical education. When adding information to DynaMed the following considerations are used for determining relevance:

1. Does this information have a direct bearing on patient-oriented outcomes?

Patient-oriented outcomes are outcomes that affect quality of life without extrapolation. Examples include mortality, incidence of myocardial infarction, and presence and severity of pain. These are also called clinical outcomes. Disease-oriented outcomes are used as surrogate markers for monitoring the effects of interventions ultimately intended to affect patient-oriented outcomes. Examples include cholesterol concentration, blood pressure and bone mineral density. Patients are only interested in these outcomes as a means for affecting clinically significant outcomes such as mortality or fracture incidence.

Because DynaMed is primarily a clinical tool for use during patient care, patient-oriented outcomes information is considered relevant and included. Patient-oriented evidence is given priority over disease-oriented evidence, with disease-oriented evidence entered only if it adds substantially new information.

2. In the absence of patient-oriented evidence, might this information be useful in clinical decision-making?

Much of medical knowledge is still lacking in terms of patient-oriented outcomes research. Clinical decisions based on extrapolated disease-oriented evidence are not proven to be appropriate. However, clinicians still need to make decisions in situations where patient-oriented evidence is not yet available.

Disease-oriented evidence is considered relevant for DynaMed in situations where patient-oriented evidence is lacking. Individual clinicians will have to determine if this information is considered relevant within their practice. When disease-oriented evidence is not presented as such in the supporting reference, DynaMed commentary will be added and may appear as "patient-oriented outcomes not assessed", "clinical outcomes not assessed", or specific commentary pointing out problems with extrapolating the information to clinical care.

3. Is this information part of a clinical controversy?

In situations where the evidence does not clearly support or refute a clinical fact, opposing views are presented. DynaMed is not designed to resolve clinical controversies and strives to present information with as little "inappropriate" bias as possible. The inherent bias towards patient-oriented outcomes is considered appropriate. Information that questions "standard" approaches and has a potential bearing on patient-oriented outcomes is considered relevant for inclusion in DynaMed.

4. Is this information that is of unique interest due to popularity?

Some medical information is not clinically relevant but widely publicized. Summarization of this type of information (often with commentary) is relevant to DynaMed users if it is likely that clinicians will be asked about it during clinical encounters. It is important for physician and patient education to point out where this type of information is not clinically applicable.

Determining Validity

Clinically relevant articles must be assessed to determine the scientific validity of conclusions and facts presented before consideration for use. Conducting critical appraisal for all articles would be wasteful if these articles would not make a change to the existing DynaMed knowledge base.

Easily identifiable study features (e.g. study method, sample size) are compared with existing studies in DynaMed to determine if new articles potentially represent the best available evidence. Articles that do not provide relevant information with validity that meets or exceeds the existing DynaMed content are excluded at this stage.

Example of importance of Step 2

Consider the question, "Is surgery effective for a painful rotator cuff tear in elderly patients?"
No randomized trials have been conducted. The best available evidence is based on two cohort studies, which are summarized in DynaMed as:

• limited evidence suggests good outcome for elderly patients (level 2 [mid-level] evidence)
• painful massive rotator cuff tears can be repaired successfully in most elderly patients
• 69 patients > 70 years old with surgical repair of massive rotator cuff tears were followed for at least 2 years
• bone-tendon repair considered good in 55 (80%), fair in 5 (7%), poor in 8 (12%) and absent in 1 patient
• 78% patients achieved satisfactory results
• Reference - J Shoulder Elbow Surg 1999 Jan-Feb;8(1):26
• consistent results reported in retrospective cohort study
• retrospective study of 105 consecutive patients > 62 years old with repair of rotator cuff tear, 92 patients with 97 rotator cuff tears were re-examined after at least 2 years
• 5 (5%) had failure of repair, 6 (6%) had complications (including 1 infection and 1 brachial plexus stretch injury)
• 87% patients had good or excellent results
• Reference - J Shoulder Elbow Surg 2001 Mar-Apr;10(2):97

This is considered more useful than simply stating an expert opinion. These articles are relevant and the best available evidence when this clinical question occurs in practice, yet would not be found in most evidence-based systems (which limit their focus to randomized trials) or textbooks (which rely primarily on expert opinion).

Abstracts in research publications often do not accurately reflect the methodologic quality and results found in full-text articles. Article summaries in other publications often do not accurately reflect the methodologic quality and results found in full-text articles.

For DynaMed, full-text evaluation of articles is required for:

1. Any article rated as Level 1 [likely reliable] evidence or Grade A recommendation [consistent high-quality evidence]
2. Any article potentially ratable as Level 1 or Grade A based on abstract-only information; full-text evaluation is necessary to provide lower levels or grades
3. Any article for which definition of absolute magnitude of effect and/or detailed description of interventions or exposures are necessary, regardless of level of evidence
4. Any article which represents the most important guidance for a DynaMed topic, regardless of level of evidence

Reports used for updating DynaMed represent the best available evidence for the specific content under consideration. Evidence may be labeled in one of three levels as outlined under Levels of Evidence.

Articles that potentially warrant the highest evidence ratings undergo complete critical appraisal using methods established in the Users' Guides to Evidence-Based Practice from the Evidence-Based Medicine Working Group (available from Centre for Health Evidence).

If serious methodological shortcomings are discovered (sufficient to affect clinically relevant results), then the evidence is labeled as mid-level evidence and the shortcomings are described.

Critical appraisal is completed by editors trained in critical appraisal.

Example of importance of Step 3

Conclusions of randomized trials need to be put in perspective of biases that may influence the evidence.  Critical appraisal includes evaluation of methodology and distinguishing clinical outcomes from surrogate outcomes.

In 2008 the ADVANCE trial was reported with a conclusion that “A strategy of intensive glucose control, involving gliclazide (modified release) and other drugs as required, that lowered the glycated hemoglobin value to 6.5% yielded a 10% relative reduction in the combined outcome of major macrovascular and microvascular events, primarily as a consequence of a 21% relative reduction in nephropathy.”

Here is the DynaMed summary of the ADVANCE trial:

• target HbA1c < 6.5% using gliclazide as initial therapy may increase hospitalization (level 2 [mid-level] evidence) with "benefit" possibly limited to reduction in development in macroalbuminuria (level 3 [lacking direct] evidence)
• based on randomized trial with only statistically significant difference in clinical outcomes limited to 1 secondary outcome
• 11,140 patients ≥ 55 years old with type 2 diabetes with history of or additional risk factor for vascular disease were randomized to intensive glucose lowering vs. standard guidelines-based glucose lowering
• 6-week run-in period included usual methods of glucose control and fixed combination of perindopril and indapamide (blood pressure lowering arm)
• randomization after run-in period
• intensive blood glucose control (target HbA1c ≤ 6.5%) with modified release gliclazide 30-120 mg daily
• standard glucose control with target HbA1c levels defined on basis of local guidelines
• median follow-up 5 years
• comparing intensive blood glucose control vs. standard glucose control
• mean HbA1c 6.5% vs. 7.3%
• composite outcome of major macrovascular and microvascular events reached in 18.1% vs. 20% (p = 0.01, NNT 53)
• macrovascular events in 10% vs. 10.6% (not significant)
• nonfatal myocardial infarction in 2.7% vs. 2.8% (not significant)
• nonfatal stroke in 3.8% vs. 3.8% (not significant)
• death from cardiovascular causes in 4.5% vs. 5.2% (not significant)
• microvascular events in 9.4% vs. 10.9% (p = 0.01, NNT 67)
• new or worsening nephropathy in 4.1% vs. 5.2% (p = 0.006, NNT 91)
• macroalbuminuria developed in 2.9% vs. 4.1% (p < 0.001, NNT 84)
• need for renal-replacement therapy or death from renal disease in 0.4% vs. 0.6% (p = 0.09)
• doubling of serum creatinine level to ≥ 200 mcmol/L (2.26 mg/dL) in 1.2% vs. 1.1% (not significant)
• new or worsening retinopathy in 6% vs. 6.3% (not significant)
• new-onset microalbuminuria in 23.7 % vs. 25.7% (p = 0.02, NNT 50)
• adverse events
• death from any cause in 8.9% vs. 9.6% (not significant)
• hospitalization for any cause in 44.9% vs. 42.8% (p = 0.03, NNH 47)
• hospitalization due to severe hypoglycemia in 1.1% vs. 0.7% (p = 0.04, NNH 250)
• severe hypoglycemia in 2.7% vs. 1.5% (p < 0.001, NNH 83)
• Reference - ADVANCE trial (N Engl J Med 2008 Jun 12;358(24):2560)

When reporting the evidence, DynaMed Editors consider all of the following questions:

1. Were all relevant outcomes reported in the original article?
2. What are the most relevant outcomes to report in the DynaMed topic?
3. For relevant outcomes, what is the magnitude of effect? This may be represented by absolute rates and number needed to treat (NNT) or harm (NNH) abbreviations, or by absolute differences in continuous variables (e.g. mean decrease in 1.3 points on 0-10 visual analog pain scale).
4. Were the findings clinically significant?
5. In the case of no statistically significant differences, were the findings robust enough to rule out clinically significant difference?
6. Are there any methodologic limitations sufficient to alter reliability of clinical conclusions?

DynaMed editors check the data against original study reports, and clinical editors review all summaries for validity and relevance at the point of care. Level of evidence labeling is done by protocol with explicit reasons stated for downgrading levels of evidence.

Example of importance of Step 4

Consider the use of statins for reducing the risk of coronary disease in elderly patients. Consider a 75-year-old woman with total cholesterol 190 mg/dL, blood pressure 150/90 mmHg, and no history of heart disease, smoking or diabetes. Should she be given a statin?

There is one randomized trial reporting clinical outcomes in this population. The original abstract states "Mortality from coronary disease fell by 24% (p=0.043) in the pravastatin group." and concludes "Pravastatin given for 3 years reduced the risk of coronary disease in elderly individuals."

The DynaMed summary reads:

• in elderly patients, pravastatin may reduce coronary mortality but may increase cancer mortality so no difference in overall mortality (level 2 [mid-level] evidence)
• based on randomized trial with results of borderline statistical significance
• 5,804 patients aged 70-82 years with history of or risk factors for vascular disease were randomized to pravastatin 40 mg/day vs. placebo for mean 3.2 years, baseline cholesterol levels 4-9 mmol/L (154.3-347.6 mg/dL)
• comparing pravastatin vs. placebo
• no differences in overall mortality (10.3% vs. 10.5%)
• 3.25% vs. 4.19% death from coronary heart disease (p = 0.043, NNT 107)
• 3.98% vs. 3.12% death from cancer (p = 0.082, NNH 116)
• 14.1% vs. 16.2% combined outcome of coronary heart disease death, myocardial infarction or stroke (p = 0.014, NNT 48)
• new cancer diagnoses reported in 8.5% pravastatin patients vs. 6.8% placebo patients (NNH 59); no significant difference in new cancer diagnoses in meta-analyses incorporating other trials, but these other trials were based on younger patients
• no significant differences in women or in primary prevention in subgroup analyses
• Reference - PROSPER trial (Lancet 2002 Nov 23;360(9346):1623), editorial can be found in Lancet 2002 Nov 23;360(9346):1618, commentary can be found in Lancet 2003 Feb 1;361(9355):427 and in Lancet 2003 Mar 29;361(9363):1135, commentary can be found in ACP J Club 2003 Jul-Aug;139(1):9
• DynaMed commentary – though reduction in coronary mortality is statistically significant at p < 0.05 (95% chance that mortality reduction is less than zero), the findings do not demonstrate clinically relevant mortality reductions throughout the 95% confidence interval which ranges from NNT 57 to NNT 2,387

Attention to absolute risks instead of relative risks, attention to multiple outcomes (not just the "statistically significant" outcome), and attention to subgroup analyses, make this critical trial summary much different when comparing objective reporting with the original abstract.

Evidence-based summarization of articles is necessary, but insufficient for a point-of-care reference. Evaluating individual evidence reports requires synthesizing multiple evidence reports.

Addition, deletion, and organization of information within DynaMed is done with consideration of levels of evidence. When multiple articles are present on the same topic, preference for inclusion and organization is based on the quality of methodology, e.g. preference given to data derived from randomized controlled trials over data from prospective observational studies, which is given preference over retrospective studies, which is given preference over anecdotal reports. When data of lesser quality does not add any substantially new or different information, this data is then deleted from DynaMed.

Clinical editors review all synthesis results for validity and relevance at the point of care.

Example of importance of Step 5

Reading the result of a single study can be misleading if unaware of related studies. Yet during clinical practice there is not enough time to read many separate study summaries.

Coenzyme Q10 has been promoted for use in diabetes and the synthesis in DynaMed - Diabetes alternative treatments (biologic therapies) topic - provides a quick summary of 5 randomized trials:

Coenzyme Q10:

• coenzyme Q10 has inconsistent results on blood pressure and glycemic control in 4 randomized trials
• coenzyme Q10 100 mg twice daily may improve blood pressure and glycemic control (level 2 [mid-level] evidence)
• based on 2 randomized trials without clinical outcomes
• coenzyme Q 100 mg twice daily reduced blood pressure by 6.1/2.9 mm Hg and HbA1c by 0.37% in 12-week randomized placebo-controlled trial in 74 patients with diabetes (Eur J Clin Nutr 2002 Nov;56(11):1137)
• coenzyme Q 100 mg twice daily reduced systolic blood pressure and HbA1c in 12-week randomized placebo-controlled trial in 80 patients with type 2 diabetes and dyslipidemia (Atherosclerosis 2003 May;168(1):169)
• coenzyme Q10 100 mg twice daily may NOT affect glycemic control, blood pressure or lipid levels (level 3 [lacking direct] evidence)
• based on 3 randomized trials without clinical outcomes
• no significant differences in glycemic control, blood pressure or lipid levels in 6-month randomized placebo-controlled trial in 23 patients with type 2 diabetes (Biofactors 1999;9(2-4):315)
• no significant differences in glucose, lipid levels or blood pressure in 12-week randomized placebo-controlled trial in 40 patients with type 2 diabetes; coenzyme Q10 reported to improve resting blood flow and endothelial function in brachial artery (Diabetologia 2002 Mar;45(3):420) 
• coenzyme Q10 100 mg twice daily did NOT significantly affect insulin dose, glycemic control, cholesterol levels or general well-being in 12-week randomized placebo-controlled trial of 34 patients with type 1 diabetes (Mol Aspects Med 1997;18 Suppl:S307, Diabet Med 1999 Apr;16(4):312 in Alternative Medicine Alert 2004 Jun;7(6):61)

Deriving overall conclusions and recommendations from the evidence synthesis is required for a comprehensive point-of-care reference. In DynaMed, multiple evidence reports of similar quality are organized such that the overall conclusions quickly provide a synthesis of the best available evidence.

Treatment Overviews (the ultimate synthesis of evidence for a clinical topic) are based upon all of the available evidence in the treatment section, and selection of the most important concepts. As new DynaMed topics are created and existing DynaMed topics are critically revised, Treatment Overviews are explicitly linked directly to the supporting evidence synthesis.

Editors confirm that overviews are clinically useful and accurately match supporting data.

Example of importance of Step 6

There are conflicting opinions regarding whether or not to use antibiotics for acute sinusitis, and which antibiotics to use. The clinician needs to be aware of this controversy, including the best available evidence, recommendations from authoritative sources, and sufficient guidance for clinical decision-making (such as drug doses) YET WITHOUT being told how to practice (cookbook medicine) or being misled by a single author's opinion ("Here’s what I do")

This is the "antibiotics" portion of the Treatment Overview for Acute sinusitis:

• antibiotics controversial
• benefit is neither proven nor disproved, best evidence summaries find antibiotics statistically superior to placebo but magnitude of benefit small
• recommendations from CDC evidence-based guidelines
• acute bacterial sinusitis does NOT require antibiotic treatment, especially if symptoms are mild or moderate
• antibiotics should generally be used only for
• patients with persistent purulent nasal discharge and facial pain or tenderness who are not improving after 7 days
• patients with severe symptoms of rhinosinusitis, regardless of duration
• narrow-spectrum antibiotics are reasonable first-line agents, since no evidence for superior outcome with other newer agents
• amoxicillin 500 mg PO three times daily
• doxycycline
• trimethoprim/sulfamethoxazole 160/800 mg PO twice daily
• appropriate duration of antibiotics not clear
• 3-5 days of antibiotics are as effective as 10 days
• no trials identified comparing shorter regimens (< 10 days) with longer regimens (> 10 days)

The final step in DynaMed’s evidence-based methodology is changing conclusions when new evidence alters the best available evidence. This step is crucial because new evidence is published every day. Having new evidence summaries handled separately from reviewed content in a manner requiring the clinician to search in two locations to synthesize the entire story would make finding the best available evidence more difficult.

As soon as new evidence is evaluated using the 6 steps governing systematic processing, it is added to the appropriate DynaMed topic(s) in context. This process allows immediate and comprehensive access to the best available evidence as it occurs.

This process occurs EVERY DAY in DynaMed.

Example of importance of Step 7

Prophylactic antibiotics is a common practice for children with a urinary tract infection (UTI). As of 2008 there were six randomized trials which evaluated symptomatic recurrent UTI as an outcome. The largest of these trials had 338 patients and all of these trials had methodologic limitations. Putting all of these trials together the best available evidence suggested no benefit for prophylactic antibiotics. In 2009 a larger high-quality trial showed a reduction in symptomatic recurrent UTI, prompting a change in the overall evidence synthesis:

From the Urinary tract infection (UTI) in children topic:

• inconsistent evidence for use of prophylactic antibiotics against symptomatic UTI
• long-term low-dose trimethoprim-sulfamethoxazole decreases recurrent urinary tract infections in children (level 1 [likely reliable] evidence)
• based on randomized trial
• 576 children (median age 14 months) with ≥ 1 symptomatic urinary tract infection randomized to trimethoprim 2 mg/kg plus sulfamethoxazole 10 mg/kg suspension vs. placebo daily for 12 months
• all children had at least 1 urinary tract infection, 42% had known vesicoureteral reflux
• enrollment ceased before 780 patients recruited due to slow accrual rate and without knowledge of outcomes (did not affect power for primary outcome)
• symptomatic urinary tract infection defined as symptoms plus positive urine culture
• symptomatic urinary tract infection within 12 months in 13% with trimethoprim-sulfamethoxazole vs. 19% with placebo (p = 0.02, NNT 17)
• no significant differences in adverse effects
• Reference - N Engl J Med 2009 Oct 29;361(18):1748, editorial can be found in N Engl J Med 2009 Oct 29;361(18):1804
• other trials with mixed results
• antibiotic prophylaxis may not reduce recurrence of febrile UTI in children with and without primary nonsevere vesicoureteral reflux (level 2 [mid-level] evidence)
• …