Research Design, Biostatistics, and Literature Evaluation
Assessment
Study design
| β’ | Were the hypotheses and study purpose clearly stated? |
|---|---|
| β’ | Is the study sample representative of the population with the disease/syndrome? |
| β’ | Are the inclusion/exclusion criteria too restrictive? |
| β’ | Did the study meet power? Was a sample size calculation described? |
| β’ | How were blinding and randomization conducted? |
| β’ | Is the study design translatable to clinical practice? |
| β’ | How were the primary and secondary end points defined? Have those definitions been |
validated in critically ill patients?
Outcomes
| β’ | Is the primary outcome scientifically valid and patient-centered? |
|---|---|
| β’ | Are the secondary outcomes clearly described? |
| β’ | How were adverse effects defined and analyzed? |
Analysis
| β’ | Were the statistical tests appropriate? |
|---|---|
| β’ | How were the data analyzed (intention-to-treat, per-protocol, as-treated)? |
| β’ | How large was the treatment effect? |
| β’ | Will the effect size be duplicated in clinical practice? |
| β’ | Did the author(s) provide an interpretation of the study findings and describe them in the |
context of the available knowledge?
Medical literature
Remaining current with evolving literature is a necessary skill for the critical care clinician.
Knowledge gained from primary literature can be objective, can limit bias, and may be translatable
compared with experiential knowledge.
The findings of clinical research can be limited to the conditions of the study and may not easily
confer the same benefit in clinical practice.
Number needed to treat (NNT) β Quantifies the anticipated effect of a treatment in a patient
population on the basis of study results. A low number signifies an effective treatment; a high
number indicates a less effective therapy. NNT = 1/ARR
ii.
Number needed to harm (NNH) β Quantifies the associated harm after exposure to a treatment.
A low number signifies a harmful treatment; a high number indicates a safer treatment. NNH
= 1/attributable risk
| d. | The study results limit the findings attributable to chance but may not fully exclude chance. |
|---|
Remember: Studies are conducted in samples and may or may not be representative of the population.
Fragility index: The number of non-events in the overall sample that need to occur to render a
significant result nonsignificant. In the CRICS-TRIGGERSEP trial, an additional three events for
the outcome of 90-day mortality would have rendered the trial findings nonsignificant (shifting to a
p=0.053), indicating the fragility of the final results (N Engl J Med 2018;378:809-18).
Knowledge accumulated through clinical experience and for a sustained period is valuable.
How does an individual patient differ from the patient sample of the study being applied?
When possible, this type of knowledge should be reinforced with scientific evidence. Example: N
Engl J Med 2009;361:1925-34
Pathophysiologic reasoning
Application of physiologic concepts to drive or reinforce therapeutic choices