hello everyone today I'll be discussing
how to appraise a clinical trial I'll be
splitting this into three parts this is
part 1 which will serve as a brief
introduction part 2 will cover the
relatively easy questions and
fundamental issues to examine when
appraising a trial and part 3 will cover
the relatively challenging questions and
some of the more nuanced issues
appraising evidence is a skill that must
be practiced therefore after you've
watched the videos I strongly recommend
that you find a paper of interest and
relevance to a recent patient of yours
and go through it and practice the
analysis I'll be presenting a decade
after finishing my training I've come to
believe that appraisal and proper
application of the medical literature is
one of the most underappreciated skills
in all of medicine it's amazing how much
of the care doctors routinely provide is
based upon studies which are either
flawed have been misinterpreted or
misaligned to patients well outside the
studies population but as a doctor
you'll never be able to distinguish care
based on weak or irrelevant evidence
from that based on strong evidence
unless you master the appraisal process
and you'll never master appraising
evidence without practicing it if you
watched my introduction to
evidence-based medicine you may recall
the five days of EBM first assess the
clinical problem second ask a question
that could be answered using the primary
literature next acquire the evidence
which will usually but not always be a
clinical trial then appraise that
evidence and finally apply the evidence
to your patient these videos will be
addressing that fourth a which is the
one that most trainees find to be the
most challenging there are four primary
reasons we appraise evidence after we've
acquired it rather than just accepting
the author's punchline in the papers
conclusion first we want to be sure that
we agree with the authors as to whether
or not the study results could have been
the result of chance in other words do
we agree with their assessments
as to whether or not the trial was a
so-called positive or negative one
second we want to assess whether the
study was biased meaning was there an
aspect of the studies design that could
have influenced the accuracy of the
results and spoiler alert all studies
are biased it's just a matter of how and
to what degree next a closely related
reason to determine if the study is
valid I'm going to talk more about
validity in part 3 but in short a
balanced study is one that represents
truth or reality an internally valid
study is one that accurately measures a
true effect in the specific group of
patients studied an externally valid
study is one that also accurately
measures a true effect in a broader
population beyond the specific study
participants and last we want to gather
information that what helps to
contextualize the study results so that
we can apply those results to our
patients in an informed way there are
many approaches to appraising trials
I've heard some EVM experts say to
ignore the intro and discussion and
focus on just the methods and results
I've heard others advised to ignore the
methods section as well and I've heard
yet others say all you need to appraise
the paper is to examine the charts and
figures and you can literally ignore the
text now I won't claim that my specific
approach is the only one people should
use but you really should not be
ignoring any part of the paper all of it
will be relevant to your appraisal there
are several ways in which one can look
at all the information but the most
logical to me is to go through a series
of questions about the paper in which
these questions are grouped based upon
the sections of the paper for example
the introduction methods results and
discussion and both because this series
of questions is relatively long and
because those of you watching are coming
to this video with a wide range of prior
experience with the medical literature
as I mentioned in the beginning I've
divided the questions into separate
lists the first list is composed of the
easy questions most
appropriate for medical students which
will be covered in part two of this
video the second list predictably
is composed of the challenging questions
which may require more familiarity with
either clinical medicine or research
methods these will be covered in part 3
before starting the appraisal process
it's necessary to become familiar with
the sections of a scientific paper if
you aren't already this is
straightforward first of course is the
title followed by the author's the
largely unwritten rules are complicated
as to which individuals get to be listed
as an author and what they're listed
order is however one of the main roles
in the medical literature is that the
individual who made the most significant
contribution to the study and who
usually has written the greatest portion
of the actual paper should be listed
first and the so called principal
investigator who is usually a senior
mentor to the first author is listed
last next comes the abstract the
abstract is a miniature version of the
full paper with each of the sections
being summarized in two to five
sentences the purpose of the abstracts
compact summary of the paper is to help
clinicians decide if the paper is
relevant to their clinical questions
prior to committing the time to read the
whole thing
unfortunately in practice many
clinicians read the abstracts in place
of the full paper thus assuming based on
excessively optimistic faith in peer
review that the papers results and
conclusions are valid it obviously saves
time to do this but I don't recommend it
for important clinical questions relying
solely on the abstract completely skips
the appraisal step of the EBM process
and will lead you to accept too many
conclusions that's true without
considering why the study may be biased
or not as relevant as it first appears
after the abstract the paper proper
begins with the introduction where the
author's summarize the relevant previous
research that has been done on the
clinical question they are investigating
and they make the case as to why the
clinical question was worthy of
investigation in the first place then
are the methods in which the author's
explained what they did
this is followed by the results which is
an objective description of what was
observed within the results section will
be a handful of figures which are visual
representations of that observed data
finally is the discussion in which the
authors make conclusions from the
results which usually includes
hypotheses to explain the observations
well-written discussions will also
propose future studies that can test
these new hypotheses and they also will
covered limitations of the study you can
summarize these four sections as the
author stating why we did this what we
did what we observed and why we think we
observed this the last issue I'll
discuss in this introduction is the
frame of mind one should have when
approaching the appraisal of a clinical
trial most clinicians start off
accepting the results and conclusions of
the authors and then look for reasons
why those results may be biased and why
the study might be limited I think
that's going about it backwards instead
I use strong null hypothesis of trial
appraisal you should assume a trial is
garbage until its authors convince you
otherwise
now that might strike some of you as a
bit flippant but you may have also seen
my previous video on why most published
medical research is wrong if most
published research is wrong then it's
only logical to approach a brand new
study with the prediction that it is
more likely than not to be wrong also
therefore the author's need to really
work at making their case that their
trial was legit and that clinicians
should accept their conclusions part 2
of this video we'll discuss the easy
appraisal questions grouped according to
the eight four mentioned sections of a
paper
you