Welcome to the 2017 National Healthcare Safety Network, or NHSN, Quick Learn series.
These CDC presentations are an educational resource for facilities working to prevent
Healthcare Associated Infections, or H-A-Is.
This session will review basic statistics for NHSN analysis.
In this program, we will introduce fundamental principles of statistical testing, including
the p-value and confidence intervals, and discuss their application to the Standardized
Infection Ratio, or S-I-R.
We will also review the difference between statistical and practical significance and
how to use this knowledge to translate analysis into prevention.
This presentation assumes you have a basic understanding of the S-I-R and its calculation.
Statistical testing in NHSN is used to determine if there is a significant difference between
two values.
To put this into context, we would use the results of statistical testing to answer questions
such as: Has our S-I-R changed over time?
Or, is our hospital’s S-I-R significantly different from the national baseline?
Remember that an S-I-R of one-point-zero represents the national baseline.
If there is a statistically significant difference between two values, then the difference between
those values was likely not caused by random chance.
Let’s walk through an example.
Our hospital performed fifty colon procedures in the first quarter of 2016.
There were four surgical site infections identified, and NHSN predicted 3.2 SSIs during that time
period.
We can calculate our S-I-R as four, divided by three-point-two, which results in one-point-two-five.
The hospital wants to know whether that S-I-R of 1.25 is significantly different than the
national baseline of one.
We measure statistical significance using p-values, which tell us about the rarity of
the data.
In our earlier example, a p-value would give us the probability that our S-I-R of 1.25
happened by random chance alone.
NHSN uses the conventional cut-point of five percent to determine rarity: that is, if the
p-value is less than or equal to zero-point-zero-five (or, less than 5%), we conclude that our S-I-R
was not the result of random chance, and therefore it is significantly different than the national
baseline of one.
If the p-value is greater than point-zero-five, there is a higher likelihood that the S-I-R
was a result of random chance and that it’s not statistically different than the national
baseline.
Within NHSN’s S-I-R reports, you will see that the p-value is already calculated for
you, so that you can easily interpret whether your S-I-R is significantly different than
the national baseline.
However, you may also perform this calculation on your own using either the NHSN Statistics
Calculator or the available SAS macro.
Refer to resources at the end of this presentation.
In the previous example, our facility had a Colon SSI S-I-R of 1.25.
After reviewing the NHSN S-I-R report, we determined that the p-value was 0.6169.
This p-value is above 0.05, and therefore we conclude that our facility’s SSI S-I-R
was not significantly different than the national baseline.
Another statistical measure that’s commonly used is the ninety-five percent confidence
interval.
This interval is a measurement of statistical precision and is interpreted together with
the p-value.
We use this interval as way to address the idea that a hospital’s S-I-R is a measurement,
and that this measurement is different from an unknowable, absolute truth.
For example, our hospital’s SSI S-I-R for 2016 Q1 was 1.25.
Hypothetically, if our hospital repeated SSI surveillance for the entire quarter, would
you expect the S-I-R to be exactly the same?
(short pause) Probably not.
In reality, we know that HAI surveillance is never repeated for any time period, but
statistics allow us to consider the possibility that different results might have been seen
if our hospital conducted the same surveillance multiple times.
Because we can never know the absolute true performance of our facility, we must account
for the potential variability in S-I-R values.
Let’s look at an example.
Our hospital’s SSI S-I-R was 1.25.
After reviewing the NHSN S-I-R report, we noted that the 95 percent confidence interval
ranges from zero-point-three-nine-seven, to three-point-zero-one-five.
Because this interval includes the value of one, we conclude that our S-I-R is not significantly
different than one.
If the 95 percent interval did not contain the value of one, we would conclude that the
S-I-R was significantly different than one.
The interpretation of the 95 percent confidence interval almost always agrees with the interpretation
of the p-value.
The 95 percent confidence interval also provides us with a measure of precision: a wide confidence
interval indicates that there is lower precision in the S-I-R, meaning that the volume of data
from the facility used to calculate the S-I-R, such as the number of procedures, was low.
This table provides examples of S-I-R values and their 95 percent confidence intervals.
Using the 95 percent confidence interval, we determine whether or not each S-I-R is
statistically significant.
In addition, we look at the width of the interval to determine whether or not the S-I-R is precise.
Another way to view the S-I-R data is via a line graph.
Take a look at this example.
Six facilities are identified with letters A through F. The dotted blue vertical line
represents the national baseline of 1.0.
Facilities A and B have non-significant S-I-Rs as their confidence intervals cross the dotted
line; their S-I-Rs are not significantly different from the national baseline.
Facilities C and D have statistically significant S-I-Rs.
This is because the entire confidence interval does not cross the dotted line.
Their S-I-Rs are significantly lower than the national baseline.
Facilities E and F also have statistically significant S-I-Rs that are significantly
higher, or worse, than the national baseline.
Statistical testing, although very useful, may not paint a complete picture of your facility’s
performance.
As we mentioned before, statistical values such as the 95 percent confidence interval
are highly influenced by sample size, or exposure volume.
A facility that performs 1,000 procedures a month may find an S-I-R of one-point zero-five
significant, whereas a facility that performs only 10 procedures per month may find the
same S-I-R value to be not significant.
This is called statistical power, the ability of the statistical test to detect statistically
significant differences.
Often, facilities with low exposure volume may never obtain a statistically significant
S-I-R.
Even if the S-I-R is not statistically significant, there may still be opportunity for infection
prevention.
We recommend using a combination of statistical and practical significance to make infection
prevention decisions.
In the previous example, facility ‘A’ had an S-I-R of two-point-four-three.
Because it’s greater than one, it means that the number of observed infections is
higher than the number of predicted infections.
Even though this S-I-R is not significantly different from the national baseline, there
is still an excess number of observed infections, and prevention activities may be needed.
Another way to examine practical significance is to look at your S-I-Rs over time.
Instead of comparing your S-I-Rs to one, you can use the NHSN Statistics Calculator tool
to statistically compare your current S-I-R to a past S-I-Rs.
You can also look at your S-I-R values qualitatively over time.
This may help identify glaring increases or decreases in your H-A-I incidence.
If you do perform this comparison over time, it is very important that both S-I-Rs in your
comparison must have been calculated under the same national baseline.
A 2015 S-I-R calculated using the updated 2015 national baseline cannot be compared
to an S-I-R that was calculated on the original national baseline.
This slide provides links to helpful resources, such as instructions for using the NHSN Statistics
Calculator, as well as the location of SAS macros and S-I-R rebaseline information.
This concludes the NHSN Quick Learn on basic statistics for NHSN analysis.
For additional questions, please contact NHSN user support at NHSN@c-d-c-dot-gov.