Welcome to the 2017 National Healthcare Safety Network, NHSN quick learn series.
These are brief and informative presentations from the Centers for Disease Control and Prevention.
They serve as an educational resource for healthcare facilities working to prevent Healthcare
Associated Infections, or HAIs.
In this Quick Learn, we provide new users with an introduction to NHSN Data Analysis,
specifically focusing on CMS Reports.
We educate new users on the utility and significance of data analysis.
We provide guidance on how to approach basic data analysis within the updated NHSN application.
New users learn how to navigate the updated site to generate datasets and access the analysis
reports.
This presentations demonstrates how to execute and customize reports for a given healthcare-associated
infection, HAI type.
Lastly, we discuss how to understand and interpret the analysis results.
Why is analysis important?
To begin unpacking this question, we need to understand that data is a form of currency-
in the sense that it holds limitless value.
It would be a disservice to collect data simply for the sake of storing data.
In order to maximize the power of data, we have to dig deeper to uncover its utility
and significance through the process of analysis.
Analysis helps us to understand and break down our data.
Through the NHSN data analysis tools, you are encouraged to take full ownership of your
facility’s data and build a narrative around it.
It can also be used to highlight areas that your facility excels in as well as areas that
could potentially use improvement.
This ties directly into understanding how well your facility is performing in its prevention
efforts.
The analysis tools also serve as a way for you to perform quality assurance by conducting
internal validation to promote accuracy.
Your facility’s data may potentially be used by the CDC, the state health department,
CMS, special study groups, etc. so it’s important you understand your facility’s
metrics.
Lastly, be proactive by using your data to take action!
Allow the data to be a source of empowerment-not discouragement.
The first step in performing data analysis in NHSN is generating datasets.
Once you are logged in, this can be accessed by selecting the Analysis tab on the left
hand side and clicking Generate Data Sets.
Keep in mind that the default time frame of the dataset is the three most recent full
calendar years so make adjustments if needed.
Once you are satisfied with the timeframe selection, proceed to select Generate New.
Generating datasets copies and freezes the data, specific to your individual user account.
This process takes a few minutes but you can access other areas of NHSN or log out while
it completes this process.
The data used for analysis is a frozen copy so if you make any changes within NHSN after
you generate datasets, it will not be reflected in any of the reports until you generate datasets
again.
In order to access NHSN reports, navigate to the left hand side and select the Analysis
tab.
In the drop-down, select the Reports option.
Take note of when the last data set was generated to ensure you are utilizing the most recent
dataset for your analysis.
The Analysis Reports page is the central hub for running NHSN reports.
The data analysis options are organized into folders that house sub-folders and reports.
You have the option to expand or collapse one or more folders at a time.
Although we will be focusing on CMS reports in this presentation, the basic steps to run
and modify reports can generally be replicated for other types of reports.
The CMS reports within NHSN will include the data that will be sent to CMS for the quarterly
deadlines and that are applicable to the selected quality reporting program and HAI measure,
according to the rules required for completion of those data.
More information can be found in the resources listed at the end of this presentation.
Under the CMS Reports folder, select the subfolder with your appropriate facility type such as
Acute Care Hospital, Critical Access Hospital, etc.
Within that subfolder, navigate to the desired healthcare-associated infection, HAI type
for your analysis.
For the purpose of this presentation, we will be acting as an acute care hospital interested
in examining Central Line-associated Bloodstream Infection or CLABSI data.
Once you select your desired report, you will have three options.
You can either run the report, modify the report or export the data set.
Select the Run Report option if you do not need to customize the report by any specific
parameters.
If you are interested in tailoring your report, select the Modify Report option and a new
window will pop up.
This new window will allow you to modify certain criteria such as the report format, and the
time period.
The Title-Format tab allows you to change the title of your report as well as modify
the output format.
You also have the option of showing descriptive variable names by selecting the respective
checkbox on the top left.
For further customization, the beginning and ending time period can be adjusted to stratify
by one of the four date variables in the drop-down option.
The filters tab gives you an opportunity to create additional rules for your output.
This allows you to set inclusionary and exclusionary criteria for your output.
For example, you may only be interested in looking at neonatal ICUs in your analysis,
so you would filter this report by the variable location Type.
Of all the tabs, this is where the heart of customization lies because there are a variety
of variables that can be utilized to customize your report.
Keep in mind that the more filters you add, the less output you will also have because
each additional rule narrows down your data.
Within the Display Options tab, you are able to adjust the group by variable according
to your preference.
For the purposes of CMS reporting, the data is summarized on a quarterly basis so that
will be the default option; however, this can be adjusted to fit your reporting needs.
Once you have finalized your modifications, you can choose to either run, save or export
your report.
Please remember to save a hard or electronic copy of your data prior to the CMS quarterly
deadline.
This is to ensure that you are aware of the data that will be sent to CMS on behalf of
your facility.
Once you have ran your report, you will generate tables that look identical to these, assuming
your output is in HTML format.
You will notice the first row in the first table on this slide includes different elements
such as organization ID, the location type, and the group by variable.
Each table in your report may include slightly different sets of variables so keep that in
mind as you move forward with your analysis.
In the first table, let’s look at the infection Count and the number of predicted infections
variables.
These are very important variables because they are the figures used to calculate your
standardized infection ratio or SIR value.
The infection count is the numerator and the number of predicted infections is the denominator.
The SIR value is the summary measure used to track healthcare-associated infections
over time.
This measure is risk adjusted for facility, location, and or patient-level factors, depending
on the HAI you are measuring.
So using the 2015 baseline, if a facility has an SIR value greater than one that would
indicate that there were more observed infections than predicted, based on the national 2015
data.
Likewise, if a facility has an SIR value lesser than one, this would indicate fewer observed
infections than predicted which indicates some prevention progress.
In the first table, we can see that the SIR value for the CC location type in the first
quarter of 2015 is 2.249 which indicates that the facility had more observed infections
than predicted.
We can cross-check this by dividing the number of observed infections in the first quarter
of 2015, which is four for this facility, by the number of predicted infections, which
is 1.778, and we derive the same SIR value of 2.249.
In the second table, we notice that in the third quarter of 2015, we do not have a calculated
SIR value.
This is because the SIR is only calculated when the number of predicted infections is
greater than or equal to one.
Please make it a practice to pay close attention to the footnotes whenever you run any reports
because they contain useful background information about the data contained in the report.
There are two measures that indicate whether your facility’s SIR value significantly
differs from the nominal value of one, which represent that the number of observed infections
is equal to the number of predicted infections.
The p-value is the probability value used to determine whether a significant difference
exists between your SIR value and the nominal value of one.
For NHSN purposes, we use 0.05 as the agreed upon standard of determining significance.
This means there’s a five percent chance that the difference between the two values
is due to random chance.
The difference is considered significant when the SIR p-value is less than or equal to 0.05.
In this particular example, the SIR p-value is less than 0.05 so this would be considered
a significant difference.
Furthermore, the difference is considered significant whenever the 95 percent confidence
interval is not inclusive of the number one.
In other words, when both the lower and upper bound are either less than one, as is shown
in this example, or greater than one, then there is a statistically significant difference.
As indicated in this example, the range does not include one which means we are 95 percent
confident that the true SIR lies within this range.
As you become more familiar with the NHSN application, you will quickly learn that it
will serve as a great tool to perform internal validation.
Specifically, the reports can help to shed light on any areas where there are discrepancies
or inconsistencies.
For example, this facility has all three months of data reported for the first quarter of
2015.
However, for the fourth quarter of 2015, there is only one month of data reported for this
facility so this could potentially be an area of concern.
Please always cross-check your data to ensure that there are no issues or discrepancies.
In order to promptly identify any issues, we recommend running reports on a regular
basis so that if any issues arise, you can resolve them quickly, and before the CMS quality
reporting program deadline.
For additional information, please refer to these resources regarding the SIR Guide, the
2015 baseline and more in-depth analysis training.
For any additional questions or support, contact our support team at NHSN @ CDC dot gov.