The U.S. Census Bureau developed a Statistical Testing Tool to allow data
users to carry out statistical testing with the American Community Survey, or
ACS, and other Census Bureau surveys.
This video provides step-by-step instructions for using the Tool to
determine whether differences between ACS estimates are statistically
different. The Statistical
Testing Tool can be used to compare two estimates
or many estimates, such as comparing estimates for all counties in a state.
The Tool uses Z-scores to determine statistical significance at the 90%
confidence level. A statistically significant result means that there is
less than a 10 percent chance that the observed difference between the
estimates occurred randomly. The Statistical Testing Tool can be
found on the Census Bureau’s Web site at census.gov/acs.
From the ACS homepage, click on “Guidance for Data Users”
Then click on “Statistical Testing Tool.”
This Web page provides a link to the Tool and a step-by-step written tutorial.
The Statistical Testing Tool includes multiple sheets, or tabs:
The Overview tab provides information on the importance
of statistical testing and the Tool’s basic features.
The Instructions tab provides step-by-step instructions on
how to copy data into the Tool and interpret results.
The Statistical Testing for Two Estimates tab allows you
to insert estimates and margins of error when comparing two
estimates.
Suppose you’re a city planner in Henderson, Nevada, who wants to
understand how commuting patterns changed in your city between 2015 and
2019. First, access the data you need from the
Census Bureau’s data.census.gov Web site.
Refer to the written tutorial for guidance on accessing these data.
From the results page, copy and paste the desired row labels, estimates, and
margins of error into the Two Estimates tab.
To copy cells, left-click and drag the mouse to highlight the desired cells.
Then right-click on the highlighted cells to select the option to “Copy” or “Copy
with Headers.” Both options work with the Statistical Testing Tool.
Start with the 2015 data. Then repeat these steps to copy and
paste the 2019 ACS estimates and margins of error into the spreadsheet.
The results are displayed in the last column. They show that the share of
workers in Henderson who traveled by car, truck, or van decreased between
2015 and 2019, and so did the share who drove alone. The shares who
traveled by public transportation—and who worked from home—increased.
The mean travel time to work also increased.
There were no statistically significant differences between the remaining
estimates.
The Statistical Testing Tool can also be used to compare many groups or
geographies. Suppose you’re a state policymaker in
Pennsylvania interested in understanding how child poverty rates
differ across counties. You can use the Multiple Estimates tab
to conduct statistical testing across all counties in the state.
First, access the data you need from data.census.gov. For this example, you
access poverty statistics from the 2016 through 2020 ACS 5-year estimates.
To obtain these estimates in an easy-to- use format for the Statistical Testing
Tool, download the data using the “Zip” option.
Next, find the desired estimates from the downloaded data. Select the
columns for the percent estimate and the associated margins of error for
children living below the federal poverty line and hide or delete the other
columns. To make the results easier to read, you
may want to sort the data from lowest poverty rate to highest poverty rate.
Then copy and paste the Geography Area Names into the Statistical Testing
for Multiple Estimates tab—in the column titled “Label.”
Select and copy the child poverty estimates and associated margins of
error from your downloaded data. Then paste the data into the “Estimates” and
“Margin of Error” columns. The statistical testing results display in
the table’s far right columns. In this example, they show that the child
poverty rates in Butler, Chester, and Montgomery counties are not
statistically different from the rate in Bucks County.
Continuing across the row for Bucks County, the remaining cells display a
“Yes,” which means the child poverty rate in Bucks County is statistically
different than the rate in Pennsylvania’s remaining counties at the 90%
confidence level. Refer to the Census Bureau’s written
tutorial on the Statistical Testing Tool for additional guidance.