In the previous slide, the outer tier was the individual usual 90% confidence
intervals.
Now with the extra Bonferroni correction the inner tier is the usual confidence intervals
and the outer tier is the ones that have been both Goldstein-Healy corrected and Bonferroni
corrected.
So it's quite a mouthful, but it does present, again, another way presenting a lot of information
on one graph that lets you make at least approximate inferences in a fairly rigorous way.
I gather that most people are fairly familiar with R and I've sent Joe a copy of the R Code
used to make every graph in this plot.
So I'll show snippets of that R Code just to explain the key ideas of what's going
on.
But if you'd like to delve into it more deeply, Joe will be able to share that with
you, so you can try it on your own computer at your own leisure and feel free to ask me
afterwards or follow up.
I'll have my email on the last slide if there are questions about details.
Some of these will be a little verbose, I wanted to show the kind of data set structure
that we need in order to make the Package work.
So here's an example of the kind of data set that we're starting with.
Here's a data frame I’ve already created called South data.
The data for mean travel time to work for the southern states.
And I'm just looking at the 11th row, which happens to be Delaware.
Its - we have its estimated rank, the state name, the abbreviation, the estimate itself
and the standard error.
And one thing that is nice to do for the sake of attractive printing of these tables and
we have the table off to the side from the graph is to make sure that the estimates are
all formatted to have the same number of significant digits.
So here although it's really 25.30, it's being represented as 25.3 and we want
to add that padding of the extra zero.
So this code is essentially just using some built-in R functions.
A function called Format C, which makes surehttps://www.youtube.com/timedtext_cs_queue that everything has two decimal places past
the decimal.
And then this one for the standard error, since
all these standard errors are between zero
and one, we're just cutting off the leading zero here.
It's not necessary but it looks a little nicer, so I wanted to include that code in case someone
wants to follow up later and - and try to use it themselves.
And so what that ends up giving us is a data set where for a couple of rows and a couple
of the columns, we have again the rank, the state
name, but now the Estimate.Print is the version
that has 0.30 in it instead of just 0.3 and SE.Print is the version without the leading
zeros.
Here's the core of what you would actually do when you're ready to make the plot.
So the function that is built into our R Package is called RankPlotWithTable, so it's going
to give you that ranking plot of whichever type you choose, plus the table on the side
that gives the estimates and their standard errors.
And so for that we need two lists of parameters.
We're gonna create a table parameter list and a plot parameter list.
The table parameter list needs to know the estimated rank to print, the names for each
state, the estimates to print. So here's the Estimate.Print version of that and then the
SE, the standard error to print next to everything.
For the plot data we're going to do also estimates and standard errors, but we're going to give
it different versions here; we're going to give it the numerical versions instead of
the ones that we saved as pretty looking strings.
We're going to give it the names as abbreviations instead of the full state names, because those
would be a little too long to print on the plot.
We can choose a reference state; so a reference name refName here is LA, we're going to use
Louisiana as the reference.
We give it the confidence level we want and then plot type here's where you choose which
of those plots you're going to actually make.
Are we making the shaded columns plot or the comparison intervals?
Here we're going to make a difference plot.
So this is the confidence intervals for differences with Louisiana.
And then we just give this list of parameters and this list of parameters to our RankPlotWithTable
function and we get the plot directly out of that.
So that's a fairly basic, straightforward example.
Once you have the data built in, there's a couple things that are not super pretty about
this, and so on the next slide I'll show you how we can adjust the parameters to correct
these.
Here we see District of Columbia is almost running into the ranks, so we might want to
adjust the spacing between these columns a little bit so that long names can fit.
The legend here is overlapping some of the data. We might move it from the top right
to
the bottom left instead, so that it's not covering
things up.
Here, the label Theta K minus Theta K Star, showing that we're interested in differences
from Theta K Star.
That's running into the x-axis labels as well.
So we're going to move the Thetas down a little bit further.
So the code for doing that is, we could go back to tableParList and plotParList where
we defined them, redefine them again with these extra commands, or we could use the
dollar
sign to access the list and just add another piece to that list, or overwrite a piece of
that list saying column two should end at about two-thirds of the way through the space
available for the column.
So that's just going to move the right edge of that state names column a little bit over.
For plotParList$legendX - that's saying we're going to add this legendX parameter to this
-
we want it at the bottom right. The way that R specifies legends is a bit weird and it's
X value is the name it gives for these kinds of words of bottom right to top left whatever,
so I would just stay with that in case people are already familiar with using legend and
base R graphs.
And then Theta line is going to move the Theta symbol down by a few lines.
And then we just give those parameters back to the plot.
And now we have a slightly cleaner looking plot.
We see that the column here has been moved over to the right, Theta has been moved down,
the legend has been moved over.
So setting these things up is pretty straightforward.
Adjusting the plot parameters is straightforward as well.
The Package has documentation for each command that explains what are
parameters that you can change.
If we want to change the plot type itself, remember we have the difference plots.
Say we want to show those comparison intervals instead.
All I do is say plotType = “comparison” instead.
This range factor is just going to shrink the plot for some reason by the default.
On this particular data set it gave access limits that were a little too wide - too much
white space - so I shrank it down a little bit.
And now here's our comparison intervals plot for the same data sets.
So you can represent the same data set several different ways very straightforwardly with
only minor modifications.
The next two examples I have are from slightly different data sets.
This is for another American Community Survey data set where all - I can walk through the
process of reading in the .CSV file.
So if you actually download the comma separated values file from American FactFinder, how
do you read that into R and then make the plots?
And I have another example with using SAIPE data of calling the Census API so instead
of downloading a file you call the API on the website and that brings the data into
R. We have a comma-separated value file from the 2016 ACS.
This is a different Ranking Table, the percent of occupied housing units that are owner occupied.
So among the housing units what percent are owner occupied instead of, I guess, renter
occupied?
We use the read.CSV function in R to read the .CSV data set in.
There's a couple of things we add in this case “skip = 1” to skip the first row
because there's extra, sort of, text data that the ACS sprinkles in that isn't
part of the numerical data set itself.
So we're gonna skip that.
“stringsAsFactors = FALSE” is ensuring that things will represent as either numerical
or as character rather than being coerced into R’s factor format which is not right
for our purposes here.
I'm going to change the column names of what's left.
So I'm gonna change it from the default names in the data set to the FIPS code, the state,
the estimate, and the margin of error.
Because it comes as margins of error instead of standard errors, I'm going to convert those
to standard errors. So we're going to take the margin of error divided by the appropriate
normal z-score and round that.
I have another data set travel time 2011 that’s part of our R Package, that one has all the
state names and their FIPS codes in it, so that's handy to just call that data set and
merge it together with this one by FIPS code so that we can have the abbreviations
and the regions as well.
So in this particular .CSV it has the state names Maryland, Virginia, and so forth but
it doesn't have the abbreviations MD and VA.
It doesn't have the Census region.
So we're going to merge that in as well.
And then I'm going to subset it to be just the northeast region states for this particular
example.
And then the rest of it is much like what we saw before for the previous example.
Now that the data set’s actually in R and it has the right column names and so forth,
we're going to order it; we're gonna order the rows of this data set by the values of
the estimates, so that their ranked appropriately.
I'm gonna remove the default row names because they give you the wrong ordering.
I'm going to format the estimates and the standard errors for printing.
I'm going to add a rank - a column - so that we have - you know - what the ranks of each
state
are and then we see that it's formatted the way that we want it.
So this is the same kind of format structure that we had for the previous example.
And then it's very much simply plugging in the right variable names into the right places.
So again we have the table parameter list and the plot parameter list.
We give it the ranks, the state names, the estimates to print, and the SEs to print.
For the plot parameter list we do the same thing but the numerical values rather than
the printable character values.
In this case I'm going to choose Pennsylvania as the reference state and columns as the
type of plot and call our plotting function.
And here we have the shaded columns plot for the northeastern states with Pennsylvania
chosen as a reference point.
So you can see that most of these are either substantially different or measure with adequate
precision, so that the significance of those differences is there but Pennsylvania versus
Vermont are not significantly different.
Vermont is not significantly different from a few of the other states and so forth.
And here's that feature that Eric noted earlier is that for example Maine and Vermont are
not different statistically significantly in this plot even though New Hampshire is
between them and it does differ from Maine.
And the last example is using the Census API.
So this is the smaller area Income and Poverty Estimates - the SAIPE Program.
And we have their 2016 poverty rates and the margins of errors - margins of error for those
poverty rates for each county in Maine now.
So we don't have to look at state-level data; we can look at county level, we can look at
MSAs, we can look at demographic groups instead of geographic groups; there's no reason to
subset only by geographic populations.
The variable here is percent of related children aged 5 to 17 in families in poverty.
And the whole API call is quite long so it won't show up here, unfortunately.
But if you are interested, the Census API documentation is very good and you can see
how to replicate this and in the R Code you'll see the exact API call here.
But what this does is essentially go to the Census data set website - the API website
- ask for exactly this data, these variables for these geographic levels - and then bring
it back.
The structure of this kind of data is not a .CSV like we had in the previous example.
it's what's called JSON.
I don't recall what JSON stands for.
Does anyone happen to know?
JavaScript something-something, I think.
(JavaScript Object Notation)
However, this example will process things from the Census API, essentially getting rid
of some of the junk that comes with it that we don't need for our purposes and making
sure that the right columns are numeric instead of strings and giving it the right variable
names.
And so you get a data set that looks like this.
You have the county in Maine, you have the estimate, and you have its standard error.
And so from this, we're gonna do the same thing we had before where we want to reformat
the estimates for printing.
In this case I don't know of any standard set of abbreviations for the county names
in Maine, so I'm just going to take the first three letters of each county name and make
that the abbreviation.
I'm going to add the ranks, I'm gonna make sure it's ordered correctly, and now we have
- here's the county, here's the estimate, here's its rank, the abbreviation for York
County is YOR and so forth.
Just as before we can still - if we assume that the correlations are not something to
worry about in this particular example, for the sake of assumption, we can use the same
kind of commands that we had before.
Make the table parameter list, the plot parameter list, and feed it to this function.
The difference here is that before we were looking at states and so the table said the
word "states" as it lists the state names.
Here I'm telling it the place type is county so that it will say the word county instead.
I'm adjusting the column spacing a little bit again.
I'm making it an individual plot type.
And we're going to have this Goldstein-Healy corrected plot.
So we're gonna say GH = TRUE, make the Goldstein-Healy correction and then adjust the x-axis limits
and adjust the legend position.
So there's our Goldstein-Healy plot; you can see that there's a lot of overlap between
these estimates for counties in Maine, but if you were just coming at this and had already
had, you know, Somerset County and Cumberland County in advance in mind, you would see that
they don't overlap.
So for this Goldstein-Healy approach with the right wording you could say that there's
a significance of that difference, whereas some of the other ones that do overlap you
could not.
And again we can make that the two-tiered version of this.
Where to do that we're going to say tiers = 2.
It will automatically figure out which tier is wider and which tier is narrower and put
a legend on the plot so that it's clear whether the marginal or the corrected ones are the
inside and the outside tiers.
I'm going to change the text padding just to move the text out a little bit further
from the estimates, so that the extra width of these outer tiers doesn't run into
the text.
And then we have our R Plot like so.
And finally this had no Bonferroni correction; if you want to do the demi-Bonferroni correction,
you add another parameter: Bonferroni = “demi”.
By default that's gonna have Bonferoni = “none,” but it will choose - if you choose demi it
will add that correction to it as well.
And again just changing the text padding moving it out even further changing the x-axis limits.
And there are the demi-Bonferroni correction versions of this plot.
The Package itself - "The Ranking Project" - is
the name of the Package.
It's on CRAN, the Central Comprehensive R Archive Network.
It's the main repository for R Packages.
It's been tested and approved on varieties of systems.
I don't know if the Census Bureau has approved it for internal use.
That is a question I'm not sure about.
But when we published this in March 2017, the
RStudio blog chose it as one of their top
40 new Packages.
It’s had almost 1,800 downloads since then.
You can install it just using the install Packages command in R. We have a couple of
vignettes, which are essentially help files with illustration of how to use the command
for a particular data set and what that looks like.
If you are interested in contributing, then on Github we have the development version.
There's a couple of open issues I still need to address.
There is a research report here from 2014.
We have the American Statistician article that's - the proofs have been sent out so
it should be coming out sometime soon.
My email is my full name jerzywieczorek@gmail.com or my website is civilstat.com, if that's
easier to remember.
And I look forward to talking with you about this further.
Thank you very much.