FELIPE HOFFA: It's 4:00 PM in New York City,
and it's impossible to get a cab.
Can we use data to fix this?
I'm Felipe Hoffa, a developer advocate for Google.
I am here to attend a hackathon hosted by NYU
and the Rubin Center For Transportation, and the TLC--
Taxi & Limosine Commission of New York--
where top data scientist and researchers will
use data to solve this problem.
How?
Well, we've got data.
We got all the taxi trips.
Chris Whong, how do you manage to get this data?
CHRIS WHONG: The Taxi & Limosine Commission
has an internal data analysis team.
And they released a tweet with a visual of available taxis
by time of day.
And my response, as an open data advocate, was,
is the raw data available so that everyone
can do this kind of analysis?
And the response on Twitter was, it's not
available for download, but it is
available from a Freedom of Information Law request.
I sent in the form.
And within a couple hours, I got a response.
So a couple days later, I had 175 million taxi trips
in CSV format on a hard drive.
FELIPE HOFFA: 175 million taxi trips.
And then you managed to share this data with the world?
CHRIS WHONG: People started coming to me,
to meetups that I went to to actually get the data.
And there were so many people messaging me
that, finally, I was like, it's time to put this online.
And within 24 hours, it was all over Google BigQuery.
There was a really awesome reddit thread
of people doing simple analysis from BigQuery,
and just slicing it up, and figuring out
what's in the data.
FELIPE HOFFA: So all the New York City taxi trip for 2013,
and now shared with BigQuery, and anyone can easily
analyze them.
So stay with me to see what happens tomorrow,
what solutions we find.
And remember, at the end of the day, someone will be fired.
Wrong show, sorry.
Traffic!
We are here at NYU Rudin Center for Transportation.
And today's a historical day.
It's the first time the TLC Taxi & Limousine
Commission for New York City is meeting and working
with citizens to take a look at the data
that the TLC is sharing with us today.
We'll try to come up with solutions,
answers, discoveries, and I can't
wait to see what happens today.
RODNEY STILES: Today, we shared all of 2014's trip
data with the additional information added.
And the way that we did it is we loaded it
onto Google Cloud Storage and loaded it
into BigQuery, which is how all the participants accessed
the information today.
FELIPE HOFFA: Hello.
I'm here with Ben Wellington, a visiting professor at Pratt
and also the data scientist behind iPod New York.
So when you were looking at the--
BEN WELLINGTON: It's a parking ticket data set up.
And I wanted to find out how much revenue in parking tickets
every fire hydrant in New York City was making.
So first, I mapped out the top 250 grossing fire hydrants
in New York City, in terms of ticket revenue.
And these were the two, by the way, top grossing hydrants
in the entire city.
And if you look at the amount of revenue they were generating,
they were generating over $55,000 a year
among the two hydrants in parking ticket
revenue for the last, at least, five years.
It turns out that it looked like a bike lane between the hydrant
and the parking spot.
So the car would pull in and say to themself, well,
I'm not next to the hydrant.
There's a bike lane between me and the hydrant.
A month later, they actually repainted the parking spots
by adding these stripes here, so they prevent future people
from making this mistake.
So this is a time when open data has
changed the infrastructure of New York City in a small way.
But imagine this happening locally
in everyone's neighborhood.
If every person got together and found
some of the issues in their own neighborhoods,
put their minds together, we can all
start to make changes very locally.
JO RAUSEN: I think that collaboration with citizens
is really important, because again,
we come from the policy from what we know,
working with a lot of industry stakeholders
that are very, very vocal.
So it's really great, when we finally
get citizen voices in here.
These are things that actually matter to me,
and that are important to me, and can
help us understand better what their problems are,
and just be a part of that dialogue.
JEFF FERZOCO: I am excited to be here today
to help the TLC better understand
exactly what is happening for taxis between 4:00 and 6:00 PM.
Ultimately, what we're going to do
is take the data from BigQuery, put it into redash,
download it, bring it into [INAUDIBLE]
and make a map of where people get picked up.
See where the cabs are spending the most time,
and look at them specifically between 4:00 and 6:00 PM,
because between 4:00 and 6:00 PM,
cabs shift over from one driver to another.
And if a driver doesn't have to do that shift,
it can take advantage of the extra amount of people that
need service during rush hour.
Data that is open means that it's accessible by anyone.
And having it be open means that people can look at it.
They can come up with their own ideas.
They can start making decisions that make cities better
places to live.
And having open data be open, and free, and available
makes that all happen a lot faster.
SARAH KAUFMAN: I think the best part of today
was that we've been focusing so much on the data leading up
to this event.
We've been looking at the data nonstop.
And today, we've had five taxi drivers here talking
about the issues that they deal with every day.
You can look at data all day long,
but you may not be able to see it from the other side.
ALLAN J. FROMBERG: It wasn't just numbers.
It wasn't just mathematics in ones and zeroes.
There was very much a human factor present here.
And the fact that we were taking that approach to data,
adding the human factor in with the data,
and putting it all in a bowl, and it was better
than the sum of its parts.
MALE SPEAKER: Right now, the agency
is really taking a hard look at our rules,
and figuring out what makes sense today
and what doesn't with all this emerging technology.
And this is one piece in the puzzle.
This is another part of that feedback loop
to take this information and create a progressive policy,
and so I think this will definitely
be a part that process.
BEN WELLINGTON: Our agencies do great work,
but they have limited resources.
So when they embrace the open data movement,
it allows citizens to give back and find
some of the maybe issues or problems,
and suggestions of how we can run our cities better.
SARAH KAUFMAN: With hackathons next year,
I hope that we'll see these kind of micro improvements using
civic hacking to resolve issues.
We have a lot of smart New Yorkers,
a lot of people who can look at data, look at numbers,
and understand where the issues are and suggest policy changes
that we might not have seen otherwise.
We had access to more than 160 million taxi trips taken
during 2014, and a glimpse into the for the for-hire vehicles'
data, like Uber and Lyft, confirming
that there are 20% less taxi drivers on the road
when we most need them.
We spent the day going deeper into the data.
Noel Hidalgo and David Marulli looked
for patterns that may reveal the best shift change locations.
Arlene Ducao and Chris Willard explored the potential
of having three shifts instead of the traditional two.
Jeff Ferzoco, Ben Wellington, and [? Carson Kim ?] took
Uber's data to compare their diver's behavior with
traditional taxis'.
And Damon Wischik went deeper into why
we have the current shifts, and also,
surfaced interesting behaviors about tipping.
Why would people tip more after a New York Knicks loss?
There's a lot more to discover, and it's your turn to dig in.
Thank you for joining us.
This was an amazing day to see how citizens, taxi drivers,
academia, government can come up together and look for solutions
using data, using open data, got and the right tools to do it.
That's it for today.
Make sure to follow us for the next Big Data Story.
I'm Felipe Hoffa.
Goodbye.