Hello.
In this video, I'm going to look at The New York Times
API, which is an API that you can query to get information
about articles in The New York Times, shockingly.
Quite a surprise.
Yeah.
So again, I'm not suggesting that this
is the be all, end all of APIs to use,
but this is a nice case study of finding an API that's
out there, how do you navigate its documentation,
how do you figure out how to form an API query
and look at that data and make something happen
on the screen in a web page using JavaScript, HTML,
and CSS.
And this thing over here above my head--
I just would rather have that as a hat--
called p5 GS.
So the first thing, once again, as in the previous video
where I looked at Wordnik.
The first thing is I can spend a lot of time
without even writing any code or looking at any code
because the first thing I need to do
is just figure out, like, huh?
There's an API?
Like what?
So one thing, you'll see, I'm already here.
But I'm at developer.nytimes.com.
So once again, you're at the mercy of the API.
How good is the documentation?
Are there examples?
Is there an online tool that helps you navigate the API?
And again, this is a case where there is a good one.
And this is that question, like you went to The New York Times
website, and you found all this data, and you want to use it.
Is there an easy path of that, or do you
have to take some sort of side road?
In this particular video, I'm showing you the easy path.
And in another video, we'll look at some side roads
to how you might scrape or parse data that isn't presented
in a API, json friendly format.
So all of that aside, here I am.
Welcome.
Get started.
This is nice, friendly language.
It makes me happy that someone is welcoming me to the page,
and I could get started.
So the first thing you're going to need to do, by the way
if you're doing this on your own, is request an API key.
So again, you've got to sign up for an account, get an API key.
I don't believe-- actually I don't know this one.
I have no idea what I'm talking about all
of a sudden, which is most of the time.
But I don't know whether there's any threshold
of querying this API where you suddenly
have to pay for its use.
But certainly, what I'm going to show you today,
getting a little bit of data here and there,
is something you can do for free.
This is a great thing.
This is something that if you see is good news for you.
You're looking for an API tool, meaning
you can figure out how the API works
through an interface in the browser.
It's all the same exact thing with Wordnik.
So I could click around and look at various things,
available APIs, but I'm going to go here directly
to that API tool.
And here you can see API console.
Actually, before I go that, let me click to available APIs.
So you can see here, there's a lot of APIs.
There's event listings and geographic data, most popular
API, movie reviews.
There's lots of different sub-APIs, different paths
within the larger API.
I think I'm going to stick with, for simplicity, just
this article search API.
So interesting, this is a nice data source
because it actually goes back to 1851.
So I can query The New York Times to ask for
or I can search.
I can say, give me all the articles
that have the word "rainbow" in them or the word sea-bow.
There's a such thing as a see-bow
as I learned in a previous video or porcupine, whatever.
Pick your word of choice.
You can find the articles.
So I'm want to start with that one.
And I could click on it.
And here's some documentation of how it all works.
But once again, I'm going to prefer to go back and go
to this API tool.
And here I am.
Hi.
Moment of technical difficulty, but I'm back,
and I am on the API console.
So the first thing that I'm going to do here
is look at how the API console works
to figure out what the query is I want
to make to The New York Times.
So I'm in the article search API,
but, of course, I could change to a different one.
And what I want to do is click here.
This is what's known as endpoint,
being the point of end of the API
where I'm going to get something.
But really, I think endpoint is referring to search
or maybe that's the method.
It doesn't really matter this terminology.
The point is I want to search the API,
and this is the path for doing that.
So I can click here, and I can see now, again,
I have an interface.
And yes, I want the format to come back as json.
That's going to work well for me.
And then q is the search query term.
So what's the query term that I'm
looking for articles that contain that particular word?
So I don't know.
I really should think of something different,
but I'll just keep using rainbow.
By the way-- no, never mind.
And then, there's other things.
I could filter the search.
Oh, there's a begin date and an end date.
So I could restrict only getting articles from like 1957
with the word rainbow in it.
I could sort what I get back by a different order.
So all that stuff is interesting,
but it's also all optional right now.
I'm going to leave it out.
So I encourage you to look through this.
Try it on your own.
Add different parameters to the API search.
But the point is I want to hit this now.
And once again, the most beautiful button
in the internet is the Try It button because that's
going to let me just try the API without having to write
the code to see how it works.
Now one thing here you can see, The New York Times
will often give you some results with a key known as sample-key.
I've overused the sample key.
So I tried this a minute ago, and it's
going to tell me, no, sorry.
You've overused the sample-key.
But I have registered and logged in for my own API key.
So I'm not going to include this process in the video.
If you have trouble, ask a question in the comments.
Maybe I can help you.
But once you've made an account, logged in,
you will get your own key.
And I'm going to take that key, and I'm going to go back here,
and I'm going to paste it in here.
And then I'm going to hit Try It.
And once again, we can see this is exactly what I want.
The API tool formed the API query for me.
And also, we can scroll down.
You can see it's showing me the data.
But this is a bit awkward to look at it here,
so what's a little bit easier for me to do is grab this URL.
Once again, open up a new tab in the browser,
paste it into the browser, and now I
can start to look at the data as it looks in the browser itself.
So I can start now.
So first of all, it might look like this to you.
You might get this result. I have a particular browser
extension which formats json for me nicely.
I demonstrated in an earlier video.
I think if you just Google Chrome extension json
formatter, you'll find one.
But even so, this is a good example of how a lot of APIs
will give you back a ton of data.
So really there's some detective work involved here
in figuring out how to find the data.
So let's say what we want to do is show on screen
the headlines for all of the articles
that contain the word rainbow and maybe even
a short snippet paragraph from those particular articles.
So let's think about how we might do this.
First of all, one thing I often like to do
is start minimizing these.
You can see that really the API is just giving me
back an object with a response property, a status
property, and a copyright property,
which is important to note that we're getting some copyrighted
material here.
So I can unpack this response property.
There's a meta property and a docs property.
Meta probably has some information
about the API query, how many calls I have left,
that sort of thing.
Docs, which has 10 items--
I know this in advance--
that's giving me 10 articles.
So most APIs won't give you back all 10,000 articles
all at once.
They're going to give you 10 at a time.
And there's a way of paging through them,
but that's a bit beyond the scope of this particular video.
But what I want to do is here look at docs,
and docs is an array.
So you can see these are each element of the array.
So each one of these elements of the array
is one particular article.
So in this particular article--
now I get to zoom back out here and look--
multimedia, headline-- there we go-- keywords.
There's so much information-- publication date, byline.
You can see some things are right
there, like snippet, lead paragraph, abstract.
I'm kind of interested in the headline,
which is now in another object.
So I can unpack that and see main, kicker, print headline.
So we can see, I can spend all day here looking into the API.
But I can kind of imagine now if I
want the headline is response.docs
index, 0 for the first one, dot headline.main.
So I weirdly feel like I have that memorized now,
probably because I've used The New York Times
API a bunch of times.
But this is the thing that you're
going to be needing to take notes,
practice, going to make a mistake, try it again.
This is the thing you're going to want to do on your own.
But now, we are ready to go back and start to write code.
We've looked at the API website.
We've looked at the documentation.
We've used the tool.
And we've got a working path to getting data.
And so now I'm going to go back to p5.
And I'm going to say just at the beginning loadJSON,
and I want the API URL--
I'll have that as a variable-- and a callback called gotData.
So I'm going to make a variable, which is the URL,
and I'm going to paste this URL right in here.
Now for expediency here, I'm not going
to go through the steps of breaking that URL into parts.
I've done that several times in several other videos now.
And, in fact, I would say probably the exercise for you
at the end of this video is to have the user input the search
word and then get particular articles back
with that search word.
Right now, I'm just going to get rainbow articles.
But the first thing, what I need to do right now
is write this gotData callback.
And remember, the gotData callback needs an argument.
That's where the data from the API
will fill that variable with the data.
And I can just look at it to make sure it's
working in the console.
I'm not really going to look at in the console,
but I want to just hope that I see something there.
And you can see, there is something there.
So hopefully, we can feel pretty confident
that what's there is exactly the same as here.
So now let's think about this again.
Response.docs index 0 dot headline.main.
Let's work with that.
Data.response.docs index 0 dot headline.main.
Did I dare possibly get that right?
Let's try it again.
Let's run it.
Looks like I got that right.
Let's look at the second article to see what the headline is.
I'm changing that number 0--
"Hollywood Dreams".
That makes sense that you might have a rainbow
in your Hollywood dreams.
My Hollywood dreams consist of YouTube videos
about data and API, which is sort of a strange Hollywood
dream some might say.
I don't know where I'm going with this.
Let's stop now.
Where are we?
Oh, I've only been doing this-- no, no.
I forgot I have two sections here.
Anyway, I'm back.
So blah, blah, blah.
What do I want to do with this?
Well I want to make something appear in the browser.
Now there's a lot of interesting applications here.
And maybe I'll get to some ideas for you
towards the end of this video.
But right now, let's at least make
it loop through and show me every headline and perhaps also
a snippet.
So let's go back to this.
You can see snippet is actually just response.docs index
0 dot snippet.
Snippet being like a short paragraph or a snippet
from the article to give us some more information.
So one thing that happens here is that I have an array--
data.response.docs.
And you know what I think can be useful?
This is a useful technique.
If I just make up a variable called articles and say
articles is data.response.docs,
I'm going to need it every time I try to look for data.
I'm going to say data.response.docs blah, blah,
blah, data.response.docs da, da da, da.
If you remix this, it has kind of like a rhythmic quality
to it-- data.response.docs da, da da, da.
But I'm going to need to do that over and over again,
so it's kind of useful if I just store that in a variable called
articles because now I can say things like what I want to do
is have a for loop.
I goes from 0 to articles.length I plus plus.
And I could say something like--
whoops, let me leave that down here for reference.
I don't need to say data.response.docs headline
main.
I just thing to say articles headline main.
Well, what I need to say is--
and I'll say createP articles index i.headline.main.
So this should now loop through every single object
in that docs array--
each one having a headline object and each headline
having a main being the main headline.
So if I run this now, you're going to see,
here we go, I've got Rainbow international, Painbow Charms--
Painbow?
What's a Painbow?
That sounds terrible.
No, no, no.
No painbows-- rainbows.
East Village Birdman abandoning his nest of CDs.
Anyway, there's some interesting articles about rainbows.
But maybe what I want to do actually is say createElement.
I'm going to make an H1 element for every headline.
And then I'm going to just quickly--
I'm kind of like trying to wrap this up here.
I'm going to make a paragraph element
for every snippet, which would be
articles index i dot snippet.
So again, there's a lot of data there--
articles index i dot headline.main,
articles index i dot snippet make an H1 make a paragraph.
And now when I look at this, we can
see I've now got like my own version of The New York Times
right here of a bunch of articles and little snippets.
And right now, if I wanted to change what it's searching for,
I've got to go into my code into this URL.
I could change it to porcupine, and I could run it again.
And you can see, I've got now some prickly in name
but refined on the table.
I've got some articles about porcupines.
So here's what I would say to you.
First of all, just showing you the beginnings
of what you could do here.
So one thing that I think would be a good exercise
is to make this work but not have to go type into the code
to change the word.
Add a text input field.
Add a button.
When the user enters something in that text input field
and clicks the button, you get the articles
with that search term.
Another thing you might think about doing
is, can you add query string arguments?
The only query argument I have right now is q
equals porcupine.
But if you go back to the tool, there was a begin
and an end in terms of dates.
So could I get all the articles from a particular year?
Could I even have that year be selected by the user?
That's something you might try.
Another thing you might look into,
and I might need to add some link or something in this,
is that The New York Times will give you
the frequency that a word appeared in The New York Times
for a given year.
So you can look at interesting trends
like when did the word computer start
getting used often in articles.
This is a very totally trite example,
but you could think of something like how many times the word
war used versus the word peace.
So anyway, you'll be more thoughtful or creative
about what kinds of comparisons might
you be interested in doing.
But that's something in this article search API.
You can also get something about word frequency.
So there's a lot of possibilities
of things you might explore.
If you make stuff, share them with me.
I would love to hear about it.
And I hope that this was useful to you
in how you might use The New York Times API.
This is me signing off.
WFMXY YouTube ITP hot lights room,
weird place over the window and a computer and--
goodbye.
I'll talk to you.
I don't have a hashtag for you--
#uncomfortablepersonDan.
Goodbye.
Oh wait, this mouse fell asleep, sleeping mouse again.