In this video, I want to look at loading JSON data not
from a local file--
like a file like birds.json that's right there
in your project--
but from a URL.
And this is one step along the way
to querying an API, to saying like, hey, weather map thing,
give me some weather data.
Or, hey, New York Times, give me some New York Times data.
But I want to start simpler than that.
So, first of all, this is actually really, really easy,
in the sense that all I need to do
is replace birds.json with a URL that gives me JSON data.
But it brings up a question about how
you work with loading data beyond just this preload thing.
So the preload idea was super convenient.
It guarantees that all the data is loaded
before the program starts.
So you've got all the data.
You could do anything you want with it
as the program's running.
But this breaks down very quickly and very easily.
So, for example, here's a scenario
where you want to have some sort of weather visualization,
like you're going to show the sun if it's sunny out,
or some rain if it's raining.
And you have a button that just says Load Data.
So every time the user presses that button,
it should get the most recent data, the current weather.
So you can't use preload for that
because you can't preload the future.
Boy, wouldn't that be great?
So when you invent the whole time travel thing,
you can use preload.
Like preload time-- maybe that's how we get time--
I don't know.
But you can see how this doesn't work, right?
You can't preload the data.
You've got to load the data while the program is running.
Certainly, you could also make the same case
of, as a program is running, you just
want to continuously-- like if you're trying to visualize
financial data and it's just changing every three
seconds, in your draw loop, you might want to query the data,
and query the data, and query the data.
You also don't want the program to stop animating while it's
waiting to load the data.
So you need to load the data in what's
known as an asynchronous manner, meaning when the data can
arrive at another sort of-- in another sort
of like parallel universe of your code that's running.
So your code is trucking along.
It's starting, it's animating, it's looping,
it's doing its stuff.
And at any point there, it could say, hey, load some data
and let me know when the data is loaded.
So how do you do that?
We said previously in preload, var birds equals loadJSON.
This is the synchronous way.
Preload loads the data the program waits and waits
and waits and waits to start.
The asynchronous way is to just simply only
call the loadJSON, function, pass it the path of the data
that you want to load--
birds.json or some URL--
and then what's known as a callback function.
So I'm going to call my callback function gotData.
So this, by the way, is exactly the topic
that I discussed in a previous set of videos
all about looking at HTML elements,
like buttons and sliders, and what
happens if you have a callback when you click the button.
So I'll try to link back to the video about callback
and events.
This is exactly the same idea here.
I now need to write a function to handle
the asynchronous nature of the data arriving later
at some other time, separate from my program that's
running in its animation loop.
So if I say function gotData, this function
is the callback for loading the data.
I'm starting a process.
Hey, go and load some data.
And when you're done, let me know.
The let me know is trigger the code in here.
Now, this isn't magic.
This isn't some default thing that just happens.
This is the nature of how just about everything in JavaScript
works.
But loadJSON and this methodology, this
is specific to the p5 JS library.
Of course, with native JavaScript,
or jQuery, or all these other frameworks,
they all have something similar to this.
So, conceptually, this is a concept
that's across the board in working with data.
But the syntax and this function name is specific to p5 JS.
So I give it the URL.
I give it the callback.
And then this function is triggered
when the data is ready.
Like, the data is ready.
I got to sing again.
That's good.
The question is, where is the data?
Like I've completely lost-- like here,
the data is in this variable called birds.
That's very obvious because I'm saying birds equals loadJSON.
But there's no variable anywhere.
There's just the URL, the loadJSON function,
and the callback.
So the special way that p5 works,
and that a lot of things in JavaScript works,
is if you add a parameter to your function definition,
your callback definition--
I'm calling it data--
p5, behind the scenes, will fill that variable
in the function with the data.
So it's like you said this data variable equals
the result of loadJSON, but it happens asynchronously.
loadJSON from this URL.
When you're done, trigger this function.
When that function's triggered, the data
will be in that variable.
Got it?
I don't know what to do.
That's all I can say about it.
But send me your messages.
And write them and get a carrier pigeon,
or Twitter, or whatever, email.
Contact me, say hi.
I need-- human interaction is a useful and important thing.
So let's make that happen over here.
So, in other words, instead of saying data equals loadJSON,
I want to say-- and I don't want to have preload, actually,
at all.
Like I could actually-- just here, I could say--
actually, let's-- I could say--
I'll just put it in setup.
I'll make this more interesting later.
I could just say loadJSON birds.json.
They're coming for me.
Help, everybody.
My heart is racing a little bit, so maybe somebody
knows that I might need some medical attention.
birds.json is the name of the file.
gotData is going to be the name-- that's a name that I'm
making up for the callback.
I'm going to create that function called gotData.
I'm going to give it a parameter right there.
So now this is the process.
Start the process of loading the data.
When it's loaded, execute this function,
fill this variable with data.
And now I can say, console.log data.
Oh, sorry. println--
I like to use the--
it doesn't matter.
But I just want to see that this works.
And I'm going to run this.
You can see that's happening, and it's
happening super fast because the data is loaded
and then this is triggered.
Great.
It works.
So now let's move on from this birds data set
and let's think about what it means now
to pass a URL into that function and how
we might query the data at a particular moment in time
while the program is running.
So what I want to do is I'll use a simple--
an API that I like to use, which is a URL for JSON data
that I like to use, which tells us how many people
are in space right now.
So I'm going to go grab this URL right here.
I'm going to paste it into the browser
just to see that this works.
And you can see-- look at this.
This URL is giving me JSON data.
What is it giving me?
It's giving me information about the number of people in space.
And you can see it has-- it's an object with the number 6.
And it's got, then, an array of people,
with their craft, the International Space
Station, and their name.
So the number of people currently in space is 6,
and these are all the people and that's
the spaceship that they're on.
Now, ideally, it would be great if this was a data set that
changed very often, so if I hit refresh,
it would say 7, and then 8, and then 4.
But I guess people aren't like going to and from space very
often.
So this isn't necessarily the best demonstration
of why you would need this.
But in some future videos, I'll get to data
that changes and that needs to be refreshed in your program.
So all I need to do now, if this works,
is take this particular URL and paste it right in here.
So you can see that loadJSON works
not just with a local file but with a particular URL.
And now, if I run this--
oh, I don't see anything.
I wonder-- hmm, interesting.
And I didn't even get an error.
I'm going to just add something.
Close your eyes and don't pay attention to what I'm doing.
Yes, it worked.
So this is a good moment to have this discussion.
Security is a thing.
Authenticating with your name and a password
and allowing a browser--
your browser and your machine to talk to your camera
or to talk to another server.
Working with data opens up this big can
of worms of what you're allowed to do
and what you're not allowed to do.
And there comes a time in your life
where you'll find a URL, and that URL,
you know it gives you JSON data because I pasted it
into the browser right there and I see that JSON data
right there.
But I put it in my loadJSON function and nothing happened.
Now, honestly, I didn't even get an error here.
There's a bit of a secret that I think
I'll show you, which is, right up here in the browser window,
there's this settings.
And I can click on that and I can get some more error
messages.
And you can actually see the error message that came.
XMLHTTPRequest cannot load.
There's no Access-Control-Allow-Origin
header is present.
Origin is therefore not allowed access.
This error message is the bane of my existence.
I always forget that it might happen,
and sometimes it doesn't happen and sometimes it does happen.
This has nothing to do with you or me.
This has to do with how the server,
the place that the URL address is configured.
And this URL does not necessarily just
allow you by default to grab the data.
However, there is something called JSONP.
The P stands for Padding.
I like to think of it like a safety net.
It's like I'm in a room with padded walls
and I won't hurt myself asking for the JSON data.
And I feel like that would be an interesting topic
for another video, to delve into that more deeply.
So, sometimes, if you're not allowed access
in the sort of traditional way, you
can use this thing called JSONP to get the data.
And you know that one of the ways that I do
is I put the URL in loadJSON, and when it doesn't work I just
add JSONP and I hope it works.
And, usually, it's going to--
I tried first without it.
If it doesn't work, I try it with it.
You can write your--
I'll maybe include a link to more information about JSONP
in the description's video.
So what do I mean by adding it?
So it's an optional third argument
to the loadJSON function.
So the loadJSON function requires a URL and a callback.
And, optionally, you can add a third argument,
which is the string jsonp, which allows
you to get past this access origin control
crazy error thing that sometimes comes up,
where the server's not allowing you to get clean JSON.
You need JSON's little padding.
There's no padding over here.
Hurt a little bit.
I'm a very sensitive and fragile person.
So anyway.
So, fortunately, if I type jsonp as that third argument--
it's good that that happened because it's
going to happen to you-- and I run it,
you can see that the data is now arriving there.
I'm seeing it.
So let's do something with this data
to finish of this particular video, that's
already 11 minutes long.
So if I look at this data, one thing I might like to do
is just draw a circle for every single person that is--
I'll draw a circle for every single person
that's in space right now.
So, interestingly enough, if I know
that the data is available in the gotData function,
that's a place where I could do some drawing.
So, for example, I could say for var i equals 0,
I is less than data.number--
that's the number 6--
i plus plus.
And I could say ellipse at random width, random height,
16, 16.
I could say fill 255.
This is going to be the least interesting data visualization
anyone has ever made in their entire life.
But we can now see-- and I could say createCanvas 200, 200.
I can now see that when I run this,
I now get this lovely canvas with six circles on it.
So this is-- look.
I did data visualization.
I loaded a URL.
I gave it a callback.
I even used JSONP for some padding.
Wait, I have my padding song.
But that's different.
That's like CSS.
Anyway.
I need a separate video which s just the padding song.
And then I pulled out the number.
I made a loop and I drew circles based
on how many people in space.
And if I could just keep refreshing this,
I'll get it in a different location.
But if somebody would just leave space right now,
then you would see five circles instead of six.
So this is the basic idea.
I do think, however, there are a couple other pieces
of this that I want to demonstrate,
although you could stop watching now if you want.
You could stop watching at any time if you want.
So one that I did, it was kind of nice.
Like, this is just static.
I don't need anything to animate here,
so I just put the drawing stuff in gotData.
But let's say what you actually wanted to do
was have a particular animation that's going on.
So you want your-- like, for example,
I want the background--
I want to have--
this isn't going to be interesting,
but let's say I want to have this line that
goes across the screen from x to 0, x to height.
x equals x plus 1.
And if x is greater than width, x equals 0.
And let's say stroke 255.
So, in other words, I want the circles
to appear with this animation.
So if I were to put the drawing back in gotData--
I know I'm doing this kind of quickly
and scrolling around like a madman here on the screen--
this is not going to work.
Because, remember, draw is looping
to always draw that animation.
gotData is an event that happens just once when
the data is ready.
So that happened and it drew those circles
but it immediately got overwritten by draw.
So there's an easy solution to this problem, you might think,
which is, OK, just take the circles
and put them here in draw.
So now what's-- so there's a lot of problems with this.
Let me run this.
First of all, it's not working.
Why is it not working?
Now, hopefully, I probably have some error message
which we could see.
Uncaught ReferenceError, data is not defined.
Well of course it's not defined.
I can't use this variable data down here and draw.
gotData is a callback.
That variable data is only available in that function.
So one thing I could do is I could say var--
I'm going to say spaceData.
I could make up a global variable and I could say,
spaceData equals data.
So this is now--
I need a way of--
this a sort of simplistic way of doing it but it will work.
I need a way of connecting the data that
came in to something that's happening in draw,
and a quick way of doing that is just have a global variable.
So I, one, ask for the data.
When the data comes in, I immediately
store it in some other variable, a variable
that I can access in draw.
So now if I run this-- and you can see me accessing it
there in draw.
If I run this again--
ah, now what's happening?
Oh, sorry.
I have to change this to spaceData.
Ah, cannot read property number of undefined.
Oh, what happened?
Ah, what's not working?
I know what's not working.
I'm pausing to let you think about it for a second.
Asynchronicity.
If that's even a word, I'm not sure, but this is the issue.
Remember, how are all of these-- what's the sequence here?
Let me try to get everything here on the screen.
Let's just move this animation for a second
below here so we can see the pieces that matter.
There's setup, where I'm asking to load the data.
There's gotData, where I'm retrieving the data.
And there is draw, where I am using the data.
What's the order of the program?
setup starts first.
Make the canvas.
The canvas is there.
Load the data.
That's a process that's now happening in the background.
Let's go to draw.
Start drawing, draw with spaceData.
The data's not loaded yet.
This is going to take a while.
You've got to query the URL, make some connections
across the internet, get the data back.
So spaceData is not anything until this happens, but draw
has already started.
So there is a nice way of dealing with this
and it has to do with a nice trick that you can do
in JavaScript, which is using a Boolean test, an if
statement, which is a concept you're probably
quite familiar with.
We usually think of a Boolean expression
as something that evaluates to true or false.
Like, if 7 is greater than 5, this is absolutely true.
But what does it mean for me to say if spaceData?
Now, one of the nice things about JavaScript is it just--
all it ever wants to do is to have things be true or false.
Like a number, it can be evaluated to true or false.
Like if it's 0, it's false.
If it's any other number, they think it's true.
A string probably could be evaluated.
If it's the string false, that's probably
going to evaluate to false.
So what is actually the value of spaceData
before the data comes in?
It's undefined.
If you have a variable that's not initialized--
like up here, spaceData is not initialized-- its initial value
is undefined.
And undefined evaluates to false.
So spaceData-- and defined, like an object,
just automatically evaluates to true.
So you can basically say if spaceData
as a way of saying if spaceData exists.
So spaceData, it doesn't exist until the data has actually
come in.
So if I just add that-- whoops.
Oh, I was in the wrong view.
You're going to have to live with that.
I can write here--
I can say if spaceData, then do this.
So now, even though draw starts looping, this part of draw
won't actually happen until spaceData gets built here.
And now we can see this works.
Now, of course, I'm drawing them at random locations.
Here's actually a really quick trick for me
to get that to stop.
I could just say randomSeed.
It's kind of a topic for another video,
but randomSeed is a function that seeds the random number
generator so you always get the same exact sequence
of random numbers.
And that's going to work for me here,
at least just in a sort of quick and dirty way.
So you can see I now have an animation and a visualization
at the same time.
I'm not drawing the part with the data
until the data has come in.
So I was very long-winded in this particular video.
I could have done this in half the time.
But what I would suggest to you right now, if you're
looking for an exercise to do, is go to this particular data
source--
I'll link to it below--
and see if you can at least add now, add the people.
Like what could you add?
The craft and the name as text next to the circles
or something.
You're going to use the rest of the data here
also and quickly get it to work is this asynchronous callback
way.
And I think now I've actually covered enough
that the next step it can really be
is, what if you've got an API, like the New York Times
or Instagram, that's serving up data?
How do you query that API with loadJSON and do stuff that way?
Boy, this is a 20-minute video.
Oh, I don't know about this.