Imagine you're walking in the forest
and you encounter an object.
What questions come through your mind?
First of all, what if this?
Is this a stick?
Obviously it's a stick.
Secondly, hmm, what can I do with it?
If you are a child like this little fellow,
poking somebody with it may sound like a great idea.
Or if you're older, you can probably
use the stick as a walking aid.
Find some more sticks and maybe you
can use them to build a shelter.
Or you can use them to keep warm on cold winter nights.
The stick is an example of an entity.
And the many uses of it are called affordances.
This video is about schema.org actions.
And you can think of them as entities plus affordances.
What's so exciting about actions?
I think that actions can enable some of the experiences
I only read about in science fiction books growing up.
But let's start at the beginning.
Hi, I'm Jarek Wilkiewicz.
If you watched the previous two videos in this series,
you know that my DJ buddy Shawn and I built a system
to make the data from our music store
machine readable and available in the Knowledge Graph.
We use schema.org and JSON-LD so that when the user says,
OK Google, and asks to listen to a music artist we carry,
our mobile application will be ready to fulfill
users' requests.
Shawn and I published the information
about the artists we carry in our store.
We used JSON-LD markup and embedded it
in artist web pages on our site.
We associated our mobile music app with the artists
we carry so that our app could be triggered by the OK Google
voice command.
Remember my story about the stick?
As I told you earlier, the many useful and not so useful
things that can be done with it are called affordances.
And the affordance is the quality
of an object, or an environment, which
allows an individual to perform an action.
What does this have to do with schema.org actions?
Let me explain.
If you take a look at this JSON-LD markup,
you will notice that the potentialAction property
identifies a ListenAction affordance.
This means that the music artist from our store
can be listened to using our mobile app.
Let's revisit this simple architecture diagram
from our first video.
The JSON-LD markup from our store site
is processed by a crawler and then saved
in our small Knowledge Graph running in an open source graph
database called Cayley.
The crawler parses the markup from our site
and transforms it into triples.
Triples are data representation well-suited for graphs.
Once we have the triples, we can load them into Cayley.
As you can see in this visualization,
green nodes represent the potentialAction.
Much like other structured data about our music store entities,
we have loaded the potentialAction triples
into Cayley.
We have also loaded a music data set from Freebase.
Now, Cayley can answer handy questions such as,
how can I listen to Nerdcore?
As you can see, the query produces an Android app
as the result.
The app can be invoked to play the music we want.
To show you how this all works, we
will use our Android action simulator and our sample music
app.
Let's try to listen to Nerdcore.
With the simulator, we can issue the query for the genre.
The full query expression sent to Cayley
is similar to the one I showed you earlier.
This should trigger our sample music app.
Oops!
Or maybe not.
Why isn't this working?
Let's look at the AndroidManifest
of our sample music app.
Oh, we forgot to include the intent filter.
We may need to fix that.
MEDIA_PLAY_FROM_SEARCH handling was missing from our manifest.
Now we can try again.
Sure enough, the music starts playing.
[MUSIC PLAYING]
All right.
So far, we've covered how a mobile app can be triggered
in our response to a voice command,
and I showed you the JSON-LD markup which enables this.
What about the web?
Schema.org actions allows us to specify web actions as well.
For example, similarly to the mobile app,
we can indicate that the artist from our store
can be listened to on the web.
Note that the HTTP link identifies
the URL where the user can listen to the artist.
Besides the ListenAction, what other types of affordances
can we specify using schema.org?
A lot of them.
Not all of them apply to all entities, though.
For example, ListenAction may not
work very well with a stick.
OK, let's look at another affordance
called the SearchAction.
To illustrate how the SearchAction works,
we will use our web simulator to help us invoke actions.
Let's fire it up.
All right, let's point the simulator at our music site.
As you can tell, the simulator detected
we have a SearchAction available.
How did it do that?
It's quite simple.
As you may have guessed by now, we
used schema.org actions markup on the main page of our store.
Here's what the markup looks like.
Let's highlight a few things.
The potentialAction target is the HTTP urlTemplate.
Remember that in the mobile example,
we were using an Android deep link.
The urlTemplate is defined by RFC 6570.
And while that's a very awesome spec,
we won't get into the details here.
Suffice it to say, that the query parameter is properly
substituted by the client, or, in our case, the simulator.
Let's now invoke the SearchAction.
As expected, we found Dual Core.
Since the simulator also detected the ListenAction
in the JSON-LD response, we can invoke it.
This takes us to our jarekandshawnmusic.com site
where users can listen to Dual Core.
The target URL the simulator uses
is defined in the JSON-LD response.
Now, we can hit the Play button and enjoy.
[MUSIC PLAYING]
All right.
Let's think about the search use case for a second.
We use schema.org actions to describe the search endpoint.
The simulator sent the GET request
to the URL defined by our markup with a query expression.
Note that we have associated a RESTful API affordance
with an entity serialized as a JSON-LD document.
This is an example of a Hypermedia API.
The Hypermedia API topic is quite fascinating in itself,
but we won't get into it today.
If you would like to learn more about Hypermedia APIs,
click the link on your screen.
As you can tell by now, schema.org action spec
is quite expressive, and I have barely
scratched the surface in this video.
I am very excited about the potential
of schema.org actions.
I think it's a step towards making
the dream I shared with you in the first video a reality.
OK, what do we do next?
Let's learn more and clone the repository
that we have shared to try it out.
You can also get more information from
developers.google.com/schemas.
Last but not least, if you find this technology interesting,
please let me know.
I would love to hear from you.
Thanks and have a great day.