- Great, okay,
so I'm a peer student at the University of Washington
where I work with Jeffrey Heer.
Today I'm going to be talking
about Idyll, a Markup Language
for Authoring and Publishing
Interactive Articles on the web.
So what I mean when I talk about Interactive Articles
we can just look at a few examples
just to kind of see.
So first, is this essay Explorable Explanations
which was written by Brett Victor.
It kind of, at first glance,
it looks like just your typical blog post,
there's just text maybe some images in aspurce,
but it actually has this additional interactivity
built inside of it.
There's these numbers that the reader can actually scrub
and see how a change to the number effects the text.
There's these interactive graphics,
that these you can play with
and see how updates to the graphics
will effect other widgets on the page.
Victor published this in 2011
and a lot of these ideas
have kind of parmeated into other spaces.
For example,
Distill pub
is a machine learning journal publish by Google.
They're really just taking up this concept
of trying to use a multi-media
and kind of interactivity to make
concepts easier for people to understand and adjust.
So here's seeing a kind of standard technical article
that also has kind of these
interactive elements embedded in it as well.
So, the user can directly manipulate parameters
on these diagrams, causing computations to be,
sort of, re-run in real-time in the browser.
And hopefully the reader can gain a more intuitive
understanding of what these parameters are actually doing
and how they effect computation.
The (mumbles) article is also popular in news-media.
So if for example, the New York Times
publishes a lot of really rich multi-media
articles
The Guardian and Five thirty Eight are also known
for publishing this style of article.
Here we're looking at something that is explaining
why the New York subway system has so many delays currently
and you can see that there's, kind of,
again data visualizations, animations,
interspersed in the text itself.
This is kind of, these are all kind of
common design elements that we see
a lot in this style of article.
Again, here's something, here's a fixed graphic
that you see kind of update in response to readers
scroll input.
The graphic on the right advances kind of
in response to the readers scrolling through the article.
Personally, I think these things are really cool and fun to
to read, but there's actually some,
some sort of more concrete reasons
why we might be interested in making them.
For one, there's research to show
that they're actually more engaging for readers
if they are designed well.
They also, they have the potential to gain a large audience,
and kind of garner critical acclaim for their authors.
If you're a journalist these types of things are important
for advancing your career,
and kind of gaining respect amongst your peers.
If you're a researcher this is important,
because you want your work to get out there
and you want your work to have a big impact.
Just one data point on this,
Distill has published, I think, twelve articles or so
over the past couple of years
and they've had over two million
visitors, two million unique visitors to their site
reading this articles.
So they are really getting the,
the research out there in a big way.
There's also the potential
that this kind of interactive presentation
is actually more efficient for the readers
to learn new concepts.
This is kind of more of a hypothetical at this point,
but there's some research in education space
that sort of points in this direction.
It's also an area for future work,
but generally it seems like there's a lot of reasons
why these are good and
we would want to, sort of,
encourage people to use all the capabilities of the web
as a presentation platform,
rather than just relying on static text.
Unfortunately, they are typically really hard to create
and they're expensive
because they take a lot of time to create,
because they're hard.
This is kind of mainly due to the fact
that they require writing Custom JAVAScript and HTML
to produce,
basically everyone of those examples that I've showed
is more or less, the authors had to write
a custom webpage to make that.
So its not, there's not really
a whole lot of tooling available
that allowed people to create really high,
sort of high production
quality interactive articles like this
currently.
They're also, in addition,
to being hard actually to implement
they're hard to implement well.
Things like, the page loading quickly,
running at sixty frames a second,
there's a lot, kind of a lot of user experience issues
you can run into, and a lot of performance issues
that you can run into.
Especially, if you're not a professional web developer
this things are kind of big obstacles to overcome.
And finally, it's really hard to work in this code setting
with journalists and editors
who are really just used to working inside of programs like:
Google Docs or Microsoft Word.
Also, they're used to working in like CMS's
like WordPress for example, to interactive articles
typically exist completely outside of the CMS eco-system
so they require some
additional infrastructure to set up as well.
In response to these kind of difficulties
that we see
we create IDYLL, which is a toolkit
for creating data-driven stories
and explorable explanations.
That's kind of like the non-academic tag line I guess.
There's a few different research contributions here
I'll just go through them quickly
then discuss in more detail.
High level, we created a document model
for interactive articles.
We created a markdown-based specification language
that implements this model.
We created a component library
that sort of supports a lot of common design tasks:
based on a corpus of articles in this kind of format.
Then, we also created a suite of build,
runtime, and deployment tools
to make it easy for non-expert programmers
to kind of incorporate best practices
in web development and deployment
into their work.
(clears throat) What this looks like in action,
just quickly is,
on the left you're seeing kind of the extended mark-down
and on the right is an example on IDYLL article.
The left is just kind of two sections of text here,
and the article itself
is kind of utilizing this guitar widget,
that sort of sits alongside the text.
These you can actually go and tune the string on the guitar
and sort of things update and responds to that.
There's audio, but I didn't want to make you sit to like,
this bad guitar tunings, sound,
sounds like Guitar Center.
And, (laughs) one thing to know
is...
the, I know it doesn't create these graphics for you.
Like if you really want a custom graphic like this guitar
you're still going to have to use,
through some other tool to do that.
But it really concentrates on making it easy
to reactively parameterize
kind of any sort of component that you can throw at it.
What you're seeing on screen now
with the guitar and the tuner on the left,
these are two different components,
but they're using IDYLL's reactive runtime
to kind of communicate amongst each other.
The Document Model includes data and state variables
that are defined with these tags.
The data variables just read from CSV or JSON files.
Variable declarations work like you'd expect.
You can also embed components to either effect layout,
or render content to the screen.
For example, a table that shows the rows of the CSV file
could be displayed with that mark-up.
There's also kind of some helper functions
to make it easier to create
these kind of scroll-based presentations.
For example, that un-entered view tag
there gets fired every time,
the component comes into the readers view ports
or it comes onto their screen.
That's something that you can just add
to any component that you embed in an IDYLL article,
its just kind of provided by default for you.
Here's kind of the very hello world example.
So we start by declaring a variable x giving it a value.
Then there's some text,
there's a title that says hello world,
there's text that says the value of x is blank,
and then a component that displays the value.
And finally, a range slider
that has it's value bound to x.
When you actually run that,
this is sort of what it compiles to.
So you get a range slider you can manipulate
and the displayed value of x changes automatically.
Kind of the important thing to know is that
you'll notice there's no code here
to actually
tell IDYLL to update the value of x anywhere.
It's just directly bound to the value of the range slider
and anytime that range slider updates,
update of it is sent,
so it tells any other component on the page
that is reading from that value to update as well.
A lot of boilerplate code
for kind of creating the style
of responsiveness is not needed.
Under the hood what this looks like for components
is we
expose a function called update props.
Any component can access this function
and basically push changes back up to its properties.
In this case the value property
of the range slider was set to x,
so whenever the reader changes
the range slider by interacting with it,
it's call back is called,
the update props function gets called
that sends the value back up.
It kind of abstracted the way the fact that
x is the thing that's updated.
The range slider only knows about,
there is a thing bound to my value property here.
This same API is exposed to
user provided components as well.
If you're writing a custom component,
for example that guitar and you want
to sort of effect change somewhere else in the document,
this is kind of exactly
the tooling that you have available.
There's always some sort of top level
document (mumbles) and components can effect that.
We include twenty-eight components by default.
There's kind of four categories:
input widgets which generally map onto HTML inputs,
presentational components which are things like charts,
tables, graphs,
things that effect the layout,
so if you want to put content off to the side,
or make it full screen or full width
or make more of a
fixed component in the background
that other contents spills over,
you can use their components.
There's helper components
that do sort of standard web development tasks;
like adding social metadata or analytics to your page.
Here's just an example
of kind of combining these basic default components.
Here we have wired together a text input
to create an equation that gets passed into a chart;
and to a couple range sliders at the bottom,
which can control the domain of the chart.
This whole experience actually
was created without using any custom JAVAScript
its just IDYLL mark-up.
This one does use Custom JAVAScript,
but generally what you're seeing is
a scroller layout that sort of allows
the author to specify these different background panels
that come in and out of view
as readers get to different sections of the page.
One thing that we did with IDYLL
that is helpful for kind of non-expert programmers
is that we kind of abstract away
all the really messy details
of like JAVAScript build tooling.
If you've already used Webpack or something like that
and (mumbles) against it, you don't have to do that.
Using IDYLL and kind of basic JAVAScript
thats practices like minifying your code
doing rendering on the server side
so there's not like an initial flash on the page,
transpiring code
so that it runs in older browsers et cetera,
it's all kind of built-in to the compiler.
Its also a one-liner to just deploy projects to the web,
in like five to ten seconds
you can go from running your site locally
on the command line,
to actually having a live link that you can share.
In evaluating IDYLL there is kind of a few guiding questions
that we're concerned with.
Is it expressive enough to create the types of rich output
like the examples I've shown in the beginning,
and is it an improvement actually
over just writing raw JAVAScript and HTML,
and can technical and editorial colleagues
kind of collaborate on things together here.
Then, we kind of used a few different ways to evaluate it.
One, we went through a kind of long iterative design process
to arrive at the design that you're seeing now,
which included kind of feedback cycles
with thirteen experts, who kind of guided our designs.
We had a sort of a strong (mumbles)
that this would be something useful
based on kind of the iteration that
we did with this experts.
We released the project
as an open source library about a year ago.
Since then its garnered over a hundred-twenty thousand
page views of the examples in the documentation.
The command line tool has been downloaded
over thirty- thousand times.
It seems like the people in the open source world
are kind of picking up on it.
They enjoy it, by using the library
and also kind of looking at the examples.
We did two pilot studies.
We worked with
under-graduate computer science students
at University of Washington.
They used it for a final project
for a data-visualization class.
Where the assignment was to create
interactive explanation of any algorithm
of their choosing in computer science.
Nineteen groups of four students attempted this.
They're all non-expert web developers,
although they all had some programming experience.
They are able to successfully use the tool to make articles
and they produced some really high quality output.
We, also, were able to analyze the types of features
that they were using in their mark-up.
We found that they were really heavily utilizing
the standard built-in components that we offered.
That sort of indicated that they were probably
writing less code than they would've been
if they were simply using standard web-technologies.
Finally, we did a case study with Folo Media,
which is a small non-profit investigative news outlet,
where I worked with a journalist on their team;
sort of to directly create this article that you're seeing.
This is fully skinned to look like they're site design,
so it didn't feel out of place on their site.
It uses custom
audio clips
that are embeded on the page,
along with kind of the scroll-based graphic presentation,
and some other data-visualizations that come later on.
This was a really interesting experience,
because the journalist that I worked with
he had experience using R,
but he was kind of otherwise non-technical.
After some initial training
we were able to work together pretty well;
getting this set up.
They really saw the value in using a tool like IDYLL there.
This is a case where this is a very small organization
who probably wouldn't have had the capabilities otherwise
to sort of set up the entire infrastructure
that something like this would require.
The tools also being used in the wild
by kind of a number of programmers and researchers
just for writing on their personal blog.
I-tripple-e Vis is happening next week,
and there's a workshop there
that solicit submissions of explorables explanations
of machine learning
that accepted IDYLL documents as one of the formats.
Just recently, this thing happened,
which is called the 2018 Explorables Jam,
which is a online community of people
who are interested in creating explorable explanations.
Several of them used IDYLL for their submissions,
so it's cool to see folks using it out in the wild.
I'm gonna just skip over this a little bit,
and go talk about the future.
We really see IDYLL as a useful tool
as an analytics platform,
because we have the kind of reactive state.
Graph we can keep track of what
features people are using
on sites, and hopefully use this for future work
to kind of better evaluate these types of articles.
We also see this as being a very useful target
for GUI based tools or sort of WYSIWYG tools
to compile to in the future.
With that, I want to say thank you
and I'll take any questions.
(crowd applauds)
- You have time for at least one question?
- [Guest] Hi, (mumbles) first of all thank you
for mentioning Brett Victor and giving me feedback,
I absolutely love this stuff.
What I wanted to know for me as a programmer,
how the interaction works for custom components,
if you want to build custom articles
and having the different needs for the different articles?
How you get from the react level
back to the mark-down level?
- [Speaker] Essentially,
you have this update props functions
that if you're in react and you have a handler,
say there's a click events or something,
you can say based on whenever this click handler fires
I want to update one of my properties.
That allows you to update a property
without knowing anything about what's in the mark-up,
if that makes sense?
- But, imagine I have this custom (mumbles) component
that I wrote and I want to integrate it
in an article I wrote in IDYLL.
- Oh yeah, that'll just work better for all
- Hmm?
- That should just work.
Kind of out of the box
- Okay so you just mention it by name and then--
- Yeah, yeah.
- Thank you.
- Yep.