- What I'm going to do is just put
three ideas in the minds of our panelists here.
And then, as they said on Saturday Night Live,
ask them to talk amongst themselves.
And I'll join the conversation a little bit.
But so, at the beginning, you remember
I showed you that little graphic with the cars going
back and forth from exploration to explanation.
And I think you saw some amazing tools for exploring data today.
And then you saw, especially in our last presentation,
a goal of explaining to the public what
the data mean or could mean.
But, Blacki, I think you maybe didn't give yourself
quite enough credit because, when
you have these very interactive graphics,
sure, people can do exactly what you
intend them to do with it, kind of the way you have
to put it into print and give them what you
think they might do with it.
But then the interactivity lets them
go ahead and do their own analysis almost at the level
as if they made the graphics themselves.
And I'm sure that that happens from time to time.
So that's an example of this kind of possibility
of going back and forth from exploratory
to explanatory tools.
But I just want to bring two other ideas
into the sort of topic here.
And one is what our friend Edward Tufte
likes to call the lie factor.
So several of you talked about machine learning
and computers and partnerships.
And can we have computers do something?
And we do something else.
Or maybe we go to the beach.
Though, it's going to be very hot.
Anyway, but what I was going to say
is, when you have a known bias--
so, for example, all of us here and many people in the audience
know that, when people look at lines and shapes and volumes,
humans tend to systematically underestimate and overestimate
particular things compared to their true area, for example.
So, when a computer looks at that,
they don't do that, right?
And so there's a possibility of correcting for that
or not correcting for that.
And so the more we bring machines into the visualization
world, the harder it is to make choices
about what it is that you present to humans because do
you want to take their cognitive bias into account, which,
by the way, would mess up this explore/explain
thing a little bit unless you had a little tab for is
this a human, or is this the computer in the data.
OK, so you have to have that.
And then the last point is we're having a conversation where
I think all five of us and many people here understand
all of these talks and what we're
talking about and a lot of the nuance of what we're talking
about, but I'm guessing that a lot of people
here are very interested in all of this,
but didn't completely understand everything that everybody said.
And that's because even an incredibly educated group
of people like this who come and spend an afternoon at something
like this don't have the same kind of background
that people are given in say writing.
You know?
So they spend a huge amount of time
on teaching writing in school, but Felice and I
were having this conversation the other day
about visual literacy and visual thinking.
And there's very little education about that.
So, in our very few minutes and then in our reception,
what do you think about the role of these tools,
these new tools, in going back and forth
between this exploratory and explanatory world?
How do we take cognition and perception and this live factor
problem into account?
And what do we do about explaining this to everyone?
Feel free to just jump in.
- Do you mind if I ask a follow-up question to get it
going?
So, Blacki, as I said before the talk,
I'm a huge fan of your work.
And your Bloomberg example where you
go through the climate and sort of the storytelling,
I just showed in my class two weeks ago
as this incredible example.
And we looked at different visualizations
with climate change data to sort of look
at different strategies.
But one of the questions I got, and I'm like I don't know,
is one of my students asked but how do they come up
with these visualizations.
Like, to me, the concept of working in a newsroom,
and you have to do this incredible explanatory
visualization in a really short amount of time,
I'm like oh my gosh.
But you must do an explore step first, right?
Like, when you get the data, how do you
go through the sort of I call it "explore,"
sort of sifting your data, to come up
so quickly with that beautiful explanatory graphic?
- Well, first, people don't even understand
that like what data journalists do is not just
make the visuals, right?
We are standing in a newsroom and pitching stories.
And that means that we've been like reading the research,
and we are like calling researchers and just
asking them a lot of questions like help me understand this.
And like that particular graphic was just
like a figure in the IPCC report, you know?
And it was like I understood it was
like very kind of complicated, but I--
like we understood like, OK, you could guide somebody
through it, and it makes a lot more sense.
You know, you mentioned something about the explorer,
like exploratory data analysis, and then kind of there's
the presentation.
I think we do a lot of the exploration in the act
of trying to develop the story.
And then, because that's the story
we're trying to tell, essentially,
what we're doing is we usually try
to guide somebody through something complicated
and then just kind of leave them with now
you can explore kind of fully.
So, you know, this is--
you can call it scrollytelling.
I kind of hate the term, but you scroll, and you know.
But, really, just the process of like extracting out
the parts that you think are really meaningful and kind
of presenting those and almost teaching the reader
how to understand a chart, that ends up
kind of tying into this whole like the literacy--
like the visual literacy of these charts
that like they have a vernacular to themselves.
So you're kind of like--
if you do it well, you're trying to explain the chart.
You're trying to guide somebody through complicated data.
You're trying to simplify a lot of things.
And you're trying to make somebody feel something.
It's kind of-- it's a lot-- too much to do in one story.
- If I can.
So that's great.
And let me just guide the sort of live factor question
to you, Danielle, and then the education question a little bit
to you, Arvind, because, you know,
you're giving this talk about, oh, look,
I can make this complicated code less complicated code.
But a lot of people don't even understand code at all.
And I'm not saying that they shouldn't have to, OK?
And so part of the question is how much should they have to.
And what should they do?
And then how much do we worry that computers and humans don't
see the same thing?
- Yeah, no, I think you're absolutely right in thinking
about sort of how much technical expertise, whether it's
in terms of computer science, you know, coding
and things like that or even visualization or data
expertise, do people need to be able to build those things.
And that's why we've sort of approached--
our work is sort of this idea of layered abstractions,
right, where we're trying to raise the level of abstraction
that people are either thinking in terms of,
working in terms of, et cetera so
that, if you're a novice data visualization practitioner,
we're not asking you to use D3 or Vega
and work at the caliber of folks at The New York Times.
We're saying here is a much higher-level library called
Vega-Lite where you can just talk in terms of,
oh, I want this data field on the x-axis.
I want this other data field on the y-axis and so forth.
And you don't need to understand what's
happening behind the scenes.
But, as you build that expertise,
maybe you start taking additional steps
and expanding your knowledge.
And that's where sort of a system like a Lyra
comes into play, that drag-and-drop system I showed
briefly, because what Lyra is trying to do
is smooth out that entire notion of abstraction.
It's not fronting to the user, you know,
you're using Vega-Lite.
You're using Vega.
You're using D3.
It's saying, you know, just interact
with the tool in some sort of fluid way,
and the tool will figure out the most--
the level of abstraction that makes the most sense.
And so, in our user studies and in deployment,
people have told us that Lyra has
been a really interesting platform to teach data
visualization for that reason.
- Right, there's definitely, though,
another level of abstraction, which
is Michelle's question essentially to Blacki
of like how do you even know what
it is that you're trying to make in the first place.
And that's what I think--
that's why I brought up the apples
because, in other words, when it's bar charts with apples,
five-year-olds can understand.
- I mean, not to take up too much more time,
but Michelle's question reminded me
of some really early classic work in visualization
was around the system called Many Eyes that was built out
of IBM by Martin Wattenberg, Fernanda
Viegas, and their group.
They're now at Google, actually, in Cambridge.
And what they found in that system was you give folks
an interesting data set, and people will go ahead
and like do really interesting things trying
to find interesting patterns and interesting places in the data
and tell little mini stories about it,
you know, relate it to their own expertise,
their own anecdotal data, et cetera, et cetera.
They found all this like crazy, fun activity happening.
For whatever reason, IBM chose not to move--
you know, not to continue maintaining that platform.
But I think data journalism now gives us
a really interesting sort of sandbox
to explore some of those questions.
And, again, the data journalists are talking about issues
that we all care about deeply.
If you go on The New York Times for TheUpshot
or things like that, they have a Comments tab.
Luckily, it's moderated.
And those top-rated comments are fantastic.
Yet, all of those folks are talking to each other
through pure text, even though the data journalists were
talking to us in terms of data, in terms
of interactive graphics.
What does the world look like if we expose those tools
to the readers as well?
Yet another thing my group might be working on.
- Fascinating.
So, Danielle, what about this perception problem?
- Yeah, so I think it actually builds pretty nicely
on top of what Arvind was mentioning in that it's
an expertise thing, right?
And generally, we want to make things intuitive.
And we have this balance of how do
we help people see the right thing they need to see,
and how do we help them see it accurately,
and how do we make it engaging.
And, sometimes, there are tradeoffs
we need to make in terms of making things
engaging and making visualizations that
tell the right story and that make
people want to work with these things
that we can't necessarily automate the whole process
and that precision isn't always perfectly our goal.
So we want to be able to help people understand and get
the right story with their data, but there
is some tradeoffs that need to be made in terms
of what the designer is doing.
At the end of the day, when you look at historically--
and this is something that Mike [? Glazer ?] talks about
a lot--
art has moved faster than our understanding mathematically
of what artists are trying to do.
And so, in many ways, we need to be
able to leverage the expertise of the designer
to try to understand and kind of embody in these tools
how we should be representing data
and then almost use these kinds of mechanisms where we're
understanding how to do it well and how
to do it right to ensure that the kinds of things
and the kinds of patterns we're seeing
are honest, as you said before, but not to overly constrain
the space such that we're trying to automate away everything.
- I'm also glad you said embodied because we didn't have
a chance to talk at all about virtual reality
or augmented reality or any kind of visual perception
that's only recently become possible.
So sorry about that, everyone.
But I think now we can take some questions from the audience.
How about just this few of the people who were standing up?
And everybody else who has a question
can ask it over snacks downstairs in a few minutes.
So go right ahead.
- Well, thank you very much, very interesting lecture.
I consider myself highly educated,
but, looking at all of this, I'm wondering--
I think I'm lost sometimes.
And I'm wondering, if I am, I wonder how many people
who else could be.
I have one question for you, though.
All these extrapolations that you are making,
is it from the y- and x-axis?
- Well, I don't--
I think it really depends on the chart.
And I don't--
I wouldn't-- I mean, there was just so much shown there today,
stuff that was like 2 or 3D, geographical.
So it's not just like a x and y thing.
So I fully understand.
- And I think, if we bind ourselves just
to thinking about the x and y, while position is incredibly
powerful, we're actually limiting ourselves
in terms of the vast breadth of encodings
of visual representations that we can have.
So there are a lot of patterns that can stand out
to people that have absolutely nothing
to do with the physical position of data
within the visualization.
Traditional visualization-- scatter plots,
line graphs-- there's a reason we see those everywhere.
People understand them.
They know how to use them.
These are things that we've seen since we
were in elementary school, and they're incredibly powerful.
But there are creative things we can do, especially
as the variety and the veracity of the data
sets that we're using change, that will allow people
to really more freely express and navigate and potentially
see really interesting things in their data
that they couldn't otherwise before.
- I have a kind of a perception question
and a little about some examples I've seen.
I don't know if you're familiar with the articles where they'll
take a picture of a panda and a picture of a vulture,
and they'll sort of just adjust some of the pixels perfectly
so it still looks like a panda and a vulture to us,
but the AI will think, neural network,
well, that's really a 61% it's a vulture or a panda.
And then I think it's sort of like your one with the Golden
Retriever because you've got that Labrador that
looks like a Golden Retriever, but you say it's a Labrador.
So it seemed-- and so it seems as though the machines,
even though we both are getting to see pretty well,
we seem to somehow see differently.
And then the question is who's right.
Or is it sort of not the right question?
That's the end of it.
Did that make sense?
I mean, was the panda really made into a vulture,
but we didn't notice it?
- That's a good question.
I think part of our approach has kind of just
been thinking about how we can use visualization
to make sense of what machine learning models are learning
and leave that sort of semantic step up to the human.
- Right, I think--
I'm so glad you brought that up because a lot of what's
happened in the last few years is that people have gotten more
and more interested in what these machine learning models
are doing because it's true that they can do amazing things.
But, if we totally don't understand what they're doing,
it's a little bit scary.
And some of you were probably at a talk
we had earlier this year by Ben Shneiderman,
this academic year, that was about
algorithmic accountability.
And so I recommend that you go back and look at that talk.
Actually, I guess it was almost a year ago now.
It was in April of 2018.
Anyway, this woman in the blue here
was the last of the three questions I said.
So I'm sorry, but this will be the last question.
And everybody else can join us and chat some more
at the reception.
So go ahead.
- Hi, I can't remember which one of you said it,
but it really struck me when you said that interacting with data
was like a dialogue with the data.
And you could question it and get answers.
I thought that was really clever.
And then, later on, I was looking at it and saying
but, you know, when you're doing those lovely little slide bars,
and things are all changing, I have a feeling that correlation
isn't causation gets lost because, suddenly, you're
causing.
You know, you're changing x, and it is causing a change in y.
And so I'm just thinking visually to people,
you've kind of taken one of the underpinning
to the really strong underpinnings of stats
and said, nope, it does.
So it was just, how do you not do that?
- Yeah, we don't know.
So, you know, we know a bunch of stuff
on the pure, static, visual side.
We don't know a whole lot on the interactive side at all.
And part of that has been because so much
of the interaction, in terms of how we construct
these visualizations, has been at this extremely low level
of abstraction.
It's been here is just JavaScript code.
And so, when you're dealing with JavaScript code,
it's really unclear.
How do you frame that experimentally like Danielle
was with all the marks and colors and shapes and sizes?
We didn't have that vocabulary to even know
what set of experiments we should be running to ask
that question you just asked.
- Also, I would just add that one other point that's
really important is, when Michelle was talking
about our Glue project, one thing
that is important with reference to your question
is that you can make these kind of interactive diagrams.
But then, when you're done, if you think you see something,
you can get the data out.
And you can make new plots.
And so that's true actually for some of the tools
that you heard about in other contexts today,
but I think that, as that becomes easier and easier
to expose the data that underlies even
the interactive visualizations, people
are going to be able to make those judgments for themselves
and see whether there really is a causation, not just
some fake correlation that they saw for a second.
So it was a good question.
Thank you.
And so thank everybody for coming.
And let's thank our speakers again.
[APPLAUSE]
[MUSIC PLAYING]
That was a great talk.
We'll see you all downstairs.
Thank you very much.