- Thank you very much for the introduction
and we've started this track with the topic of aviation,
and we're also gonna finish this on aviation.
And I'd like to welcome you to my presentation
about our paper, Turbulence Ahead,
A 3D Web-Based Aviation Weather Visualizer,
which is based on my masters thesis that I did at the
Institute of Cartography and Geoinformation at ETH Zurich.
So first, I'd like to talk about the motivation,
why we're wanted to look into turbulence.
So I assume that most of you
have come here to this conference by an airplane,
and if you used an airplane,
you're probably very familiar
with this kind of sign up there.
It means during the flight, if it pops up, it means for us,
let's go straight back to my seat, fasten my seatbelt.
And I have to probably expect turbulence to occur.
So maybe you have wondered,
"How does the pilot actually know when turbulence occurs?"
And maybe sometimes you're sitting in your seatbelt
and you're excited for the turbulence to come,
and there's no turbulence happening.
So yeah, so maybe you've wondered this once.
And actually, if you talk to pilots,
they have a lot of knowledge about turbulence.
And they think about turbulence in terms of categories.
There's light turbulence, moderate, and severe,
and especially severe turbulence
is something that every pilot tries to avoid.
And why is that?
Of course, because it means that an aircraft
may be out of control for a period of time
that can lead to accidents or injuries
of the flight crew or even of passengers.
So a pilot knows that severe turbulences
are always to be avoided,
but already moderate turbulences
can really decrease the experience
of the passenger in the aircraft.
So turbulence, no good thing for pilots.
So, how does a pilot know when turbulence occurs?
Of course, they access weather data.
And they have different sources of weather information,
and one of them is that even before they start their flight,
they look at 2D maps that predict turbulence areas.
And there are so-called significant weather charts
that predict turbulence areas,
always six hours ahead of time.
So this is what they look into
before they even start the flight.
And when they are in flight,
they of course have other sources, such as weather radars,
that show them turbulence areas.
However, for the weather radar,
they only see turbulence areas
that are actually coming with rain,
or anything that reflects.
But clear-air turbulence is something that can only see
from the significant weather charts,
and it's really hard to predict that.
And the weather radar also has other problems,
such as that the reach is only about 150 kilometers
and we can also see from this image up there
that the angle is restricted.
So one of the pilots that we worked with on this project
said that with the weather radar only,
the pilots only would be partially blind
when it comes to the weather information around them.
Of course, they also access other information,
such as communicating with the ground personnel,
with air traffic control, as we've heard in the first talk,
or also even with pilots that are flying ahead of them,
asking them, "So how's the weather up their front?
"Can you tell me something about the weather?"
However, this information has the problem
that it's only textual.
It's maybe also the perception of the person
that is in this airplane or on the ground
giving the information,
making it really hard to compile all this information
for the pilot to make decisions
based on the weather information that they have available.
So in our work, we looked into
how can we improve this situation for the pilots?
And of course we're not the first
to look into the dimensionality of the weather information.
Already in the '80s and '90s,
people started experimenting with 2D and 3D displays
to show weather information for pilots.
And this particular study here from 1994
actually concluded that the 2D displays
are more feasible for pilots
because it made them perform better, make decision faster.
However, from the picture,
we can already see back in the '90s,
3D graphics weren't so visually compelling
and really hard to understand for pilots, as well.
More recent work that we also considered
for our prototype that we designed during the studies
were visualizations of atmospheric data very generally.
So meteorology, and also environmental science,
they have come up with a lot of compelling
and visually compelling visualizations
of wind information, clouds, and so on,
as you can see here from one example
from the Helmholtz Institute in Leipzig.
And we took also this as a inspiration for our prototype.
However, with a lot of weather products in aviation,
they usually have in mind meteorologists to look at them,
and not pilots.
So usually the target audience
is more towards environmental scientists
than actually pilots, which is a problem.
So what was the research question behind our work?
So what we basically wanted to do is
we wanted to compare the 2D maps
that pilots use these days,
those significant weather charts, with a 3D prototype
that essentially shows them the same information,
but visually more compelling,
in 3D, and with color, and interactive.
And with this comparison,
we looked at different measurements such as the performance,
so how fast can pilots make decisions
based on how the weather information is visualized to them.
We looked at the cognitive load,
so we tracked how difficult it was
for the pilot to extract information.
We looked at the usability of the systems,
and the last and probably most interesting one
about the spatial awareness,
so how does it impact the pilot's spacial awareness.
And we expected that the 3D prototype
would always outperform the 2D map.
So how did we proceed to answer those research questions?
The first step was to make a first draft
with the data available,
and to consult with a subject matter expert.
Namely, a pilot from an internationally operating airline,
to understand how they read the map,
and what information they need to make decisions.
The second step was to make a pilot study after that
to come up with a good design of the first prototype
that we then used in a online user study with 64 pilots,
where we collected a lot of data that we analyzed
to then answer those research questions.
So let me dive into this a little bit more in detail.
So for the first draft, as I already said,
we consulted with a pilot
that first had to explain us how to read those 2D maps.
Pilots, when they get trained,
they learn how to extract the information.
Even though those maps are neither really nicely looking,
than actually easy to understand for the pilots.
But they are experts in that.
So that was the first step.
And then we extracted different objects from those map
and prepared the data in a geoinformation system
called ArcGIS Pro,
and then we saved the data onto a database
and from there developed a web application
that we then visualized in the web.
The library that we used to visualize the 3D data in the web
was ArcGIS API for JavaScript,
which is essentially a library that builds on top of WebGL
to visualize 3D content in the web.
To come up with a design
that is nicer to use for the pilots,
we did the pilot study with the pilots.
This is intentional, to sound ridiculous.
And we did a design iteration
with three subject matter experts
where they could change a lot of things on the application,
such as the vertical exaggeration of the objects
or the colors, or the representation of the 3D objects.
And they came up with three different designs
that sort of look similar, but have some differences.
And we run them back to them, asking,
"Where do you have things in common?
"Where do you agree?"
To come up with the final design of the prototype
that we then used for the user study.
The prototype's interface was inspired by an application
that the pilots are already using these days.
It's a navigation application
that they have on their tablets in the cockpit.
That was from Lufthansa.
And if you look now at the final design
of the interface of the prototype,
it looks very similar, has the same colors,
has some of the functionalities,
such as the day and night mode.
So it's just to improve the learning curve for the pilots
when they do the online study with the new prototype.
In our user study, there were a lot of pilots
interested in participating,
and we had initially 142 pilots
that wanted to participate in the study.
However, only 64 of them actually finished the study,
and I'll explain in a second why.
And we divided those in two groups
to have a counter-balance with the two conditions.
And those groups were very similar,
and there are no significant differences between the groups.
And here, you also see one of the reasons
why some of them dropped out earlier.
The study took on average 51 minutes to conclude,
and so we're really happy
that so many pilots still participated.
And this allowed us to collect a lot of data.
The study itself was designed
as a within-subject experiment with a counter-balance,
with one group starting with the 2D condition,
and the other one with the 3D condition.
Every user had to first fulfill a map task,
where they got presented a map
and had to answer questions
regarding their flight preparation.
The second task was a spatial awareness task.
They also got presented the map,
but then the map disappeared,
and they had to draw every object
that they remembered from the map that they saw previously.
This is the whole procedure of the study.
And after every map task,
the participants had to fill out a questionnaire
about cognitive load, the NASA Task Load Index,
as well as a usability questionnaire,
the system usability score.
For the spatial awareness task,
they were first asked to draw all the actual intersections,
so objects that are penetrated by the flight path,
and after that, they had to also draw all the intersection,
all the objects that potentially intersect
with the flight path.
Pilots are aware that the data that they get presented,
that they are only certain to a certain degree.
So they know that some of the objects
might move faster than the prediction anticipated
from their experience.
And now lets look at the results of the study.
So the first measurement that we looked at was performance.
And here we found four out of the eight questions
that they had to answer were significantly different,
whereas for three of the four,
the 2D performed better than the 3D.
And with our last one,
number eight that you can see up there,
also the one that took the longest to answer,
the 3D actually performed better than the 2D.
However, those results, we have to be very critical about,
because we also found group effects,
which means the counter-balance didn't work.
So a bit more exciting results about cognitive load.
So here we found that
for five out of the six parts of cognitive load,
there are significant difference.
And the 3D out performed the 2D,
meaning that it was easier for the pilots
to extract information.
Also, for usability,
we found that the pilots were much more satisfied,
and it was much easier for them to use the application.
For the 2D we found a score of 60%,
and for the 3D, 78%.
And if you're familiar with the system usability score,
average to 2D was even below-average here.
And of course, also, this difference was significant.
And then the spatial awareness.
So for this drawing task and the sketch maps
that the pilots produced,
there was not really a good measurement to use
to analyze this data.
So we introduced our own measurement,
the so-called aviation weather objects similarity score,
or short, AWOSim.
And I'd like to quickly show how it works.
So here we have the baseline,
so this is what the pilots should have drawn
after looking at the map.
We have at the beginning of the flight path
a clear-air turbulence area right there, close to Zurich,
and then two jet streams that cross the flight path,
and at the end, a cumulonimbus cloud.
The first pilot has a score of 70.6%,
drawing two of the four objects that he was supposed to draw
with pretty accurate size, and also the position
along the flight path was pretty accurate.
The second pilot scored even better, with 80.6%,
by drawing all of the four objects
that he was asked to draw.
And then the last pilot scored a little bit less,
because he also draw objects
that he wasn't supposed to draw,
not being able to identify and localize the objects
that are actually intersecting with the flight path.
For spatial awareness,
for the first question about the actual intersections,
we found significant differences between 2D and 3D,
3D being better, and allowing better spatial awareness
for the pilots.
For the second one about the potential intersections,
we didn't found any differences,
mainly because of sparse data
because most of the pilots did not fill out anything.
And we think it's because the question
wasn't phrased correctly, they didn't understand correctly.
So to conclude what we found with the user study,
for performance, as I already said,
no clear results unfortunately there.
But for cognitive load, usability,
as well as spatial awareness,
we found that the 3D prototype
with all the improvements with dimensionality, adding color,
and interactivity, performed better overall
than the 2D maps that they're using now.
I'd like to give a quick outlook about this research project
because I'm not at ETH anymore, unfortunately, now.
I graduated form ETH.
But the research project is still ongoing in the group,
and what we're looking into now
is how to bring this actually into the cockpit
to improve the situation about aviation,
whether for the pilots,
to improving the usability on tablets,
or by also introducing new sources of information
such as adding dynamic content, different time stamps
of the weather information, or wind simulation,
as you can see in those images up there.
And this is just a quick teaser
to show how the application looks like
and what you can also see in the demo afterwards,
so I think its right after my talk,
the demo reception is gonna start in a second.
And we have the desktop application here as well
as a tablet for you to try it out.
And so I'd like to encourage you to come to my demo, D-59,
and with this I would like to thank you for your attention.
I'd like also to thank ETH and the Institute of Cartography,
especially my supervisor David, who's also here today.
I'd like to thank Swiss International Airlines
as our industry partners.
And with this, I'm open to your questions.
(audience applauds)
- [Woman] We have time for a few questions,
before the demo reception.
Okay, while people are thinking about a question
to ask, I had a question.
So it's interesting that 3D maps were found
to lessen the cognitive load,
in terms in representing this information.
Were there specific types of encodings
that really helped with that,
with minimizing the cognitive overloads?
So that's one question.
The second question is, given that you are using 3D maps,
are there particular new types of encodings with new data
that you find more convenient
to encode in a 3D representation as opposed to 2D?
- Okay, so, about the first question.
So one of the biggest problem with the 2D maps
that the pilots are using now
is that they have a lot of visual clutter,
because the maps try to actually put the 3D content
or 3D information which is important for the pilots to know.
For example, if an object starts,
at what height it starts and ends,
they put that as label onto the map.
So we were able to declutter that in 3D
by just visualizing this information
and not having it as attached like the 2D.
So I think this is the main thing for the pilots.
And the second question is what we did to help them see
if the flight path actually goes through the object,
which is something that they can only guess from the 2D,
most of the time, because the flight level
is not visualized on the 2D map.
So we visualized the flight path differently
with a different color, because we were able
to assess this information in 3D quite easily.
So they could immediately see
where they should pay more attention on this application.
- [Woman] Cool, thank you.
- [Man] Sorry?
- [Woman] Yep, we have time for another question, yes.
- [Man] How did you work with the pilots?
Did you try to have real scenarios
of the use of the map and the weather map?
Like for example, can you say something
about how efficient it would be to avoid a storm,
or to devise a new path in the clouds?
Was it related to actual tasks?
- So that's a good question.
So we looked into this for the user study, of course,
to maybe not have just answering questions,
but having an avoidance task.
We did not pursue that,
because we used data that they use for flight preparation.
So they use that before they even start the flight.
So in that case, this is not really a critical situation.
So we went with them through the steps that they look at
on how to prepare for a flight,
rather than being in-flight
and have to immediately react to a situation.
But I know there's a lot of studies
that did this kind of task, avoiding a turbulence area.
So this is the common thing to test this out,
and we will in the future, also,
try to apply this for in-flight scenarios,
and in this case we'll definitely run studies
that will have the task like this to avoid a certain area.
- [Man] Thank you.
- Thank you.
- [Woman] All right, with that,
I'd like to thank out last speaker
and all the speakers of this session.
Thank you very much.
(audience applauds)