- All right.
Well good morning.
Thanks of coming around at this early hour.
My name is Thomas Roggla, I come to you from
the National Institute of Mathematics and Computer Science
in Amsterdam, Netherlands and today I'm going to present,
I'm gonna talk to you about a project that we ran
which is called, From the Lab to the OB-Truck
where we went from a lab study, from a series
of lab studies to deploying on a multi-screen broadcasting
at the FA Cup final in London in 2018.
So we went from planning, to lots of programming,
to running a series of interviews, to this,
the final deployment at the actual stadium
during the game in London.
To give you a little bit of context
this little project was part of a bigger
EU-funded project called 2-Immerse which aimed to
develop a platform for interactive, immersive,
multi-screen broadcasting.
And it involved partners from all over Europe,
most notably in this part of the project
it was us, CWI, BBC, British Telecom and a company
from Sweden called Chyronhego which they mostly
deal with live on-screen graphics targeted at
mostly sport events.
And the platform is a decentralized microservice
architecture and has been open sourced a couple months back.
We ran a couple trials with owners of high-profile
content, mostly sports content, for instance the
MotoGP and as we'll be talking about in this presentation,
the FA Cup.
The project also had a very successful presence
at the International Broadcast Convention in 2018
and won a series of awards as well.
So during the development of the project
we faced a series of challenges.
The first one of these challenges was to actually
understand how live broadcasting works,
what are the roles and workflows involved
in creating live sports broadcast.
As you see here you have a lot of screens,
lot of panels, lot of buttons, you have a lot of people
that work together and everyone needs
to depend on everybody else in the truck
in order to guarantee a timely delivery of the broadcast.
So this is the typical inside of a outside broadcast truck.
You have, yeah screens here on top, you have
a director which is calling the shots,
you have a series of producers, you have people
that are responsible for audio, you have people
that are responsible for on-screen graphics,
and a series of people who are, yeah they communicate
with the main studio, they prepare replays,
things like that.
And the second big challenge in our project was we--
Sorry.
Yes, the second big, another challenge in the project
was to integrate the architecture that you have
onsite, the cameras, the video delivery infrastructure
to integrate with the existing 2-Immerse platform.
Yeah here on the side you see,
you have the cameras in the stadium,
and for us they need to run to an encoder
and the video later goes into the CDN
which is then delivered to peoples homes.
Also we have in our truck we have the production tools
which facilitates how the broadcast comes together
and where things go on screen and all of this.
And then we had of course to actually
deploy the thing at the FA Cup final
and make sure it shows up in people's homes.
In order to do this we made use of a broadcasting
method called object-based broadcasting
which differs from traditional broadcasting
in that instead of the content being assembled
at the truck on site, all our assets are usually
transmitted via the internet and they're
only assembled at people's homes.
This means that the content can adapt itself
dynamically to different screen sizes,
different devices and it can also distribute itself
to multiple screen and offer also interactive
content where people watching the football game,
they can interact with other people,
they can pull up statistics and tap into
live screens and updates and different cameras,
choose their on camera angles.
In order to facilitate this we implemented
a series of production tools.
Using the production process as we understood
from live events involves the pre-production
of pre-recorded content such as interview
maybe highlights from last match, things like that,
then you have on-site editing of live content
from the cameras on the pitch,
of course all your mixing and all your video mixing
and insertion of on-screen graphics, final transmission
of the finished broadcast to people's homes.
Yeah, as I said pre-production process
is actually a lengthy process.
You have to prepare lists of players,
data, graphics, things like that.
And all of this is already very well served by existing
software and professionals they know how to use
this software and we also found that this is
sort of difficult to integrate with object-based
broadcasting, this is really targeted at traditional
linear broadcasting.
So, we developed a tool, a pre-production tool,
which sort of tries to, instead of replacing any
workflows it tried to augment workflows.
It can, it allows the producers to sort-of compose
a high-level structure of their program
in terms of a sequence of events
you have pre-match content, for example,
and you have a couple interviews,
then you have your live stream of the actual match
and then you have maybe the break, the second half
of the match, and post-match interviews.
And it sort-of allows you to put together
this high-level structures and insert
pre-recorded assets but also placeholders
for live streams which then at the actual broadcast
are then actually filled in by the thing that,
by the match, by the live match.
During the live phase, production teams
also need to be able to react to events that might happen.
For example in a motorcycle race this might be a crash.
In a football match this might be a foul or a goal,
and you also might want to see different replays.
These events they are likely to happen,
one might say in a football match
you even want them to happen, but the thing is
you don't know when they are gonna be happening.
And for this we implemented yet another tool
called the live triggering tool.
It consists of an interface like this,
you have these big buttons that are predefined,
you have things like name tags for players,
for the referee you have the little score clock
in the corner and the goal counter and all of this.
And this interface allows a graphics editor to sort-of,
on queue of the director, bring up a little interface
to fill in names and queue them.
Because this is then pushed to a second interface
which looks very similar, and from here
allow the events that are configured in this interface
have been queued up here and on command of the director,
clicking the button here inserts them into
the actual broadcast.
We found also that while studying
the going ons in an OB truck we found
that TV professionals they're very much used to
big buttons and hardware devices,
so we found this little thing.
It's called Stream Deck, it's actually a device
that is meant for, I'd say Twitch streamers?
It has 15 buttons and each one of these buttons
is backed by a 72 pixel LCD so you can actually plug this
into the laptop and trigger the content from this thing.
And in my interviews producers were very impressed with this
cause this is really something that they're
used to being work with and felt very natural to them.
And I have a quick video of the whole process.
See if it works.
Yep, now you'll see foul, there you go.
The director might say I want to see a name tag
for the player that has committed the foul,
they've selected yellow card, number 4,
then a subtitle, insert.
It shows up in the little bolt
and the director says, I want to see it now,
press the button, there it goes,
and it shows up on screen.
And once you're done with it you press the button again,
it gets removed again.
So in order to evaluate this and make sure
that it is usable for live teams,
we ran a series of field studies
where we interviewed professionals and asked them
to evaluate the software that we had put together.
Inherently we used this to improve
the UI and UX of our tools which then resulted
in the software that you just saw.
And then finally we also ran, before the actual final,
during the semifinals we ran a couple pre-tests to insure
that we have a smooth deployment of the whole thing.
The software was then finally deployed
at the final of the FA Cup in London,
Wembley Stadium in May 2018.
Our project partners at BT they helped us organize
a rendezvous OB Truck with all the screens
and all the hardware to tap into the actual live streams
of the match that was being recorded on the pitch.
And we tried to replicate the actual
broadcast that was going out to people's homes
through the TV, we tried to replicate this
using our platform.
And it was transmitted to people's households,
to a select few of households where people
had their own 2-Immerse set-top box.
This is sort of what the home deployment looks like.
Right here you see this is a TV and you see
the mainstream it looks very much like,
we tried to replicate pretty much exactly what you'd see
in the traditional broadcast, but on top of that
people received additional content
on their component devices, maybe tablets or phones,
and you had different interfaces where you could customize
what you see on the tablet and also
what you see on the main screen and you can
pull up information, play your information,
statistics, things like that.
Here's another quick video,
I think you see a penalty and see a person here
with the tablet, they pull up the coach camera
on the main screen and just to see
the coach's reaction to the--
there you go, there's a goal.
Yeah, so you can do the multiple screens,
and you can customize what you want to see.
Finally, lessons learned.
Live TV productions very strictly are defined
workflows, it's a very well-oiled machine
and everybody needs to rely on everybody else.
Also, during our interviews we found
it was very hard to gain access to broadcast professionals
because they're very busy people and have tight schedules.
Moreover, rights-holders, especially of this
high-profile content, they're very protective
of their content, it's very hard
to actually develop trials and demos
because they're very scared you might do something
with their content that they don't necessarily like.
On top of that, also production teams,
because it's all, if it ain't broke don't fix it,
production teams are somewhat reluctant
to adopt new workflows so really the trick is
we want tools that augment and enhance existing workflows.
Because of our high-profile content and trials we run,
we had a quite successful presentation of the platform
at the International Broadcast Conference.
We won a couple of best paper awards
at multiple conferences and industry awards
for the HbbTV Symposium.
Also talking with professionals at the IBC
everybody was convinced that object based broadcasting
has a future because especially now because
a lot of this stuff is moving to
on demand broadcast including internet
things like that, and also it was very good
for our commercial partners like Chyronhego,
like the BT and the BBC because they broadcast sports,
their business is really broadcast graphics.
It really allowed them to try out new technologies
and see how they might perform on the market
and give them a chance to have
a small-scale deployment of these new technologies.
Again I want to point to you, if you're interested
in the software I want to point you
to the GitHub where we open sourced the entire platform
a couple months back, especially if you're interested
in the software in this talk you should
check out the editor and the trigger-launcher repositories.
They're still somewhat being developed
but I think it's in good shape now.
And with this, I'd like to thank you for your attention
and if you have any questions, please.
(applause)
(woman talking in background quietly)
- [Woman] I ask a first question, well actually
I'm a media scholar, mass media, new media scholar.
I'm wondering, you know, the software you built
often is supposed to solve some challenges and problems
the broadcasters are facing.
So in your case, you mentioned
that broadcasting professionals are very busy,
they're resistant to completely new technologies,
what other existing challenges do they face
that this particular software helps them to solve
that existing broadcasting system cannot really do.
- It mainly helps them to... traditional broadcasting
it really allows you to have content on a single screen
whereas with this system, makes it very easy to
have multi-screen interactive broadcasters
broadcasting onto multiple screens.
And it allows you to do all of this
from a centralized location.
So whereas now you might have your football match
and you might have additional apps,
you always have to download, for every program
you have to download the different app
whereas, and the broadcast is sent out by these people
whereas the app is under control of someone else completely
maybe even a different company.
Whereas this whole thing is one architecture
where you can broadcast to multiple things from one
centralized location, it's all very well integrated.
And goes to one place in the home,
you just sit down on your couch,
you turn on the TV, you get the tablet
and there you go.
- [Woman] So for the users, the users can
use multiple devices, right? - Exactly
So on one side it makes it easier for producers
to actually produce this content
and it also makes it more convenient
for people to tap in on consumers content.
- [Woman] I see, okay thanks.
Any other questions, yes please.
- [Man] So now that you're thinking
about the end user, people in their living room,
did you have any interviews or did you get
any of the actions from people who used these multi device?
- So far only really within the project.
So the project ended two months ago,
so I think now the BBC and BT they have
a vested interest so I think they're gonna pursue
this further and maybe try to integrate this
in actual TV programs now.
But yeah, so far it's only been
more or less within the project.
But the people that we did interview
they were very interested and they did ask us
to keep them updated about further developments.
- [Woman] Other questions?
Yes.
- [Man] Yeah, I was just curious about
broadcasts have commercials, so is this the kind of--
I mean, are they using that for that aspect as well?
For instance I could see an advertisement
on the main screen and then say a link on your tablet
to buy the product, is that something
that they may be pursing in the future?
- Yeah, it's definitely possible.
We've also, actually in one of the interviews
we had someone bringing up different ideas,
you could also integrate things like,
maybe a bit of a borderline idea,
but you could integrate something like gambling,
that's something one of them said.
I think it does make sense in a way,
yeah you have your, maybe a horse race on screen,
and you have your tablet and you can bet on horses,
whether that's a good thing or not I don't know.
But you know it's--