TITLE: [MUSIC]
RYAN BLISS: Hi, my name is Ryan Bliss,
a Product Manager on Teams platform.
PRADEEP ANANTHARAMAN: Hi, my name is Pradeep
Anantharaman
a Software Engineer on Teams platform.
RYAN BLISS: We're here to talk
about building
interactive meeting extensions in
Teams using our new live share SDK.
As modern workplaces transitioned from home to hybrid.
We hear a lot about the frustrations
that can come with digital collaboration.
In Teams meetings a glass screen sits
between you and the other people you work with.
With the new Team's live share SDK,
we've combined the power of fluid
with the social network of Teams.
Any front-end developer can transform
their web app into a delightful multiplayer experience.
PRADEEP ANANTHARAMAN: But another decision,
you should have a good overview of
live share SDK and scratch the surface
of building any collaborative app
through two sample walkthroughs;
a media watch together app and
an agile poker planning game. Let's dive in.
RYAN BLISS: For those who missed other announcements and sessions,
there are four key areas of the live share SDK.
First, matchmaking is automatic
and backed by the security of Teams meetings.
We do all the legwork so that
your front-end Teams can focus all their time
on building rich interactive features
that augment meeting collaboration.
PRADEEP ANANTHARAMAN: Next, we have made a new few items elsewhere that
object such as every one state and
events that can be tailored to meeting roles and context.
These are extensions built on fluid frameworks,
data objects that are transient and
ideal for scenarios where storing
objects may add complexity such
as laser pointers or notifications.
They are also the only fluid objects
that support meeting, role and verification.
RYAN BLISS: We've also built a tailored set of
media synchronization capabilities based on
our ephemeral objects to synchronize
audio and video files in just a few lines of code.
As people talk to each other during the call,
media audio is intelligently
reduced so that people can hear each other clearly.
PRADEEP ANANTHARAMAN: Finally, this previous service is fully managed,
scaled to all Teams customers and available today.
This service is backed by Azure Fluid Relay which is
currently in preview and
will be generally available this summer.
RYAN BLISS: Let's look at our first demo where we will walk through
a video template app available in our GitHub repository.
Pradeep and I are in a meeting where we
have a local app running on my computer,
tunneled through [inaudible] so we can both try it live.
As you can see, I have a meeting
SidePanel extension with a list of videos.
PRADEEP ANANTHARAMAN: Here. Let's watch this one.
RYAN BLISS: Awesome. I see it.
Since you're in control, want to press "Play"?
PRADEEP ANANTHARAMAN: Sure.
RYAN BLISS: As you can see, the video is now playing for me.
I got a notification
saying that Pradeep paused the video.
If I try and pause the video, it does pause,
but because I'm not the presenter,
it only pauses for me.
Pradeep, is this still playing for you?
PRADEEP ANANTHARAMAN: Yes. We add a feature called Suspensions,
which is especially useful for people
with disabilities like epilepsy.
That is important, they be able to post a video that may
trigger symptoms without
disrupting the rest of the group.
RYAN BLISS: Awesome. When I click "Follow Presenter",
it automatically returns me to Pradeep's position.
I'm going to take control now
so I can go to the next track.
The next video in the playlist is loaded and
you can see that the side panel updates as well.
PRADEEP ANANTHARAMAN: Let's talk through how we will do that.
First, let's review the Teams app
manifest changes needed to use
the live share SDK and related features.
If you aren't familiar with the Teams app manifest,
you can learn more at aka.ms/Teamsplatformdocs.
RYAN BLISS: At a high level, this is a JSON structure
that tells the Teams client what your app capabilities
are or permissions the user must consent to
use them and other information about your app.
Teams has a feature called tabs,
which allows you to build web applications that can be
pinned to various contexts in Teams.
The two that are supported by the Teams live share
for now are meetings SidePanel and meeting stage.
PRADEEP ANANTHARAMAN: There are also a few resource
specific authorization permissions worth noting.
First, to enable the live share SDK,
you must include live share as a delegate recognition.
RYAN BLISS: Second, if you want to use
the meeting stage tab context,
you must also have
the accompanying meeting stage permission.
But if your project requires
the meeting SidePanel context, then this isn't required.
PRADEEP ANANTHARAMAN: Finally, if you want your app
to support intelligent audio ducking,
which reduces video or audio volume
when synchronizing media playback,
then you should include the online meeting
incoming audio permission.
RYAN BLISS: In case you didn't see it in
earlier sessions of the conference,
here is just a simple example of how to
set up a Teams fluid client in just a few lines of code.
PRADEEP ANANTHARAMAN: The first thing you do at
any live share project is
define your fluid document schema,
which can find more information on from
their docs at fluidframework.com.
At a high level you're choosing
your shared multi-layer a set of shared objects.
RYAN BLISS: There is effectively no limit to
how many SharedObjects you define.
Ephemeral Media Session is
an example of a SharedObject that
we've created to simplify media synchronization in Teams.
PRADEEP ANANTHARAMAN: Next, we initialize our Teams fluid client,
get our container object
by calling join container method of
Teams fluid client with
the defined schema and you are
free to start with writing your app logic.
Let's see how this is done in
our media sample app. Okay, great.
Now that we have covered the basics,
let's pull up the code.
We're looking at the meeting stage page
where we have some hooks which are added
to cleanly separate the code
related to each object in our fluid schema.
RYAN BLISS: The SharedObject hook exports
each of the objects for a container.
In your project you could use
this hook as a template if you'd like,
but you certainly don't have to.
PRADEEP ANANTHARAMAN: Now let's look at the SidePanel.
RYAN BLISS: As you can see, the SidePanel is
also using the same SharedObjects hook,
but in this case only uses the playlist map object.
PRADEEP ANANTHARAMAN: This should allow us to selectively
synchronize the data between these two pages,
since each is actually
a separate view within the Team's client, right?
RYAN BLISS: Exactly. It's now much easier to use the meeting
SidePanel as a companion experience to your stage job.
Let's talk about the SharedObjects for a
second before we talking about the playlist feature.
The useSharedObjects hook has one job,
which is to create the fluid container and
export relevant SharedObjects to our UI.
We have two react state properties,
one for the fluid container results and another for
any error that may happen in joining the container.
The useEffect hook is responsible for the initial setup,
which in this case is when the app first loads.
Pradeep, can you talk through this
connection code we see here?
PRADEEP ANANTHARAMAN: One awesome thing about building
interactive experience through that SDKs is that you
can test either locally in your browser or in Teams.
In this hook, we check to see if that is
a URL parameter within Teams, etc.
Otherwise, we set up the local connection using
Azure Fluid Relay local testing client.
This is totally optional.
RYAN BLISS: Got it. I also see these Teams fluid client props,
which I see are also optional.
PRADEEP ANANTHARAMAN: That's right. We are defending
a fluid debugger which is helpful during
development as well as the connection setting
that get set when testing locally.
RYAN BLISS: If you prefer testing in Teams or disabling the debugger,
you don't have to define your props at all.
Next, we're defining an onFirstInitialize function,
which is another optional parameter.
This is really helpful for setting
any initial state when the
fluid container is first created.
Here is the only truly required part of
our setup where we define our fluid schema.
In this example, we have five components.
Pradeep, what do each of
these components do at a high level?
PRADEEP ANANTHARAMAN: The presence object is responsible for
tracking who is online and 11 moderator.
We'll talk about that in our second sample.
The Media Session object is
responsible for synchronizing of a media player.
The notificationEvent object can
transient push notifications to
everyone currently in the session.
Our takeControl map lets us track the presenting user.
Finally, the playlistMap tracks
our list of videos and which video is currently selected.
RYAN BLISS: Once this game is defined,
we initialize our Team's fluid client
during the container by passing in
the schema and optional on
first initialize callback and
then set the response to our upstate.
Once the container is created,
each object in our container will no longer be undefined.
So any components taking
a dependency on these SharedObjects can start using them.
Pretty straightforward. Pradeep,
can you give an overview on a fluid SharedMap?
PRADEEP ANANTHARAMAN: SharedMap is just one of
the building objects in the fluid framework.
Some of you may be already familiar.
SharedMap is a synchronized key value store,
very similar to our standard map or a dictionary.
RYAN BLISS: Once you've created your map,
you can start listening to changes
using the on valueChanged listener.
PRADEEP ANANTHARAMAN: Know that if two people are key at the exact same time,
conflicts are resolved based on last date events.
RYAN BLISS: Changing a value will trigger
the value change listener that we
defined earlier so that
each client can update their UI accordingly.
PRADEEP ANANTHARAMAN: SharedMap is an incredibly flexible director
that can be used for
almost anything you want to synchronize.
But there are a few caveats.
RYAN BLISS: All data stored through our hostage service is
deleted within 24 hours of container creation.
Don't put anything in the service
that you aren't prepared to lose.
PRADEEP ANANTHARAMAN: Another thing to keep in mind
is that these objects are not
compatible with the team's meeting notification.
Anyone this session can write to these objects.
RYAN BLISS: For most scenarios, that shouldn't be
a big deal, but it's good to know.
Anyways, let's see this in the code.
PRADEEP ANANTHARAMAN: We are now looking at the playlist hook
which takes in our playlist map object.
We have defined a few state properties
through reactState Hook,
including media items,
a list of videos in the chat playlist,
selected ID, the currently selected
video in our UI and a started flag,
which tells us when our event listener is
registered and the component is ready to go.
RYAN BLISS: How much of this code is
business logic as compared to SDK related code?
PRADEEP ANANTHARAMAN: Almost all of it. The awesome part of fluid is
how simple it is to synchronize any object.
Most of the code here are apps state,
computer properties and correct methods
be exposed to our app UI.
RYAN BLISS: Here is where we start
our set of logic.
First we add our Event Listener,
which refreshes our UI with
the latest information from the shared map.
We then get the initial values.
PRADEEP ANANTHARAMAN: The rest of those
files is as series
of methods related to
setting data to the map such as select media ID.
These each get exposed to the UI.
RYAN BLISS: Let's go back to the side panel page
and see how these are used.
PRADEEP ANANTHARAMAN: In our render section of our code,
we take each media item and
map them into a media card component.
When it was together button
on the media card is clicked,
we set the second video ID in a playlist map.
If sharing is active, we share the video to
the meeting state using the Teams SDK.
Otherwise, we skipped this step since
doing so would cause the playlist reload.
RYAN BLISS: Building synchronized objects into
your UI is as simple as that. But we aren't done.
Let's go back to the meeting stage component
to look at a couple of our other hooks.
The meeting stage page uses
the same playlist as a side panel.
But also uses all of our other objects.
Pradeep, can you walk us through the notifications hook?
PRADEEP ANANTHARAMAN: Sure thing. First thing
you'll notice is it takes a single input,
which is the ephemeral event object
written from a shared object hook.
It explodes a flag
indicating that it has successfully set
up the most recent notification
to display and send notification model.
The remote event object allows you to broadcast
out any JSON object to other client in the session.
After adding it to a fluid schema
and set it on receive listener,
which will be triggered every time
an eligible client also sent a letter method.
RYAN BLISS: The main difference between fluid shared objects and
our ephemeral ones is that
ephemeral objects have an explicit start method.
Here you simply initialize
it with an optional rolls parameter,
allowing you to choose which meeting rolls are
eligible to send events through this object.
PRADEEP ANANTHARAMAN: Let's look at this in our code.
RYAN BLISS: This code is pretty straightforward.
Similar to the sample,
there is a send notification callback method
which sends a JSON object with the text string.
In our received event listener,
we update our local app state with
the notification display formatted
differently for a local user than a remote one.
We then call the start method,
set started to true and
send an initial joint notification.
PRADEEP ANANTHARAMAN: To enable media synchronization in our app,
let's add our ephemeral
media session object to our fluid schema.
Then we create our media synchronized by passing
our HTML5 video or
audio element into the media session object.
RYAN BLISS: Next, let's define the roles we want to allow
playback control and call the start method.
That's it for synchronizing
the player for non controlling participants.
But there is one more important distinction.
PRADEEP ANANTHARAMAN: Most modern web browsers barren designed to
distinguish user initiated actions
such as pause and play.
This means that events such as buffering are
indistinguishable from a user pause action.
RYAN BLISS: To avoid this, it is important that
our media synchronizer be able to intercept user actions,
including pause, play, seek, and set track.
PRADEEP ANANTHARAMAN: If your project doesn't have custom media controls,
we have template available in
our GitHub repository based on flow into it.
RYAN BLISS: Let's look at this in action.
The media session hook is
responsible for synchronizing our media player,
which in this case is powered by Azure Media Services.
Pradeep, can you walk us through this?
PRADEEP ANANTHARAMAN: First, let's go down to where
we set up the media session and synchronizer.
If the local user isn't the current presenter,
we set the synchronizer to view only mode which blocks
synchronous playback methods from
being applied to the player.
Next we mute the player
ready for it since we don't capture
the click needed in some browser before playback starts.
For your app, you may want to investigate
more creative ways to work around this requirement.
Now let's look at our playback control methods.
Since most of these are similar,
we won't look at all of them.
Let's look at the past callback.
If the local user is the current presenter,
we call pass through the synchronizer
and a notification is sent to the other users.
Otherwise, we temporarily suspend
synchronization for the local user and pause the video.
Let's now look at how this gets right up to our UI.
RYAN BLISS: Our app uses Azure media player which has
a slightly different interface than HTML5.
We created a simple shiv that makes
Azure media player match the interface of HTML5.
If you use a different player,
you can follow the shift pattern
established in our documentation.
When all our components are set up,
we stop showing the team's loading indicator
through the team's client SDK.
Finally, we have a media player container with
a video element as a child per
the Azure Media Player Guidelines.
We then pass in relevant information for the player
controls such as the Azure Media Player object,
whether playback is suspended and our
callback method such as play and end suspension.
That's all we needed to make
a robust media synchronization experience in Teams.
Next, let's talk through an Agile Poker demo app,
which uses a planning methodology used at Microsoft.
I am a grandmaster trying to
determine the amount of time it will
take to build a watch together
experience for a user testing app.
I click the "Plan Together" button,
which shares it to the meeting stage.
I then click "Start game" and
Pradeep and I can guess the cost from a list of cards.
I'll make my guess and then click
"Submit" and Pradeep will do the same.
Pradeep, what did you guess?
PRADEEP ANANTHARAMAN: I guess five story points because while we
have some good templates available to us,
we still want to make sure the experience is good.
RYAN BLISS: That makes sense. I guess lower because of
the templates but I think you're right. Let's try again.
Now that we have consensus,
I click "Finish" and the side panel
is updated with the agreed value.
This app works similarly to this media sample.
We won't cover all of it but let's look
at a couple of new shared objects from the sample.
PRADEEP ANANTHARAMAN: Our poker app uses a new concept
called EphemeralState which allows for
a stateful object to last
as long as one client is contact to the session.
RYAN BLISS: Like our other components,
you must first define poker state in our schema.
We can then listen for a state changed event
which exposes three properties;
state, value,
and a local user flag.
PRADEEP ANANTHARAMAN: Next we start synchronization
by choosing which rows can change the state.
This component is a great alternative to
standard fluid objects in cases
where you still want notification.
Let's see how we use it in our app.
RYAN BLISS: At a high level, this app is composed of
three primary states that
only the meeting organizer or Scrum Master can change.
Each map to corresponding UI.
If we scroll back to the top,
you can see that we've used the same shared objects hook
that exposes relevant fluid objects used in the sample,
including a poker state presence,
timer, and user stories map.
Pradeep, can you walk us through pokerState?
PRADEEP ANANTHARAMAN: The pokerState react with
manages from EphemeralState object.
This hook is responsible for managing
the current step of our planning activity,
exposing the current state,
selected user story, and callbacks.
Looking inside the hook,
our set up looks a lot like the slide we showed before.
Inside of our listener,
we check to see if the state is a valid
one then update our UI with the latest state.
We restrict state changes to
only the meeting organizer
and start listening for state changes.
This is what our change state method looks like.
RYAN BLISS: Next, let's talk about
another Ephemeral object called EphemeralPresence.
Presence makes it easy to track who is online and
assign user attributes such as name and profile picture.
PRADEEP ANANTHARAMAN: The important thing to know about Presence is that
each user essentially posts metadata about themselves.
This means that nobody can
synchronize data for another user.
RYAN BLISS: Presence updates are sent at
a default interval of 60 seconds.
If no presence update is sent after two intervals,
their status will automatically switch to offline.
PRADEEP ANANTHARAMAN: Like other objects, you must first define
everyone's presence in your fluid schema.
You can then listen to Presence change event
which exposes a user Presence object and a local flag.
RYAN BLISS: When you start Presence, you can obviously define
a user ID and a custom data object.
To change data about the user,
you can call the update Presence method.
PRADEEP ANANTHARAMAN: Let's look at how we use Presence in our sample.
RYAN BLISS: The Presence's hook takes in two parameters,
the Presence object and user ID.
In this case, from the Get Context API
and the Teams client SDK.
This returns our list of users,
the number of users who have already shown
up during the costing phase,
a flag indicating whether to
look easier is the Scrum Master and
callbacks for updating user attributes.
Let's check it out.
PRADEEP ANANTHARAMAN: Like any Ephemeral object,
our set of logic is composed of
an event listener and a start method.
As we mentioned earlier,
our app relies on one person,
the meeting organizer being in charge of training states.
We get roles for our local user.
We will then filter out a list of users to those
online and then update our UI accordingly.
If we want our presence to update more frequently,
you can always change the interval.
We then assign different user attributes
and start Presence tracking.
Regarding the card banks exposed to our UI,
they all follow similar patterns.
Let's look at the report answer method.
RYAN BLISS: During the costing state of the game,
we request each person estimate
the user story cost associated with the given user story.
When the cost one end is over,
either because the timer runs out
or each user has ready to up,
each client reports their answer using this method.
This data is then used to render the answer
for each user in the discussion phase.
With Teams sharing fluid,
frontend developers can build
incredible interactive experiences
without complex backend work.
PRADEEP ANANTHARAMAN: We can't wait to see what you build
and thank you so much for attending our session.
If you want to learn more, you can
find our docs and samples
on GitHub at aka.ms/teamsliveshare.
RYAN BLISS: You can also tune into other related sessions.
Hope you enjoy the rest of your day.