So my name is Wale.
I'm the technical lead for the Android Window Manager
and Activity Manager.
Today I'm going to be presenting what we're doing
to support multi-window mode.
It's going to be a relatively short presentation, mainly
because I want to give you all a chance, at the end,
to ask any questions you may have
relating to your specific use in multi-window mode.
OK, so multi-window allows multiple activities
to be visible on screen at the same time.
The activities are resizeable from the full screen area
down to a minimum size, as allowed
by the system or activity.
In the end, we're going to be supporting three
forms of multi-window mode.
Split-screen multi-window mode, with two activities,
obviously, side-by-side.
And a divider in the middle can be
used to give more or less screen space
to one activity or the other.
Picture-in-picture multi-window mode,
where an activity is z-ordered always
on top of all of the activities on screen.
And freeform multi-window mode, where
multiple activities are visible on screen at the same time.
They can be moved around freely and also resized
to any average area dimension.
So some important things, in the end, to keep in mind--
the Android activity lifecycle model
is not going to be changing in multi-window mode.
We're still only going to have one resumed activity
in the system at a time.
And all other visible activities are
going to be in the paused visible state.
There are several reasons we are keeping this model on Android.
And one of the main reasons is, being in the resume state
is an important indicator to the system
on what kind of resources an activity can acquire,
because it, kind of, has highest important score.
For example, the camera service discovers singleton resource.
And the resumed activity is always
going to get that resource compared
to all other activities that are in the paused state.
Another important thing to note is
that we are now also factoring in the low memory score
adjustment of visible activities in the paused state
using a z-order.
For those of you that don't know,
the low memory score just got on Android.
It's basically an important score given
to all processes in the system.
The higher your score, the more important
you are to the system.
And the more the system will try to keep you alive.
And when the system is running low on resources,
it kills things that are of lower importance score first.
Right, so for example, you can have a service showing
in your background, and you cannot be resumed activity.
And the system-- when it's running low on resources--
would kill these services running in the background
first before the resumed activity that the user might
be interacting with.
So similar for visible activities
that are in the paused state-- we,
kind of, rank them in order of their z-order
and give them an appropriate low memory score adjustment.
So if the system is really, really, running low on memory
and it needs to kill visible activities,
it's going to start from the one with the lower z-order first.
However, when the system does that,
we do preserve the current window on screen
so it's still visible to the user.
And we apply it dimly on top of it.
Once the user interacts with the app again,
we then bring it back to life.
Something else that is important to note
is that, as of N, since it isn't desirable for video apps
to stop video playback when they are in the paused state,
the new recommendation, as of N, is for video apps
to handle play/pause actions in-- on start on stop
versus on resume on pause.
While it has always been the case in Android
that a paused activity can still be visible,
it has not been as visible to the user
as it will be with multi-window.
So it's a message right now pushing for it
a little bit stronger now that-- on what the various lifecycle
stages represent.
When you're on start, it means you're
going to be visible soon.
On resume means you are visible both and focused
On pause, means you are still visible but not focused.
And on stop, means you are no longer visible.
So another thing to note is that as the user resizes you up
in multi-window mode, we do issue configuration changes.
However, we only limit these configuration changes
to the configurations that would cause new resources
to be loading you up.
So, for example, if you are past two buckets
where it has a layout representation for portrait
and one for landscape-- and as the user resizes you up
in portrait mode, if they are not crossing a threshold that
will cause it to switch orientation,
we do not issue new configuration changes
or cause the app to relaunch.
However, once they cross that configuration threshold
and new resources need to be loaded,
we do reissue the configuration change for landscape mode.
It's also important to note that the configuration
change that I'm referring to here
only applies to the activity on screen
and not the overall device configuration.
So the device can be in landscape mode
while your activity in multi-window mode
can be in portrait mode.
Also, as you do today, you can continue
to undo configuration changes to avoid relaunches.
In order to avoid relaunches in multi-window mode,
you need to undo all the configurations
you have on screen.
Screen size, smaller screen size, screen layout,
and orientation.
If you choose not to undo the configuration changes,
you will be relaunching and resizing.
However, we do preserve your current window on screen
until you render your content into a new window.
And we'll replace it seamlessly, so
that the experience to the user is not jarring and appears
as smooth as possible.
So there are several new APIs we're
going to being introducing in order to better support
multi-window mode.
I'm going to be going over a few of these APIs.
So the big one is the resizeable activity manifest attribute.
This is an attribute that you set to indicate to the system
that your activity supports multi-window and can be resized
to any operator dimension.
It's important to note that we enabled this activity,
by default, for all apps, including the ones not
targeting N. And you need to explicitly tag it N
and set it to false in order to disable-- not that we recommend
that, though, for all apps.
Another important thing to note is
that the root activity of a task determines
the resizeability of that task.
You can think of a task as every single card entry
you see in the recent activity or [INAUDIBLE] activity.
So if the root activity is marked as resizeable,
any additional activity that goes into that task
is also resizeable.
However, if the root activity isn't resizeable, then
no other activity in that task-- the task will not be resized.
So no other activity in that task will be resized, as well.
Another cool new API we are adding
is the launch adjacent API.
With the launch adjacent flag, you
can launch an activity adjacent to yourself
while in split-screen multi-window mode.
For example, I have on screen the Chrome activity on one side
with an address link.
An implementation could be done where,
once you click on that address link,
the map activity is launched adjacent
to the Chrome activity.
So an activity can also use the layout styleable,
mostly in freeform multi-window mode,
to set its initial or default launching
position on size on screen.
You can set things like your default height, weight,
gravity-- whether you want to be on top, center, bottom--
what the minimum dimensions you allow the user to resize
your application to.
So in multi-window mode, as the user resizes your activity,
the layout of your app can be slow depending
on your implementation, right?
So what a user would notice is, kind of,
an effect where they perform an action in the old way
here, right, and the app is still, kind of,
taking time to re-layout to catch up to the user action.
That experience is not too desirable.
So what we're actually doing in N,
is we introduce a second UI thread in the application
process to draw a background color that
keeps in line with the user action
as the main app's UI thread takes time
to render content to catch up to the user action.
You can use the window background styleable
to customize what this color is.
So picture-in-picture mode, while it's
desirable for most apps to support
picture-in-picture-- to support multi-window mode--
it doesn't make sense for all apps
to support picture-in-picture mode multi-window.
For example, you would expect an app like Netflix or YouTube
to support picture-in-picture mode.
But you didn't expect an app like Google Docs
to support picture-in-picture mode.
So to differentiate this form of multi-window mode,
we have the supports picture-in-picture mode
manifest attribute for the app to indicate
that it can go into this mode.
With apps that have this flag set, on devices
that support picture-in-picture mode,
they can call the enter picture-in-picture mode API
to put themselves into picture-in-picture mode.
A few things to keep in mind with picture-in-picture mode
is even though the activity is z-ordered
on top of all other activities in the system,
it is actually in the paused visible state.
And the activity that is full screen
below the picture-in-picture activity
is, actually, in the resume state.
And the main reason for doing this
is, once the user puts your activity
into picture-in-picture mode, they really
want to watch your activity to the side
while interacting with the full screen activity.
So it makes sense for the activity
to be focused and resumed while your activity is
in the visible paused state.
It's also important to keep in mind that you can continue
to use the media session APIs we have today while you're
in picture-in-picture mode to keep track of media buttons,
like play/pause actions and whatnot.
We also have APIs that allow you to get notified
whenever the system or the user puts you
into multi-window mode, or whenever
you exit multi-window motor and picture-in-picture mode.
And you can also query the system
on your client's window mode.
In freeform mode, activity is allowed to display content
next to the caption/title area using the overlay with the call
caption API.
You can kind of think about the Chrome tabs
you have on your computer where the Chrome
tabs are next to the window control buttons.
With this API, you can, kind of, do something similar.
Also in freeform mode, an activity
can launch another activity at a specific location and size
on the screen using the activity option set launch bounds API.
So currently on Android we support
several APIs that affect the entire screen configuration.
You can think of the nav bar, status bar customization,
for example.
And this customization for the entire screen
would have conflict in multi-window mode,
especially when apps require conflicting configuration.
So through that effect, we have most
of this configuration as disabled when the apps are
in multi-window mode.
I have some of the examples listed on screen,
like the system UI customization for status bar,
nav bar-- show wallpaper behind-- immersive mode,
and setting orientation.
Something important to note with setting orientation--
once you say your app is compatible with multi-window
mode, having a fixed orientation doesn't really
make sense in that world anymore,
since the user can, kind of, resize
your app to any dimension, and it can change orientation
while changing that dimension.
So it kind of makes sense for that to be disabled,
regardless of the conflict you might have.
So some UX best practices and guidelines
for multi-window mode include, you
will provide a single responsive UI for your application
so that the user has a consistent experience
while using your application on the same device
at different sizes.
You'll start by designing your app for the most challenging
use case so you can uncover architectural problems early
in the design process.
And you'll follow the material design patterns
so that your app has a consistent look
and feel with other apps running on the system.
And we-- you want to only disable multi-window mode
when justified.
It's-- it would be odd if most apps on the user device go
into multi-window mode.
And your app is the odd one out.
And the user launches it, and it launches into full screen mode.
And look for various places to take advantage of all the APIs
we're exposing for multi-window mode, like the launch
adjacent for appropriate use cases.
So some more guidelines and best practices--
we recommend that apps with similar layout for phones
and tablets may use the phone tablets on-- I'm
sorry-- the phone UI on tablets, since the experience would not
be too jarring to the user as they transition from one size
to another.
However, for us, it's a completely different layout.
We recommend that the phone UI shouldn't be used.
Instead the tablet UI should be made
to work at its smallest size and scaling up from there.
The main reason for these recommendations or guidelines
is to, kind of, give the user a consistent feel
and experience as they resize your activity
in the same device.
So to simplify adapting your app for multi-window mode, what
we've found is it's better to design your app
for the smallest size first.
If you can get something working with 220dp wide or tall,
you can-- by condensing elements or removing
non-essential ones-- then you can, basically, scale up
your apps you sketched from there, adding things as needed.
A nice complementary feature to multi-window mode
is the system-wide drag and drop API.
With the Android drag and drop framework,
you can allow activities to share data
using the graphic called drag and drop gesture.
Some of the basic API for drag and drop
include, start drag and drop, update drag shadow--
the drag shadow is, really, the thumbnail
you will see when you are actually dragging--
and cancel drag and drop-- pretty cool API.
OK, so I said it's going to be a short presentation.
And I would just like to end by saying,
the Android framework-- we've done a whole lot of working
in to add multi-window mode and to make sure the APIs we feel
will be useful to our developers and the UI
are supported at the platform level.
However, that's just half the work.
We really depend on the app developers
to start picking up this feature up in the N preview,
to start using it, testing out your app,
making sure it performs less awkwardly, and resizes.
See what kind of problems you come up with, and providing us
feedback on things we can improve in the platform
to make your life easier.
It's kind of a two-sided effort to get this feature up
to a level where it really brings joy
to the-- to all Android users everywhere
as they use multi-window mode.
So, yeah.
So that's all I have for the presentation.
I have some useful links up on screen.
There are mics on all the hallways
if you want to ask any questions on the topic.
Thank you.
[APPLAUSE]
[MUSIC PLAYING]