Hi, everyone. My name is Kenneth Ford. I'm a developer advocate here at Google
working on Android apps on Chrome OS foldables,
and other larger screen devices.
In this session, we'll talk about
bringing your app beyond just the mobile phone
and the things you can do.
Specifically talking about tablets, foldables, as well as Chrome OS
and other desktop environments.
So some of the major differences on these devices
than your normal mobile phones is the screens are just much larger.
And your users will want to take advantage of this
when they use your app on these devices.
You'll also see that people are using these devices
in very different contexts.
They'll be using these devices for much longer periods
and much deeper times in each application
versus a phone that's maybe a much quicker,
pull the device out, get something done quick,
and then move on.
A lot of times for tablets and Chrome OS and desktop environments, especially,
they'll be a lot less on the go.
They'll be wanting to get more done and have, like,
deeper experiences in each app.
So let's look at an overview of what we'll talk today.
We'll talk a little bit about app continuity
and what that means for your application.
Some UX UI considerations when thinking about bringing
your app to these devices:
multi-window and some of the challenges and things to think about.
Input changes
and some different input mediums that are used,
and finishing off a bit with some emulator tools that are being released.
So, what is app continuity?
App continuity is the idea of the app continuing
when a user unfolds or folds the device.
And this also follows to other configuration changes
such as orientation and window size changes.
Things like scroll position, video playback timestamps,
text the user has entered should all continue
as if nothing happened.
Configuration changes, while something
that we know the apps have ran away from in the past,
you can't really avoid them anymore
due to the devices actually just changing states,
versus being able to lock orientation or lock resizability.
But this concept is not just a problem on foldables.
Chrome OS has a native-free form windowing system,
as well as tablets and other larger screen devices,
are shipping with free-form windows natively.
Resizing has the same challenges as folding and unfolding
in that you'll run into configuration changes
and window size changes.
Multi-window is used much more on these devices, as well,
at least, speaking with tablets,
whereas on Chrome OS and other desktop environments,
it's actually the native use case.
We've seen studies that show that people want to use apps and multi-window more
as the devices get larger.
Seen as a use case that wasn't used very often on smaller phones,
these devices are really showcasing
both the great things of apps and multi-window
as well some of the challenges.
So, these changes should be seamless to the user.
When they open or close their device or resize their window,
they shouldn't be confused or shocked
about what happened in the application they're using.
Or even in some of the worst cases, we see apps that reset the user
all the way back to the starting screen of the application.
You should be supporting different window sizes and orientations.
While with a phone, most people generally use the app in fullscreen and in portrait,
some of these devices are landscape first,
as well as just having a much higher usage in landscape
compared to a phone.
Some of the foldables, also with their different aspect ratios
and being closer to 1 to 1,
you'll see people using the devices and landscape
much higher because it's not that different
of a screen setup.
Now, you can use layout tags to keep your windows
and arrange that you support if you feel like this is an issue.
And this is the better solution than just locking
to full screen and being nonresizable.
You should handle screen size changes gracefully.
Now, this isn't just saving state or saving video playback time.
You should also be laying out things smoothly without a ton of jank.
The OS can handle your re layouts fine
if you are letting the OS handle things during configuration changes.
But if you do feel like you need to do things manually in your views,
one approach we've seen work well with partners
is to have your views just subscribe
to some type of global object that you have
that's reacting to these window size changes
and passing those directly to the views.
So let's talk about some of the UX and UI considerations
you get to think about when bringing your app to these different devices.
The first is some of the devices just natively are a different aspect ratio
from what you're probably used to supporting.
Some of the devices are as extreme as 21 by 9,
as well as closest to 1 to 1.
And this is just not the traditional kind of phone screen size that you're used to.
On top of that, free-form windows
allow you to be in almost any window size or aspect ratio.
The min/max aspect ratio tags in your manifest
allow you to limit the barriers that which you feel
like your app best performs.
And, again, this is recommended to just locking as nonresizable.
Another thing to think about is certain navigation patterns
don't always work best for every size.
Specifically bottom navigation does not stretch super well
when the window size gets very wide.
Let's look at the material study Owl and see how they handle this.
So on a phone, bottom navigation looks and works really well.
It's easy to access by the user on a phone device
with whatever hand they're using.
But when they get to a wider desktop screen,
they move it to a navigation rail
so that users can handle it and access it with their hands,
as well as just being in a more native location
that they're probably expecting.
People are used to the left side navs on these bigger devices,
as well as on a tablet, it's very close to their hands,
whereas a bottom nav would be very stretched out
across the bottom of the screen
and it'd be very hard to hit items in the middle.
The navigation rail spect has been posted on the material website.
So you should see if this is something that you could bring into your app.
And the Android component is in progress.
You should think about how to adjust your layouts
for these larger windows and screens.
Think about how your content should scale
and really think about how you can bring columns into these layouts.
One of the things that we notice is, a lot of times content just gets scaled
across the entire display.
And it becomes more of a layer cake design,
which works really well on phones,
but doesn't necessarily scale when the devices
and windows get very large.
A couple things that you can think about as well as
think about how you can make things easier for the user.
Hiding options behind menus is great on a phone
when you don't have a ton of screen real estate.
But it's unnecessary when you get to a wider and larger screen.
Your brand will dictate what's best in these situations.
So it's hard to give very prescriptive advice.
But let's look at a bit of a chat app.
So the chat app here clearly moves to something
like a chat detail view.
So you can see your chat list on the left,
as well as possibly some description or the last message in that thread,
as well as having the main chat that you're looking
as fullscreen in the middle.
Now, when your screen gets very large,
you could even pull some stuff out to a third column,
whether it's like different interactive elements
that you and the person you're chatting with can do
or just options for the current chat that you're looking.
And you should also think about
new experiences that are available for foldables and hinge devices.
These devices bring a new opportunity for experiences,
as some of the devices support different states of the device angles
that you can react to.
And so an example here is a video watching application.
Where window device is folded at a 45 to 90-degree angle,
you can display video content on the top half of the display
so it's very easy to look at for the user while having controls
and maybe playback options on the bottom half.
You can also see this in things like photo editing or video editing apps
where you can have the timelines and other editing options on the bottom half
while the video or photo you're editing
is very easy to see and look at all the changes that you're making.
You can handle this with the Jetpack Window Manager library
that we release this year.
In this image, you can see how duo is splitting the video call on it
to make it easier to use, freezers
to just put their phone on the desk and easily see the person
they're chatting to.
And you can check our intro blog post
for more information on the library and how to use it.
Multi-window brings in certain challenges as an app developer
and one of them being activity life cycles.
In Android 10, we brought in the concept of multi-resume.
All apps in this multi-window state are now in a resumed state.
So content will continue to play or update.
This does bring in a challenge for certain applications
that use exclusive resources such as the camera and microphone,
as you may have multiple apps visible and in a resumed state
that are trying to access these at the same time.
To handle this, we added the new top-resumed activity callback.
You would use this callback to try and grab the camera microphone
when you are the top-resumed activity
and knowing that you are going to have access to these.
You'll find out when you're no longer the top-resumed.
And so you should gracefully handle losing the microphone or camera access.
Being nonresizable does not guarantee these resources,
as there may be a floating activity on top of you that has access to these.
Drag and drop is a way you can bring a really optimized and native experience
to your application on these sorts of devices.
When screens get bigger,
people are using apps and multi-windows substantially more often.
And they're looking to be much more efficient,
whether it's dragging images from a web browser
or from a photo editing app into a chat message
or quickly dragging a restaurant name to share with some of their friends.
It's a way to really give a native and awesome experience to your users.
And we're seeing it try to be used more and more
as the devices get larger.
the next thing to think about and consider
when looking at your app's experience on these devices
is how you handle input mediums.
The first thing to realize is that mouse and keyboard are first-class input mediums
on a lot of these devices.
Chrome OS devices natively ship with a mouse and keyboard,
as well as some other desktop platforms that OEMs have created.
People will be using your app
with a mouse and keyboard and expect it to work.
Some of these devices may not ship with a touch screen.
Or they may not be using your app on a monitor or display that offers touch.
Making sure things work out of the box is really important.
Your list should be able to scroll with a mouse.
People should be able to navigate through your app
in some regards with a keyboard
and making sure that all items are selectable with a trackpad,
a mouse, or even a stylus.
When you talk about optimizing for these input mediums,
you should think about supporting the native behavior that people expect.
Things such as right-click, hover states,
as well as common shortcuts, especially if your app on other platforms
support specific shortcuts that your users expect to be able to use.
Right-click is something that everyone knows how to do
and expects things to work with a mouse.
Thinking about how you can move your long press options
to different context menus
based on if people are using your app with touch or a mouse
is very important.
And hover states is something that we find
really enhance the app's experience when using a mouse or trackpad,
because it lets users know what is actionable and what is not.
So one way you can test some things that we mentioned in this call
is by using the new free-form emulator that we announced.
So this is a native Android emulator image
that has free-form windows on by default.
So you don't have to mess around with the developer settings.
This is going to allow you to easily test things
like app continuity, window size changes,
and other configuration changes while keeping the same developer workflow
that you're used to.
And we also wanted to announce the launch of ChromeOS.dev,
which is our new Chrome OS developer site.
This site will have information around optimizing your Android app
as well as the different capabilities that we have
to actually natively develop your Android app on Chrome OS
and push directly to the container,
allowing you to test a lot of these different input methods
or resizing changes much easier.
Thanks for watching.
I can't wait to see the experiences that you bring.