[MUSIC PLAYING]
AMANDA ALEXANDER: Hi, my name is Amanda.
I'm a product manager on Android.
Today, my colleague Sean and I are
going to discuss all things emoji on Android, starting
with, first, why emojis are important.
Second, what recent changes Android
made, including platform-level changes,
to our font system and Jetpack library changes
to address issues with emoji.
And third, how it all works.
First, you may be wondering why we're focusing on emojis.
What are the issues with emojis that need to be fixed?
Well, we know that emojis are becoming
more and more a critically important part
of communication.
Emoji usage consistently has grown over the last 10 years,
and this year, global emoji usage
has reached an all time high.
Users on some apps send billions of emojis every day,
and more than one in five tweets now contain an emoji.
Now, one of the specific problems that we're facing
is that when a new Unicode emoji is launched,
96% of Android users do not get to see the latest emojis.
This is significantly higher than iOS users, where only 16%
of users don't see the latest emoji.
Another interesting data point is that the top 100 emojis
represent about 82% of all emojis that
are shared between users.
Unfortunately, about 20% of the time,
when a user sends an emoji, the receiver
sees a tofu, or a broken image, rather than the actual emoji
the sender intended.
So for example, on the screen, you
see that the sender wanted to send a smiley face with a tear,
and instead, the recipient got a box with an X
in the middle, also known as a tofu.
And the intent of the message is completely
lost on the recipient.
Another problem is that, as Unicode adds new emojis that
aim to represent more diversity and inclusion of emoji types,
those emojis are not fully supported on Android.
There are various levels of support depending on which
version of Android is running.
Let's take a look at some specific issues.
On Android N and earlier versions,
skin tone is not properly represented.
Here, you can see an example of a user sending an emoji
representing an arm with a brown skin tone,
but unfortunately, the receiver gets a decomposed version
that shows an arm and a square, representing
the skin color that should be applied to the arm.
Here is an example of an issue on Android O
and earlier versions of the platform, where
gender-inclusive emojis are not properly supported.
And on Android P and below, multi-person multi-skintone
emojis are not supported.
As you can see on the screen, it is not a good user experience
for the recipient and hurts communication
between the sender and the receiver.
Our goal is to make sure that every emoji is
available to all Android users, regardless of the app
that they are using.
Whether it's a social app or a banking app or any other type,
emojis are used in so many different contexts within apps
that we want to make sure that users can use them
with no issues.
To address these issues and accomplish our goal,
we have two solutions.
First, updates to the Jetpack library EmojiCompat,
which is the backwards compatible solution
for all the platforms and covers what app developers can do.
And second, updatable system fonts,
which is a forward-looking solution for S
and future versions of the platform.
Next, Sean will walk you through updatable system fonts
and changes to the EmojiCompat.
SEAN MCQUILLAN: Thanks, Amanda.
Hi, I'm Sean from the Android text team,
and I'm here to share a bit about emoji
in your application.
First, some really good news.
Starting in Android S, we introduced updatable system
fonts, and the first font that we'll update is the emoji font.
Now this is really good news as a developer because it means
you never have to think about emoji again on S+ devices.
It's just taken care of for you.
But we can't really use a time machine
and pour updatable system fonts to earlier versions of Android.
We're still working on the time machine,
but for now, if you do nothing, in your app,
you'll display tofu, or a broken emoji,
on Android before S. To fix that, you can use EmojiCompat.
We launched this library a few years ago,
and this year, we put a lot of work
into making it easier for you to use it in your application.
In AppCompat 1.3, if your app was
asked to display a modern emoji, you'd get tofu or a misrender
by default. And we wanted to make
this a better experience for both you, the developer,
and Android users.
So, in AppCompat 1.4, we incorporated EmojiCompat
into AppCompat, and this is really big news.
Out of the box, all apps using AppCompat 1.4
will support modern emoji.
To do this, we introduced a new artifact, androidx.emoji2.
We added automatic configuration so it can configure itself
to load the correct emoji font.
And then with that, we integrated it
into AppCompat 1.4, and that's really it.
This slide is all you need to know.
Just bump your AppCompat version to 1.4,
and you'll display modern emoji on API 19 and above.
You don't have to configure it or do anything.
It'll all just work out of the box.
And that's really it.
The rest is just details that melt away.
The rest of this talk is those details,
and you have my permission to click to the next talk now.
I'll wait.
Just remember to upgrade AppCompat.
Oh, hello.
I assume if you're still here, you're
interested in all those details.
Emoji2 is a new artifact that replaces androidx.emoji.
It has pretty much the same API as androidx.emoji, just
in a new package.
We added a few new classes to it, a startup initializer
and a default configuration, and while we were in there,
we added nullability annotations throughout.
Compared to androidx.emoji, we also
removed a few text use subclasses
that aren't needed when using AppCompat,
and this let us save about 14 kilobytes after RA.
Now, one of the big new features in emoji2
is the EmojiCompat initializer.
This is a startup library initializer
that automatically configures EmojiCompat at app startup.
Now it's heavily tuned for performance, and for most apps,
we recommend using the default configuration.
However, if you've already profiled and squeezed
every nanosecond out of app startup,
there's a few corners you can cut.
Now, before we continue, note that we
are talking about microseconds, so this
isn't likely to be a useful optimization for most apps.
But if you do have extensive profiling,
you can remove the startup library and thread creation
costs.
Before you go down this route-- and fair warning,
it's a bit of code-- here's a few tips.
First, make sure you initialize EmojiCompat
prior to ActivityonCreate.
This ensures that every text field will be
able to display modern emoji.
Then, just like EmojiCompat initializer,
it's a really, really good idea to defer emoji font
load until well after your first screen paints.
It does run in a background thread,
but it also does a lot of network and disk I/O--
not the sort of thing you want contending
with your first screen load.
All of that said, for most apps, we strongly
recommend using the default implementation
of EmojiCompat initializer.
It is highly tuned for performance
and should only be optimized after you fix everything else.
Another big feature we added is default configuration.
In androidx.emoji, you had to go find
a bunch of boilerplate configuration in a sample
and copy it into your app.
No more.
We added default EmojiCompat config which you can use.
It's used by EmojiCompat initializer,
and it's also available if you're implementing
a manual config in your app.
In AppCompat, we integrated emoji2 into all views.
This means you can just use a text view or edit text
and it will display modern emoji by default.
All views with emoji2 integration
have an EmojiCompatEnabled attribute
that you can use to control, well,
if EmojiCompat is enabled for that view.
This attribute is also exposed via getters and setters.
Just using text view or edit text in XML
works if your activity extends AppCompat activity.
AppCompat activity installs a layout inflater
that replaces text view with AppCompat text view and so on.
In code, you should ensure that you create an AppCompat
text view whenever you'd create a text view,
and custom views should subclass their appropriate AppCompat
subclass.
The EmojiCompatEnabled attribute is
useful to avoid emoji processing in situations where you know
text may never contain emoji.
Now, actually, emoji processing on a string
is incredibly cheap when there's no emoji,
but there are some situations where
every single nanosecond counts.
And this knob is for those times.
It's also useful if you want to process emoji
on a background thread.
All the AppCompat emoji2 integration
does is call EmojiCompat.process at the right time
after setText.
And you can disable this and move emoji processing
to a background thread.
Now, typically, this isn't needed because emoji processing
is fast enough, but if you're displaying
a lot of text in a recycle view and getting
a little bit of jank, this is an optimization to consider.
Testing modern emoji is a bit more complicated.
Due to the downloadable fonts integration,
it's hard to make a general automated testing
library that doesn't lead to either false positives
or false negatives.
And in practice, we also recognize
that most developers will just test EmojiCompat integration
manually.
So the best option is to just share
a list of emojis to test with, and that's exactly what
we did in the new docs.
They're really useful for both manual or screenshot testing.
And there's some tips for what you
need to do to configure a test emulator or device in the docs.
And that's a deep dive into what's new in emoji2.
If you're already using androidx.emoji,
we encourage you to switch to emoji2 shortly.
The APIs stayed mostly the same and are just in a new package.
The EmojiTextView and friends that were in androidx.emoji
were moved to a new emoji2 views library,
and they're really not needed if you're using AppCompat.
OK, so upgrade to AppCompat 1.4 and that's really it.
That's all you need to take away from this talk.
Before we wrap up, let's talk a bit about how emoji2 works.
This is an emoji.
Actually, no.
This is an image that I added to a slide,
and that's kind of what EmojiCompat does to emoji.
This is an emoji.
An emoji is a graphical character
that's part of a string.
It's just like the letter "I", except it
draws quite differently and it comes from a different font
file.
The thing is, computers don't really
know about emoji or really the letter "I", for that matter.
As a view from the orbit of Pluto of what a string is,
it's really just a list of code points.
Now these are numbers that are assigned
to every character you've ever seen on a computer by Unicode.
Now Unicode isn't just a format.
It's also a mailing list in meetings-- so many meetings.
Meetings where they decide things like the number 7
is actually the letter "I".
Now, before we move on, 7 isn't actually the code point for I,
but this slide looked better this way.
So what happens when you try to render this string on Android?
Put simply, as a first step, the platform
has to figure out the best font for each character.
The inputs to this are the code point as well as
any font styling requested by your app.
Now the default font for V at a normal weight that's not italic
on Android is roboto-regular.ttf.
Android iterates through the string,
checking each character, and finding the best font.
It'll check the code point and any styling,
and you can mix and match the same string,
making some characters bold and so on.
However, for this simple string, it's always going to pick
roboto-regular.ttf.
Now, when it gets to the emoji, you
might imagine it does something completely different.
After all, it doesn't look anything
like any of the other letters on the screen.
But emoji are just text.
They're represented by code points,
just like the letter "I", and just like the letter "I",
the way to draw them is stored in a font.
Android will try to look up melty face at a normal weight
without italics, but this time, the roboto-regular
isn't the best match.
Instead, it will find NotoColorEmoji.
This is a pre-installed emoji font on AOSP
and it contains a picture for every emoji.
Drawing an emoji from this font on platform
is exactly the same as drawing the letter "I".
Look up the font file and then draw it on screen.
And on Android S and above, you can be sure
that the platform will have emoji
since updatable system fonts will add
modern emoji to the font file.
But before S, we don't have any way to update the font,
and this means Android doesn't have any idea what
font to use for melty face.
Android will instead draw a fallback glyph
called tofu when it doesn't know how to render a character.
And that's where emoji2 comes in.
Before it sends the string to platform,
EmojiCompat.process is called on the string.
This iterates through and finds the emoji
that the platform doesn't know how to draw,
adding an emoji span to each emoji.
This is a replacement span, which
means it will just replace whatever's
in that section of the string.
The first part draws just like normal,
using roboto-regular.ttf, but when Android finds
the EmojiSpan, it'll hand off drawing to the span.
Inside the span, there's two important methods
that Android uses.
First, it gets the size, which tells
Android how much room to reserve for this span in text layout.
And then, when it's time to draw the string,
it'll call draw on the EmojiSpan instead
of trying to draw itself.
Inside the EmojiSpan, it knows where
the compat version of the emoji font is,
and it can draw melty face directly from that.
Going back to rendering, platform
will call EmojiSpan.draw, and the entire region
will be drawn by the EmojiSpan instead of platform.
Now this is a completely different rendering path,
and in effect, from platform's perspective,
EmojiSpan is just drawing a picture
in the middle of the string.
And that's all we have time for today.
Thanks for watching.
I hope you enjoyed the details section,
and remember, please upgrade to AppCompat 1.4
and let the details melt away.
[MUSIC PLAYING]