[MUSIC PLAYING]
FRANCESCO ROMANO: Hey, everyone Oh hey, Sarah.
SARAH SHAMSIE: Hey, Francesco.
FRANCESCO ROMANO: Did you have the chance
to see my latest camera app?
SARAH SHAMSIE: No, I haven't seen it yet,
but I hope you tested it on a tablet.
FRANCESCO ROMANO: Of course, it's the form factor data.
Look.
Look how gorgeous it looks on a tablet.
SARAH SHAMSIE: Oh, wow, it looks great.
Wait, but what if I want to rotate the tablet?
FRANCESCO ROMANO: She wants to rotate the tablet.
Why?
SARAH SHAMSIE: Well, for many reasons.
Maybe for a different angle, for better ergonomics,
things like that.
FRANCESCO ROMANO: I mean, I guess we can give it a try.
Yeah, that was kind of unexpected, I would say.
You know, I lock in the orientation for the activity
in the manifest, so I was expecting the preview will
look, like, OK-ish.
SARAH SHAMSIE: Yeah, well, that might be OK
if you're still on Android 11.
But don't you know now that the OEM can override the chance
to lock the orientation?
FRANCESCO ROMANO: Yeah, I heard about it.
But I thought it would look OK in any case.
SARAH SHAMSIE: Yeah, let me quickly
explain what's happening here.
You might think it's as simple as just rotating the image,
but there's actually three types of orientation
you have to think about now.
FRANCESCO ROMANO: Three types of orientation?
OK, go ahead.
I'm listening.
SARAH SHAMSIE: Let's start with the basics.
Do you know what natural orientation is?
FRANCESCO ROMANO: Yeah, I heard.
Please, go on.
SARAH SHAMSIE: So, natural orientation
is the orientation that a user will naturally use a device in.
So you can imagine it's something like landscape
for a laptop, or portrait for a phone.
For a tablet, it could be either of these two.
So, starting from this, we can then
define two other major concepts.
So, next, camera orientation is the angle between the camera
sensor and the natural orientation of the device.
This is likely dependent on how the camera is physically
mounted on the device.
And it is always supposed to be aligned with the long side
of the screen.
FRANCESCO ROMANO: OK.
It is always supposed to be aligned with the longest
side of the screen.
I have a question.
What's the longest side of the screen
for a foldable device, as it can physically
transform its geometry?
SARAH SHAMSIE: That is a great question.
Let's check out the next example.
So, as you noted, a foldable can have a portrait-shaped window
if it's folded in a landscape-shaped window
if it's unfolded.
For this very reason, starting an API 32 and beyond,
this field is no longer static, and instead you
should retrieve it dynamically through the camera
characteristics object.
FRANCESCO ROMANO: OK, if it's not static.
But that means that the same physical camera can
have two different orientations depending on the screen
you're using it?
SARAH SHAMSIE: Yes, exactly.
So, if you look at the picture here,
you can see what can happen if you
assume that the camera orientation always
stays the same.
It can cause mismatch between the camera orientation
and the device's natural orientation,
which can lead to distortion.
FRANCESCO ROMANO: OK, I guess that means I should never
cache this value.
SARAH SHAMSIE: Yep.
And not just for new form factors.
In general, you should not store it, and instead
retrieve it dynamically any time you need it
by calling this code.
FRANCESCO ROMANO: OK, noted.
Now what?
SARAH SHAMSIE: So, next, we need to take into account the device
rotation.
This measures how much the device
is physically rotated from its natural orientation.
FRANCESCO ROMANO: How much the device
is physically rotated from its natural orientation.
Is that degrees, or what?
SARAH SHAMSIE: So, basically, you'll
get an integer between 0 and 4 by calling Display.getRotation,
and then you can convert it to degrees by multiplying
the result by 90.
And then the result would be consistent with the sensor
orientation that we just discussed.
FRANCESCO ROMANO: OK, well, I guess I can deal with that.
But just for the preview, we have
been looking at the device geometry, the camera
characteristics, and data from the display stack.
Anything else I need to know?
SARAH SHAMSIE: Nope, this is pretty much everything.
The actual math is pretty simple.
Let's take a look at the code.
So here we can see a code snippet
that computes the rotation required in degrees
to transform the sensor output orientation for the device's
current orientation.
The rotation needs to be either clockwise or counterclockwise
depending on if the camera is facing the front
or back of the screen.
And then we basically just calculate
the difference between the two concepts
we just discussed, with a little bit of math
to make sure that the value is always
between 0 and 360 degrees.
FRANCESCO ROMANO: OK, now I see why my app was sideways.
You know, having a lock on the orientation, the portrait,
I thought--
I was assuming I could avoid dealing with all of that.
But you know, let's get back to my app
and try to apply this algorithm.
So, this tablet has a network orientation portrait.
That means the sensor orientation is 90 degrees.
But I just rotated it, and now the display rotation
is 90 degrees, too.
If I'm still good at math, it's just 90 minus 90,
so the right rotation to be applied is 0.
SARAH SHAMSIE: Yep, that's correct.
But then, why are you applying 90?
FRANCESCO ROMANO: Well, I haven't applied any rotation.
I just used the default values.
SARAH SHAMSIE: That explains the issue.
By default, texture view will take into account
the sensor orientation, but not the display rotation.
FRANCESCO ROMANO: OK, now I see why for the portrait mode
I didn't need to do that much.
OK, let me fix it.
Run it again.
Pretending it's compiling.
There we go.
Thank you.
[APPLAUSE]
SARAH SHAMSIE: Great job, wow.
Can I ask you another favor, though?
FRANCESCO ROMANO: Another favor she wants to ask.
OK, go on.
SARAH SHAMSIE: So the tablet has a lot of screen space.
Can you quickly throw up Maps in split screen?
It shouldn't be an issue, right?
FRANCESCO ROMANO: It shouldn't.
OK, let's try.
So we're at the taskbar.
We grab Maps.
And-- what happened?
Huh.
OK, I see why you asked this favor.
I guess there's a lesson to learn.
So please, go on.
SARAH SHAMSIE: Yep.
So, handling the orientation alone
does not guarantee a correct camera preview.
For example, in multi window mode
here, you might need to display a landscape-shaped camera
output into a portrait-shaped window.
And as we see here, without the right transformation,
you might end up with a display that's distorted or shrunken.
So you'll either need to crop the image, like this,
or scale it to match the aspect ratio depending on your use
case, like this.
FRANCESCO ROMANO: OK.
That totally makes sense, but how do I do that?
SARAH SHAMSIE: So we have a manual solution for Camera2.
It might take a little bit too long to explain here,
but I will leave the link here for your reference.
I really suggest, though, that unless you
need to use the surface directly,
you take advantage of the libraries
that we've built for you to help in this task.
FRANCESCO ROMANO: Sure, libraries like CameraX?
SARAH SHAMSIE: Exactly.
Jetpack CameraX can be the right solution
because it automatically transforms
the preview for every screen size
and handles the camera lifecycle through different configuration
changes or through multi resource access.
So CameraX releases are continuous,
and it's continuously updated and tested
on different devices, so you'll ensure the best compatibility.
FRANCESCO ROMANO: OK, that sounds amazing.
The only thing is, I'm already using all those Camera2 APIs,
and I haven't migrated yet to CameraX.
Is there a solution for me?
SARAH SHAMSIE: Yep, no worries.
We do have something for Camera2.
So, we just released a new artifact that
does not depend on CamerX-core.
It's called Camera-viewfinder, and you can basically
substitute your raw surface directly with it,
and let the library handle all the nuances for you.
FRANCESCO ROMANO: That sounds perfect.
But how come I didn't know about it?
SARAH SHAMSIE: Because we just released it today.
Yay.
[APPLAUSE]
You can check out how to use it at the link here.
FRANCESCO ROMANO: OK, wonderful.
OK, let me just look at the code right now.
OK, so I see at the beginning we choose the resolution
we want for the preview, as usual.
And then we build this request.
And then we get this callback.
And when the surface is available,
If everything went well, we can just
use that surface with a standard Camera2 API, as always,
instead of getting the surface from the texture
view, the surface view, applying the rotation.
That's it?
SARAH SHAMSIE: Yep, that's it.
Camera-viewfinder will take care of all the orientation
and aspect ratio considerations for you.
FRANCESCO ROMANO: Well, I guess I don't have excuses anymore.
I need to go update my app.
SARAH SHAMSIE: Good luck.
Hopefully the resources we talked about today
will be helpful for you.
And I hope all of you are also encouraged
to test your camera apps on different devices and form
factors.
FRANCESCO ROMANO: Thanks for your time today,
and have fun with your camera apps.
[APPLAUSE]
[MUSIC PLAYING]