- We wish that this had come out a little more clearly
for everyone because we feel very positive
and strongly about what we're doing,
and we can see that it's been a widely misunderstood.
- [Joanna] It's not every day an Apple executive
admits to widespread confusion over new policies
related to fighting child pornography.
Yet, here we are.
- I grant you, in hindsight,
introducing these two features at the same time
was a recipe for this kind of confusion.
- [Joanna] Last week, Apple introduced
new child protection features,
one related to spotting and reporting
illegal child sexual abuse material,
AKA CSAM, or child pornography.
And another related to spotting nudes in text messages
sent to and often from children.
- It's really clear a lot of messages
got jumbled pretty badly.
- [Joanna] A lot of people got mad and got mad real fast.
Not because they don't think child pornography is horrible,
but because of the seemingly controversial way
Apple was spotting this content
and a feeling that the company was crossing the line
by scanning the contents of their phones.
- I do believe the soundbite that got out early
was, oh my God, Apple is scanning my phone for images.
This is not what is happening.
- I sat down with Apple software chief,
Craig Federighi, to ask some questions
that have been on my mind about these features.
- Good morning.
- Good to see you.
I broke the interview into two parts,
one, the features to spot CSAM.
Two, the messaging feature.
It's all confusing, but it's important to understand
what's happening and what control you do and don't have.
- In the simplest terms possible,
how is Apple looking for child pornography on iPhones?
- So to be clear, we're not actually looking
for child pornography on iPhones.
That's the first root of the misunderstanding.
What we're doing is we're finding illegal images
of child pornography stored in iCloud.
If you look at any other cloud service,
they currently are scanning photos
by looking at every single photo in the cloud
and analyzing it.
We wanted to be able to spot such photos in the cloud
without looking at people's photos
and came up with an architecture to do this,
and it's very important we felt
before we could offer this kind of capability
to do it in a way much, much more private
than anything that's been done in this area before.
- Okay, so let's break this down a little bit more simply.
So this is my iPhone.
I get this update.
What is happening to my iCloud Photo Library?
- So if you are a customer that is using
iCloud Photo Library, which you don't have to,
but if you're using iCloud Photo Library
to store your photos in the cloud,
then what's happening is a,
a multi-part algorithm
where there's a degree of analysis done on your device
as it uploads a photo to the cloud
so that the cloud then can do
the other half of the algorithm.
And if, and only if you meet a threshold
of something on the order of 30 known
child pornographic images matching,
only then does Apple know anything about your account
and know anything about those images.
And at that point, only knows about those images,
not about any of your other images.
And it should be clear this isn't doing some analysis for,
did you have a picture of your child in the bathtub
or for that matter, did you have a picture
of some pornography of any other sort?
This is literally only matching on the exact fingerprints
of specific known child pornographic images.
- [Joanna] Let's pause and talk about
what's been so controversial here.
The National Center for Missing and Exploited Children,
or NCMEC has a database of known CSAM.
Sorry for all the acronyms.
Other big tech companies, Google, Facebook, Microsoft
scan photos you upload to the cloud
to find images that match any in the repository
of NCMEC photos.
Instead, Apple decided it wanted to do this
on your device for privacy reasons,
and that's what a lot of this controversy is about.
- On your phone, when images are uploaded
to the iCloud Photo Library,
as part of that upload pipeline,
there is a what's called a neural hash
performed on the image,
and that neural hash is then intersected
against a database that's on your device.
- [Joanna] Okay, pause again.
Illegal photos have been converted into cryptographic codes,
or a string of numbers that identify the characteristics
of an image, what they call neural hashes.
When your own photo is about to be uploaded to iCloud,
the software generates a hash of it.
The device cross-references your image hash
to the CSAM hashes.
- And what comes out of that
is basically some gobbledygook that at that point,
your phone doesn't know whether it matched anything
and neither does Apple.
But that thing is wrapped up in what we call safety voucher,
and when the photo is stored, so is the safety voucher.
- [Joanna] Pause again.
The system makes a thing called a safety voucher,
which holds the match result.
That is uploaded with the photo to iCloud.
- Then the second half of the process
occurs in the cloud where these safety vouchers,
some math is done on the collection
of all the safety vouchers
and only if a threshold of matches are achieved
after a second pass of math, does anything get learned.
- [Joanna] One last pause.
In the cloud, if an account collects around 30 vouchers
that correspond to illegal images,
the account will be flagged to Apple's human moderators,
who then review the vouchers
and see if they actually contain illegal images.
If they do, Apple reports the account
to the proper authorities.
There's a sense of when I upload something to the cloud,
I sort of think, okay, I've got to trust this company,
but on my device, I've been really led to believe,
in a large part because of Apple's marketing efforts,
that this is mine.
The info that's on here is mine, it is private.
I think what's happening here is
you're coming onto the device.
You're making me feel a little bit like,
well, Apple can do things on your device.
- I think that's a common
but really profound misunderstanding.
This is only being applied as part of the process
of storing something in the cloud.
This isn't some processing that's running over
the images you store, you know, in your messages
or in Telegram or anything else,
you know, what you're browsing on the web.
And this literally is part of the pipeline
for storing images in iCloud.
- Why did you decide to release this right now?
Was there pressure to do it?
- No.
Really it came down to, we figured it out.
We've wanted to do something,
but we've been unwilling to deploy a solution
that would involve scanning all customer data.
- Tim Cook said earlier this year,
"We've spoken out time and again-"
- For strong encryption without backdoors,
recognizing that security is the foundation of privacy.
- But isn't this, in a way, a backdoor?
- Think in no way is this a backdoor.
I don't understand.
I really don't understand that characterization.
Imagine someone was scanning images in the cloud.
Well, who knows what's being scanned for?
In our case, the database is shipped on device.
People can see, and it's a single image
across all countries.
We ship the same software in China
with the same database as we ship in America,
as we ship in Europe.
If someone were to come to Apple, Apple would say no,
but let's say you aren't confident.
You don't want to just rely on Apple saying no.
You want to be sure that Apple couldn't get away with it
if we said yes.
Well that was the bar we set for ourselves
in releasing this kind of system.
There are multiple levels of auditability,
and so we're making sure that you don't have to trust
any one entity or even any one country,
as far as how these images are,
what images are part of this process.
- I want to just move on to the iMessage feature
which you guys are calling
Communication Safety in Messages.
Simplest way possible, explain it to me,
no acronyms, no jargon.
How does this work?
(laughs)
- I'll do my best.
The capability is you turn this feature on for your child
and your child received an image coming in
that was potentially, you know, pornographic content.
The image would be blurred out.
The child would be told this content may be unsafe.
And so the child can just ignore it.
That detection happened on device.
No one was told, it's just local processing.
If the child then says, oh, I want to look at it,
then it warns the child, it tells them about
that this could be unsafe.
These things can be harmful.
Are you sure you want to do this?
Child can say yes.
Now if the child is 12 or under,
and the parent has turned on this notification capability,
the child will then be told
if you choose to continue and view this,
your parent will be notified that this has happened
because I think a legitimate fear
of some parents with young children is
what's happening that my child is being coached
to keep from me while they're being groomed?
And really we think parents more than anyone
are in the best position to parent.
- Okay, so I just want to clarify.
This technology is different than the technology
you're using with the CSAM?
- That's right, it is 100% different.
- Okay, I lied.
Another pause.
This feature is using on-device machine learning
to look for photos that might contain nudity.
How confident are you in this,
the algorithms that are looking at these photos?
I mean, how confident are you guys gonna to be that
this is a nude image and not a rocket ship?
- Right?
So this is a machine learned classifier.
It's very good.
It's very accurate.
It can make a mistake, right?
It could say a photo that-
It's unlikely to be a rocket ship,
but, you know, we've had a tough time
coming up with images that we can fool it with
to do our testing with,
'cause we wanted to be able to go through flows
in the feature without all looking at nudity,
and it's very hard to fool it,
but it can be fooled.
- Privacy experts spoke out
about both of these announcements this week.
With the CSAM feature,
the Electronic Frontier Foundation said Apple
was overreaching and creating a backdoor
and a technology that could be abused.
Many have signed a letter against these features.
Who owns this phone?
There's this famous Apple ad,
the 1984 ad where Apple is literally bashing Big Brother.
Yet with these updates, there's this thought
that Apple can reach into our pocket
and change our phone in significant ways.
- I think our customers own their phones.
For sure.
I think the feature we're talking about here,
one needs to just continue to be reminded
is about only applying to data customers
choose to store on Apple servers.
I do believe the soundbite that got out early
was, oh my God, apple is scanning my phone for images.
This is not what is happening.
This is about images that are being stored in the cloud
in an architecture for identifying
in the most privacy-protecting way we can imagine
performing that process
and in the most auditable and verifiable way possible.
- [Joanna] When this update arrives,
it could provide powerful tools
for preventing child endangerment,
but it's also a reminder of the control
that Apple and executives like Mr. Federighi
have over the products that we own.