Apple and Google's CSAM Detection - with AI

Warning : This Article discusses the detection of Child Sexual Abuse Material( CSAM). Read discretion is advised.

Imagine this :
In some dark alley, a blurry, cropped, low res photo is uploaded to the crowd - no one sees it and no alarm is raised. Yet, within seconds the system takes it down before it reaches you and closes the door on a predator.

In a world where millions of photos are shared every minute, how do tech giants like Apple and Google identify and stop CSAM — one of the darkest and most illegal types of content — with such eerie precision?

How do they know an image is abusive, even if it’s edited or disguised?

The answer lies in a fascinating blend of perceptual hashing, artificial intelligence, and global cooperation.

How Was CSAM Detection Done Earlier?

Before smart algorithms, detection relied on manual reporting:

• Users or moderators flagged inappropriate images.

• Investigators then traced it to a source.

• This was slow, reactive, and heavily reliant on human effort.

Then came an upgrade: Hash Matching — a way to spot CSAM using just its “digital signature”.

What is a Hash?

Think of a hash as a unique code generated from an image. A fingerprint but for an image.

If two images are identical, they generate the same hash — even if the system doesn’t actually “see” the image.

Example:

• You upload a picture → it generates a hash.

• The system compares this hash to a database of known CSAM hashes.

• If there’s a match, it’s flagged without any human seeing it.

This is similar to how you can recognize a song from just the first few notes.

Regular (Cryptographic) Hash – Like a Lock Key

You’ve probably heard of things like SHA256 or MD5 — these are types of regular hashes. Think of a regular hash as a super-strict digital signature of a file. Even if you change one single pixel in an image, the hash will change completely. It’s like changing one letter in a password — the whole key stops working.

These hashes are great for:

• Verifying file integrity (e.g., checking if a file got corrupted during download).

• Making sure exact copies match.

But they’re not useful if someone:

• Crops the image

• Resizes it

• Compresses it

• Adds a filter

Perceptual Hash – Like Facial Recognition

Now imagine a system that doesn’t care about small edits. This is where Perceptual Hashing (like pHash, PhotoDNA) comes in.It’s like how you can still recognize your friend’s face in an old photo, even if the photo is blurry or filtered. Instead of treating the image like a string of code, it looks at the visual content:

• Shapes

• Layout

• Light/dark areas

• Patterns

So even if someone crops the image a bit, resizes it, or adjusts the brightness — the core fingerprint stays the same.

In simple terms, perpetual hashing turns the image into black & white, shrinks it down to 32*32px to remove all unnecessary details and get it into shape. Then is analyzes the image patterns using math - specifically DCT which breaks the image into basic waves/ frequencies. It then creates a fingerprint - a short string of numbers / letters that represents the core structure of the image. This hash is then compared with a huge database of known illegal content. If there's a match, the system acts without ever needing the need of human intervention.

How Global CSAM Reporting Works

Once CSAM is confirmed:

1. Platforms flag and suspend the user

2. A report is sent to NCMEC (US) or local agencies

3. Authorities investigate, and data may be shared with INTERPOL, INHOPE, etc.

Platforms also share hashes of new CSAM with others to improve global detection.

Here's how Apple and Google both handled the problem individually:

Apple’s Approach

Apple took a bold step in 2021 by announcing a system called NeuralHash, which was designed to scan users’ photos on-device before they were uploaded to iCloud. The goal was simple but powerful: catch CSAM using hash-matching technology, all while maintaining user privacy.

• The system compared hashes of local images to a database of known CSAM fingerprints.

• If a certain number of matches were found, it would trigger manual review and possible reporting.

  

However, this approach sparked major backlash. Privacy advocates worried it could be a slippery slope — if Apple could scan for CSAM today, could governments force them to scan for political content tomorrow? Instead, Apple shifted toward a less invasive, more protective feature:

• On-device nudity detection in iMessage for children using Family Sharing.

• If a child receives or attempts to send an explicit image, the system:

  1. Blurs the image

  2. Warns the child

  3. (Optionally) notifies a parent

Importantly, this happens entirely on-device using AI, so the image never leaves the phone unless the user chooses to share it.

Google’s Approach

Google handles this problem with a cloud approach. It scans content stored or shared across several major services including GMail, YouTube, Google and Drive. Here’s how their system works:

• When a user uploads or shares content, it gets checked against a hash database of known CSAM. This is similar to Apple’s original plan, but Google does it after upload, in the cloud.

• On top of that, Google uses advanced AI and machine learning models to detect new or previously unseen CSAM content. These models analyze:

  1. Visual patterns (nudity, context)

  2. Age estimation

  3. File metadata

  4. Risk signals in filenames or chat messages

• When something suspicious is detected:

  1. It’s flagged automatically

  2. Then reviewed by trained human moderators before any report is made

This hybrid approach uses both hashes for known material and AI for unknown threats allowing Google to operate at massive scale while minimizing false alarms.

The Role of AI in Detecting CSAM

Hash matching helps identify flaws but it has one big problem - it is limited to a certain database. But what about newly generated CSAM or cleverly edited versions meant to escape detection? That's where Artificial Intelligence and Machine Learning will step in.

Unlike hashes that just compare fingerprints, AI systems are trained to understand the content of images and videos — a bit like how a human would. Here’s how they do it:

• Nudity Detection

  AI scans for patterns that resemble exposed skin, body shapes, and explicit poses. It looks beyond clothing and tries to detect sexual context.

• Age Estimation

  Using facial features, height, body structure, and sometimes background clues, AI can estimate if a person in the image is a minor.

• Context Recognition

  The model checks the full scene:

  Is an adult present with a child?

  Is there anything inappropriate in the background?

  The focus isn’t just on the people, but the overall situation.

• File & Text Analysis

  AI can also scan the filename, folder names, or messages attached to an image. If someone sends a file called "secretdoll4yo.jpg" — that can trigger red flags.

Apple uses on device-AI for iMessage nudity detection for children. If the child gets an explicit photo, its blurred, a warning is sent to the parents and the child gets a warning. All thanks to the on device TinyML systems.

No AI is perfect. A baby’s bathtub photo or an innocent beach photo might trigger false alarms. That’s why AI doesn’t act alone — human moderators always review flagged content before action is taken. Click Here to read how AI is being abused to create CSAM.

Final Thought

“AI gives platforms eyes without giving them vision.”

The system doesn’t “see” your private content — it detects risk, flags abnormalities, and then lets a real person decide if it’s dangerous or not. As CSAM threats evolve, AI gives tech companies the power to stay one step ahead — but only when it’s used responsibly, with privacy and human rights in mind.

P.S. More on how tools like PhotoDNA, NeuralHash, and Google’s Content Safety API work — coming up in the next blogs.

Stay Tuned!