Centralised moderation teams often lack the context to know what they're looking at. Fediverse admins each take care of a small, well-defined bit of overall moderation; the bit that affects accounts on their server. They know what's acceptable in their community.
"Mastodon users probably aren’t aware of CSAM on the platform unless it leaks into their federated timelines. This can happen when a fellow user on their instance follows an account posting CSAM. Ways to handle this problem are few. Though users who follow CSAM-disseminating accounts can be suspended from an instance by administrators, they can easily set up a new account on another..."
“Suffice it to say that everyone in possession of a copy of the LAION-5B images has hundreds if not thousands of instances of CSAM” | …so that’s 0.0001% of the content, then
So David Thiel at Stanford has posted a much-reported paper/story which tells us that the dataset which drives Stable Diffusion and a bunch of other AI systems, has scraped:
hundreds if not thousands of instances of CSAM (and a much larger number of instances of NCII more broadly)
…and it struck me to ask “how many images are there in LAION-5B so we can get a percentage?”
It turns out that the number of images in LAION-5B is five billion – hence the 5B:
LAION-5B was released in early 2022 by a German nonprofit that has received funding from several AI startups. The dataset comprises more than 5 billion images scraped from the web and accompanying captions. It’s an upgraded version of earlier AI training dataset, called LAION-400M, that was published by the same nonprofit a few months earlier and includes about 400 million images.
So if we generously interpret “…if not thousands…” to mean “five thousand” then some simple maths tells us that this is 0.0001% of the content, or literally “one in a million”.
This is the “needle in a haystack” ballpark – again, literally, if a heavyweight darning needle weighs 1 gram, then one million needles would weigh 1000kg, and the largest 4x4x8 haybales max-out at 2000lb / a little over 900kg.
So there can be more than 9x more mouse poop in the flour which makes your bread, than there generously is CSAM in the LAION-5B dataset.
“But this is all guesswork on your part / One image is one too many…”
The numbers are all above. Feel free to nitpick. Pick your own percentages. The FDA acknowledges that that poop in food is unavoidable, and the unstated goal of “Zero CSAM in a scraped dataset” will probably likewise be unavoidable. Thiel himself acknowledges:
While it’s not surprising that a crawl of the public internet will contain some CSAM, there’s no reason to go gather data on that scale without appropriate safeguards. The project that seeded the LAION sets made some efforts to filter content with CLIP, but it didn’t do enough.