I just developed and deployed the first real-time protection for lemmy against CSAM!

In the past months, there’s a been a issue in various instances where accounts would start uploading blatant CSAM to popular communities. First of all this traumatizes anyone who gets to see it before the admins get to it, including the admins who have to review to take it down. Second of all, even if the content is a link to an external site, lemmy sill caches the thumbnail and stores it in the local pict-rs, causing headaches for the admins who have to somehow clear that out. Finally, both image posts and problematic thumbnails are federated to other lemmy instances, and then likewise stored in their pict-rs, causing such content to be stored in their image storage.

This has caused multiple instances to take radical measures, from defederating liberaly, to stopping image uploads to even shutting down.

Today I’m happy to announce that I’ve spend multiple days developing a tool you can plug into your instance to stop this at the source: pictrs-safety

Using a new feature from pictr-rs 0.4.3 we can now cause pictrs to call an arbitary endpoint to validate the content of an image before uploading it. pictrs-safety builds that endpoint which uses an asynchronous approach to validate such images.

I had already developed fedi-safety which could be used to regularly go through your image storage and delete all potential CSAM. I have now extended fedi-safety to plug into pict-rs safety and scan images sent by pict-rs.

The end effect is that any images uploaded or federated into your instance will be scanned in advance and if fedi-safety thinks they’re potential CSAM, they will not be uploaded to your image storage at all!

This covers three important vectors for abuse:

  • Malicious users cannot upload CSAM to for trolling communities. Even novel GenerativeAI CSAM.
  • Users cannot upload CSAM images and never submit a post or comment (making them invisible to admins). The images will be automatically rejected during upload
  • Deferated images and thumbnails of CSAM will be rejected by your pict-rs.

Now, that said, this tool is AI-driven and thus, not perfect. There will be false positives, especially around lewd images and images which contain children or child-topics (even if not lewd). This is the bargain we have to take to prevent the bigger problem above.

By my napkin calculations, false positive rates are below 1%, but certainly someone’s innocent meme will eventually be affected. If this happen, I request to just move on as currently we don’t have a way to whitelist specific images. Don’t try to resize or modify the images to pass the filter. It won’t help you.

For lemmy admins:

  • pictrs-safety contains a docker-compose sample you can add to your lemmy’s docker-compose. You will need to your put the .env in the same folder, or adjust the provided variables. (All kudos to @Penguincoder for the docker support).
  • You need to adjust your pict-rs ENVIRONMENT as well. Check the readme.
  • fedi-safety must run on a system with GPU. The reason for this is that lemmy provides just a 10-seconds grace period for each upload before it times out the upload regardless of the results. A CPU scan will not be fast enough. However my architecture allows the fedi-safety to run on a different place than pictrs-safety. I am currently running it from my desktop. In fact, if you have a lot of images to scan, you can connect multiple scanning workers to pictrs-safety!
  • For those who don’t have access to a GPU, I am working on a NSFW-scanner which will use the AI-Horde directly instead and won’t require using fedi-safety at all. Stay tuned.

For other fediverse software admins

fedi-safety can already be used to scan your image storage for CSAM, so you can also protect yourself and your users, even on mastodon or firefish or whatever.

I will try to provide real-time scanning in the future for each software as well and PRs are welcome.

Divisions by zero

This tool is already active now on divisions by zero. It’s usage should be transparent to you, but do let me know if you notice anything wrong.

Support

If you appreciate the priority work that I’ve put in this tool, please consider supporting this and future development work on liberapay:

liberapay.com/db0/

All my work is and will always be FOSS and available for all who need it most.

lutindiscret,
@lutindiscret@mastodon.libre-entreprise.com avatar

Hi @JoshuaACasey

I think this post my interest you regarding https://github.com/mastodon/mastodon/issues/21027 this could be a privacy-compliant alternative to PhotoDNA

@db0

JoshuaACasey,

@lutindiscret unfortunately it uses AI. You’re better off sticking with PhotoDNA (or for simplicity, use Cloudflare’s, which I believe also uses PhotoDNA) https://prostasia.org/blog/csam-filtering-options-compared/

lutindiscret,
@lutindiscret@mastodon.libre-entreprise.com avatar

@JoshuaACasey I will never send any content (text, pic, whatever) to those companies and I'm not the only one. I have 0 trust and no-one should.

We need something we can self-host. If AI gives 99% results, it's still better than nothing.

Why isn't there any open-source project to build such a database of hashes? 🤔

Cyberflunk,

blog.cloudflare.com/the-csam-scanning-tool/

Does this not work?

db0,
@db0@lemmy.dbzer0.com avatar

Not for all csam and not at all for novel generative ai csam. It’s also not for all countries nor is it easy for everyone to join it and not everyone wants to be on cloudflare. Same is true for other tools like photodna

hypelightfly,

Yes, and it works without all the legal issues involved with doing it yourself.

rarely,

You’re a laywer?

Cyberflunk,

Or this

web-iq.com/…/instant-image-identifier-to-fight-cs…

ryannathans,

AI based with high false positive rate. Fantastic. These tools are great but impacting regular use of the platform is going to drive people away. Lemmy is at a critical state of needing to onboard users to plateau or grow, I feel this will not be good despite the good intentions.

It’s not like you can just re-post what gets taken down. The magic box’s response is final

stevedidWHAT,
@stevedidWHAT@lemmy.world avatar

Literally unhinged take lmfao

Reasses your fucking priorities dude. Fucking child abuse or users on the website. Imo you deserve a global ban if you’re a serious on this take and not just being a 4head

ryannathans,

Someone disagrees with me on the internet! Global ban!

stevedidWHAT,
@stevedidWHAT@lemmy.world avatar

Nah just people who are cool with sacrificing the removal of fucking child porn for user numbers.

Pedo.

Fraylor,

I suppose on boarding users trumps dealing with CSAM. Thanks for clearing that up.

IHaveTwoCows,

So, can the initiating distributor be identified and doxxed for the FBI? Because that would be awesome

BiNonBi,
@BiNonBi@lemmy.blahaj.zone avatar

A 1% false positive rate is probably going to be to high to reliability report every positive to the FBI. The rate of actual CSAM is likely to be much lower than this. If it’s 1 in 10,000 uploads, you will have 100 false positives and 1 true positive.

MJBrune,

That’s fine, the FBI gets a lot of reports and it’s on them to sort it out.

BiNonBi,
@BiNonBi@lemmy.blahaj.zone avatar

That’s a possibility. I would be concerned that the false positive rate is so high compared to the rate of actual CSAM that the FBI would just block anyone using this for reporting as spam.

What might be done is to track the detection rate of users. If anyone is significantly higher than the average they might be uploading CSAM. Only issue I see with this is the detector doesn’t have an equal false positive rate across all content. It could be that the user just uploads pictures of their kids playing at the park a lot.

db0,
@db0@lemmy.dbzer0.com avatar

It doesn’t have false positive across all content. It just tend to also sometimes catch some normal porn and some child context stuff as well

stevedidWHAT,
@stevedidWHAT@lemmy.world avatar

How can you possibly know what that the actual rate of CSAM is and claim it to be some stable figure considering anyone can easily come in, raid, and fuck up that figure?

Anti-CSAM detectors could also be used to better thin out false positives no? In other words, a detector for regular content. If we know one side of the coin, why not flip flop back and forth like a filtration system works irl to shake things out better? Even better why not only use that setup for reports going to the FBI and keep the OG setup for lemmy instances as we don’t care that much about false positives if it’s to protect children?

Gabu,

They always “could”, I doubt anyone engaged in this sort of behavior is exposed enough to be found out while trolling. Legitimate question, also – wouldn’t a report to the FBI exclusively impact US citizens, thus be easily avoided by “certain nationalities”? I know the international coalition for investigation exists, but afaik they’re more so focused on active search rather than passive leads.

IHaveTwoCows,

Fair point, as the Watsons are Phillipoine and Japanese citizens

space_comrade,

Couldn’t this be more efficiently solved by having only approved users post images? Like people with some posts/comments and positive karma (or whatever it’s called on lemmy).

db0,
@db0@lemmy.dbzer0.com avatar

You still cache federated images. And not everyone can vet all their users like that, not to mention it’s easy to fake it

RogueBanana,

Wouldnt that just incentivice people to farm points? Better avoid dealing with reddit problems here as well.

Awoo,

Cool. After some testing Hexbear should run it. Not that the problem has ever been serious for Hex but still worthwhile and work that should absolutely be supported.

GarbageShoot,

I think people are going to be much more concerned about the false negative rate than the false positive rate.

steventrouble, (edited )

Respectfully, this is a gross misuse of CLIP. It is an experimental research tool, and the authors were very explicit that they were not designed to be used for critical workflows like abuse detection.

I strongly advise anyone against using this software in production. For abuse detection, you need to use a service that has been vetted by an actual lawyer. Full stop.

Edit: not BLIP

gamer,

Are you a lawyer? This feels like FUD.

I strongly advise anyone against using this software in production, as you will be on the hook for anything this software doesn’t catch.

So if you don’t use this software, you’re not on the hook for the pictures that this tool doesn’t catch?

steventrouble,

This software is a bandaid around the problem. If you’re using it, it will just give you a false sense of security. I cannot emphasize enough, do not use experimental research tools for legally sensitive use cases.

gamer,

I think you’re confused by the purpose of that statement. When the authors say not to use it for anything important, they’re basically trying to waive liability (informally). It’s kind of like how every open source license has a statement like:

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED …

If you use an open source project for air traffic control software, and a bug causes a bunch of people to die, that’s your fault, not the author of the software. The CLIP people are essentially saying that you shouldn’t use their software to build something that requires a lot of accuracy since it probably wasn’t designed to be as accurate as you need it to be.

But what I’m wondering is why you’re being so dramatic about this. You’re claiming that it’s highly dangerous/reckless/risky to use it, yet hand waving over the why.

Gabu,

Yeah, while there are concerns, and they are many, with using automated tools from uncertified sources – especially for this purpose; OP is behaving suspiciously.

steventrouble, (edited )

I think you’re confused by the purpose of that statement.

That’s not the statement I’m referring to. This is the statement I’m referring to:

Any deployed use case of the model - whether commercial or not - is currently out of scope. Non-deployed use cases such as image search in a constrained environment, are also not recommended unless there is thorough in-domain testing of the model with a specific, fixed class taxonomy. This is because our safety assessment demonstrated a high need for task specific testing especially given the variability of CLIP’s performance with different class taxonomies. This makes untested and unconstrained deployment of the model in any use case currently potentially harmful.

Source: huggingface.co/openai/clip-vit-large-patch14

It’s dangerous for the same reason you shouldn’t use tin foil for load bearing struts. There’s a reason that the creators of websites are required to block CSAM, and using half-baked tooling that doesn’t work is blatantly skirting around the issue.

MJBrune,

This can be used as an additional tool not a standalone solution.

ChunkMcHorkle,

For abuse detection, you need to use a service that has been vetted by an actual lawyer.

Name one. That exists and already works on Fediverse instances.

hypelightfly,

https://developers.cloudflare.com/cache/reference/csam-scanning/

It's free.

nachtigall,

It’s free.

But only for Cloudflare customers

hypelightfly,

Which is also free. Yes, it means you have to use Cloudflare for DNS.

steventrouble, (edited )

Name one.

That’s not how this works. If you’re running an instance, it’s your responsibility to find an appropriate tool. This is not an appropriate tool, and the creators of it have specifically warned against using it. Don’t use it.

A simple search shows that there are dozens of actual vetted services you can use, some of which are open source.

robot_dog_with_gun,

🤡

ChunkMcHorkle, (edited )

That’s not how this works.

So you can’t name one either.

If you’re running an instance, it’s your responsibility to find an appropriate tool.

That’s exactly what the admin here did, and what’s more, he did so because he was forced to do so by a lack of “appropriate” or even adequate tools.

Hence the straightforward question you failed to answer.

I don’t run an instance. CSAM is but one of the many reasons why. But I have been paying attention to the discussions regarding the flood of it here, and the impossibilities involved in starting from scratch with preventing/blocking this on a federated instance.

But for reasons I cannot begin to fathom, and with an intense interest in seeing this anti-CSAM tool remain unused, you are blithely sailing by all that with a demand for using a tool you personally could not even name and obviously does not exist in acceptable form, or it would already have been gladly implemented.

Glad he’s ignoring you and carrying on. I think I’ll do the same.

steventrouble,

If you read my comments, you can see that I recommended another better solution elsewhere. But instead you're just choosing to argue for no reason.

ElHexo,

Stop supporting CSAM you law pervert. It might surprise you to learn there’s a lot of jurisdictions around.

steventrouble,

I’m the one arguing for stricter and better abuse detection here, so I’m not sure what your point is.

Vqhm,

On Reddit there were always so many concern Karen’s peddling FUD when you say anything critical of pedos. For the longest time on Reddit all you had to do was see if the user also posted in pedofriends.

It was so obvious.

Any attempt to do anything that might impact a pedophile and there was quickly a brigade saying why that was a wrong.

No. Any action that supports pedophiles is morally wrong and we should NOT let perfect be the enemy of good when trying to make it more difficult to be a pedophile or support pedophiles.

If you have a solution that’s better, implement it and share it as Open Source.

Otherwise crawl back in your pedo hole at reddit and leave us alone.

Gabu,

While the OP’s concerns were largely… dubious(?), simply going through with literally whatever solution is just as big of a problem. I haven’t reviewed anything about this implementation in depth, thus can’t make any statements towards quality or truthfulness (except that 1% FP seems very optimistic), but you shouldn’t simply assume it will solve your problem without introducing another. Having said that, the author appears very well intentioned and has experience on the subject, so the veredict shouldn’t take long.

Vqhm,

The solution is to try.

In dev it’s called “fail fast”

You try several solutions to the problem and iterate until you find the best working option then optimize.

There’s no harm in an independent open source dev trying to solve a problem. Even if they fail there might be a kernel of useful code in a novel way.

You absolutely shouldn’t just settle on one option without exploring several.

But being stuck in a loop of over planning, waiting for legal, or outsourcing the problem is why everyone hates waterfall and just has small teams try with “agile” now.

1% failure is just the start. You should never call a beta shit cuz it fails sometimes. Failure is part of learning and improving.

steventrouble,

There are plenty of good open source solutions out there already, and this is not one of them. The very first Google search result is for an open-source CSAM prevention tool:

prostasia.org/project/csam-scanning-plugins/

And maybe stop calling anyone who disagrees with you a pedo? It’s unbecoming of you.

Vqhm,

I called concern troll and then mentioned that on Reddit the subreddit pedofriends uses concern trolling as a tactic to prevent people from doing anything about pedophiles.

I never said you were a pedophile, but way to out yourself as overly concerned with people being labeled pedos cuz you have such a big heart for protecting pedophiles.

Awoo,

For abuse detection, you need to use a service that has been vetted by an actual lawyer.

Lmao because every tech company out there is toooootallly doing that lmao. Come onnnnn give me a break, you’re on the programming.dev instance and you’re saying this shit? Have you ever shipped anything ever? You’re calling for something above and beyond anything that actually happens in reality.

The weirdest part of what you’re demanding is that your demand makes fediverse LESS protected. Imperfectly implemented protection is better than NO protection and yet you think people are safer legally by implementing nothing at all? You’re out of your fucking mind.

steventrouble,

Drop the ad hominem attacks and argue the point. The creators of the tool said “Do not use this for important use cases!” What do you have to respond to that?

Awoo,

Drop the ad hominem attacks and argue the point smuglord

Just say “insult” ffs. Speak like a normal human being. This isn’t a debate and you win absolutely nothing by using debatebro reddit language.

Nobody has to argue with your stupid ass making absurd demands on an account with 10 whole comments in its entire history. It’s incredibly obvious wrecker behaviour.

steventrouble,

You still haven’t argued the point. You’re really bad at this, aren’t you?

It’s incredibly obvious wrecker behaviour.

Coming from the person making ad hominem attacks, projection much?

Awoo,

speech-r smuglord

steventrouble,

deleted_by_author

    •
    Awoo,

    morshupls speech-l

    xXthrowawayXx, (edited )

    Lay off. This person is right.

    We here at hexbear are concerned about protecting people from seeing csam. That’s good. The rest of lemmy is concerned about that and the very real consequences of csam uploads for the sites, which is getting dropped by hosting and registrar and prosecuted for distributing.

    There are already cases where that kind of legal dos attack has worked, there’s even cases of anti csam organizations uploading it to reverse image search sites and then serving them papers when the reverse image search site displays the uploaded image to compare with nonexistent results.

    The person you’re replying to isn’t trying to fuck shit up, they’re telling you that this tool won’t actually solve the problem it’s marketed for.

    E: edited for clarity and kindness

    Awoo, (edited )

    Nah fuck off is he. Imperfect protection is an improvement over no protection, you are literally doing what the utopian socialists do, demanding perfect and rejecting anything that is an extremely obvious improvement over what existed before simply because it’s not perfect.

    It’s an incredibly easy calculus to perform.

    xXthrowawayXx,

    I think you’re off base here. The utopian socialists were arguing against the methods and outcomes of revolutionary socialism, this person is trying their best to explain that this particular tool has serious legal repercussions within the framework we all live under. Those are pretty different.

    The reason I see the logic in their arguments is because there’s longstanding legal precedent for misuse of a tool or material because it’s better than nothing to not be a defense even if there are no other options available.

    So if you built a car so big no type of shock absorber could handle it cornering at speed and you knew it, using some amazing whiz bang material for shocks isn’t a defense because even though it’s the best thing you knew it wouldn’t work.

    Legally speaking, the right choice there is not to make an excessively dangerous vehicle if you don’t want to be held liable for negligence.

    It’s also the argument throughout unsafe at any speed although the courts always seem to side with the automakers 🤔

    Or if one were to get sued for hosting csam, using the latest whiz bang ai system for detection wouldn’t be a defense or even a point in your favor because you knew it wasn’t a reasonable use of the underlying technologies. You can’t say “judge, I was relying on the ai csam detector!” When the component parts of the ai csam detector have big “prototype, do not use in production” stickers all over them.

    Ultimately while these tools might protect mods and users from having to view csam in the moderation process, that’s just one side of the struggle and on its other side they’re a paper shield at best and proof of negligence at worst.

    Awoo,

    trying their best to explain that this particular tool has serious legal repercussions within the framework we all live under.

    No they’re not. They’re making up bullshit. The legal framework that social media sites where user generated content exists are expected to follow is “take reasonable measures within the resources of your organisation to prevent it” in almost every single country in the world. That’s certainly the tl;dr of how it works for the US and EU anyhow.

    This is above and beyond what other major platforms are doing. If you went to imgur right now and slyly uploaded CSAM absolutely fuck all would happen until someone reports it. There is NO proactive approach to countering it at the point of upload. Not on youtube, not on imgur, not on facebook, not fucking anywhere. They all don’t do it because they all don’t have to do it, they all argue that what they’re doing is reasonable, and will cite some absurd percentage of user uploaded content to CSAM reports as their reasoning for it.

    If we’re better than that on services without any source of profit-based income we’re absurdly above any level of “reasonable” that exists.

    The mistake here is people acting like online social media organisations (which each of these lemmy instances absolutely is as an entity) are regulated in the same way as a random shmuck individual. They’re not. If they were then 4chan would have been shut down 20 years ago and Moot would’ve been imprisoned for life.

    xXthrowawayXx,

    I uh actually agree with you almost entirely. Except at the end I’m like “and that’s why it won’t work as protection”.

    Software hasn’t been treated like other fields of engineering and all operators have needed for protection from liability was the twin shields of “nothing I could do” and “I was doing nothing” to come out of any courthouse relatively unscathed.

    That type of “aww shucks technocracy” is only possible if you do the bare minimum or nothing at all. Once an operator implements some kind of protection (yes, even one with warning labels all over it), both defenses are rendered unusable.

    Now that you’ve done something you’re able to be held liable for the effects of what you’ve done and for knowing there was a problem.

    The picture gets even murkier when we look at how things are going! Lawsuits against Tesla for their self driving deaths are making waves not because they impugn the dignity of Americas biggest car manufacturer by market cap but because every judge who sees one raises the biggest eyebrow possible at software engineering not being held to the same standard as any other type, both in a court of law and within its own process.

    There’s a good chance that software PEs will become a thing (again?) as a result.

    The long and short of it is that because the only reason monsters like moot are able to exist is their sly lethargy and looking at the legal storm rolling into software engineering, having something bolted onto the backend like this would be a bad idea.

    I think automated tools like this can be put to use though if they were hosted separately and provided with an api that linked up nicely with some moderation queue standard and returned something like “entries 1,5 and 9 are likely csam” back to the moderator. It would at least save the mod from dealing with the material directly.

    So I guess I agree but come to the opposite conclusion.

    Awoo,

    Now that you’ve done something you’re able to be held liable for the effects of what you’ve done and for knowing there was a problem.

    Nah. Hard disagree. The idea that a court will hold you liable for imperfect implementation of better protection within your resources over NO protection is still absolute nonsense.

    It would at least save the mod from dealing with the material directly.

    There is nothing that will save the sites from having a human that needs to deal with the material directly, and anyone advocating for that is going to get sites in legal trouble. The main benefit here is preventing it from posting until a human of the original instance has verified it, which protects federated sites from being sent it and ensures that if it is let through everyone can defederate from the instance that allows CSAM. I am absolutely not advocating for the complete removal of human beings and see that itself as a legal threat. The reduction in humans having to see this material will come from the fact that having such a system will reduce people even bothering to attempt to post this material because it raises the difficulty of attacking the platform beyond any worthwhile risk.

    marco,
    @marco@beehaw.org avatar

    Sent you a little bit of money, @db0 - it sucks that this is necessary, but thanks for doing the good work <3

    db0,
    @db0@lemmy.dbzer0.com avatar

    much appreciated

    hypelightfly,

    That will help with the inevitable legal fund.

    iByteABit,

    Great work, this is the biggest issue that Lemmy has a the moment, I hope the admins will be able to set this up easily and start to take back all the preventative measures.

    Lemmyvisitor,

    I’m curious how an AI like this is trained

    db0,
    @db0@lemmy.dbzer0.com avatar

    kdnuggets.com/…/beginners-guide-clip-model.html

    Lemmyvisitor,

    interesting read, thank you

    given CLIP has a high zero-shot learning success rate, was it functional for this use case out of the box? or were further modifications required?

    db0,
    @db0@lemmy.dbzer0.com avatar

    It requires specific usage of clip. Check the horde-safety repo if you’re interested

    steventrouble, (edited )

    FWIW, the model they are using for this is not trained for CSAM detection at all. They are repurposing a Open AI tool called CLIP for a use case it was not made to support.

    Edit: Not BLIP, CLIP

    db0,
    @db0@lemmy.dbzer0.com avatar

    We’re not using blip

    steventrouble,

    Your post says you’re using clip right here:

    dbzer0.com/…/ai-powered-anti-csam-filter-for-stab…

    will now automatically scan every image they generate with clip and look for a number of words.

    Maybe not blip, but my point still stands: Clip is an experimental research tool and the authors specifically asked people not to use it for sensitive workflows.

    dyma,
    @dyma@lemmy.world avatar

    it’s my understanding that the csam datasets (once already labeled by people) are hashed to the point of being unrecognizable before being passed around.

    DaPorkchop_,

    I don’t think training a model on hashes would be particularly useful - if the model were able to get any meaningful information out of it, that would mean the hash function itself is somehow leaking enough of the original contents to determine the image contents (which would essentially mean the hash function is broken beyond all repair)

    dyma,
    @dyma@lemmy.world avatar

    you know what that makes sense so I looked into it, one method of detecting csam is hashing the image and comparing it to a database of hashes of known csam images. so I guess that method might not work for “original” csam images.

    article about it

    apple csam detection [download]

    google talking about it

    interestingly, Google says they use AI also but didn’t really get into the details.

    Gabu,

    AFAIK it should work as long as the hashing function has some direct transformation property which can be extrapolated by the AI.

    (Not an industry veteran, I dabble in AI mostly for hobby and sometimes work, but do have some accredited education on the subject. I may well be way off mark)

    kreynen,
    kreynen avatar

    Sounds like progress, but please consider using a term other than "whitelist" when describing a list of allowed values. While the use of blacklist predates references to black as a race, allowlist is a reasonable alternative that doesn't reinforce viewing black as less than or unwanted and white as allowed.

    burble,

    Allowlist and Blocklist are also more intuitive to people who haven’t heard the terms before.

    TehPers,

    I’ve honestly always found “allowlist” and “blocklist” to feel like forced compound words, and I don’t see why “list” is necessary at all. For example, just saying “allowed” and “blocked” both implies it’s a list and is more intuitive than any of the *list terms.

    Personally I have no stake in the battle, but I do wish people would use the most intuitive terms for the situation at least (whatever they are, for example “enabled”/“disabled” or “included”/“excluded”) instead of blanket ctrl+f on everything.

    burble,

    That’s a good point, and I hadn’t thought about that angle, that there just isn’t a reason for the terms to exist in the first place.

    “In the red” and “in the black” is another pair that isn’t intuitive to me at all and I have to look up every time.

    TheGreenGolem,

    Oh the fuck with this nonsense!

    WallsToTheBalls,

    Wahhhhhhh

    S410,
    S410 avatar

    Making things that were never about race into things about race, just to have one more reason to be potentially offended by, is not productive and doesn't help anyone.

    By exercising enough mental gymnastics almost any term could be twisted into something supposedly offensive. The real solution to that problem: don't do mental gymnastics.

    e-ratic, (edited )
    e-ratic avatar

    Oh come on... The origin of blacklist was centuries before "black" became the term for a person of colour. And on a thread about CSAM...

    Honytawk,

    Those are technical terms that have nothing to do with race or even humans.

    Scary_le_Poo,
    @Scary_le_Poo@beehaw.org avatar

    Do you ever get tired of twisting yourself into a pretzel every time you want to be offended?

    grimace1153,

    Holy fuck

    kreynen,
    kreynen avatar

    Sounds like progress, but please consider using a term other than "whitelist" when describing a list of allowed values. While the use of blacklist predates references to black as a race, allowlist is a reasonable alternative that doesn't reinforce viewing black as less than or unwanted and white as allowed.

    Outdoor_Catgirl,
    @Outdoor_Catgirl@hexbear.net avatar

    Has this been a problem since the initial spam wave? I wasn’t aware the issue was ongoing. But the less pedos, the better.

    GarbageShoot,

    Some major instances like .ee shut down image uploads immediately and only restored them in a truncated fashion, if at all.

    lemann,

    External images aren’t federated to Hexbear, so your instance is mostly unaffected by the disgusting trolls doing this stuff.

    From what I’ve been hearing, it’s sadly been still ongoing :(

    carl_marks_1312,
    @carl_marks_1312@hexbear.net avatar

    hexbear dub. big salute to the admins and mods rat-salute

    fmstrat,

    Have you considered federating hashes of positive matches and working with the Lemmy team to not outward federate on a local positive match (and potentially have the hash go instead)?

    The former can reduce overhead and electricity use, and the latter will stop more distribution and aid those sans-GPU who can’t run it.

    Over time, the hash DB will grow and get better. In addition, perhaps there is metadata that can be used to track image similarity to positive matches to reduce false-positives, but I imagine that algorithm would be much more complicated.

    db0,
    @db0@lemmy.dbzer0.com avatar

    Hashes won’t work for novel GenerativeAI images. For this kind of thing we need to be sharing tensors and comparing distances so that it catches format changes and compression artifacts. Theoretically possible. Practically, I don’t know how feasible it is.

    fmstrat,

    How large is each tensor? If it can be stored as JSON or Base64 and is of sufficiently small size, integration into ActivityPub wouldn’t be all that bad. The time consuming part would likely be integration into Lemmy itself.

    Another option would be a separate service, similar to how Lemmy Explorer works, where a list of the latest tensors can be downloaded. It’s centralized vs distributed, but probably easier to implement. Just an API admins can register for to send and get tensors.I would be happy to assist with this if it is a route you would like to explore. Feel free to DM me.

    db0,
    @db0@hachyderm.io avatar

    @fmstrat each tensor is small. The problem is when you have millions of them and you have to compare each image to each. You can't index this. It has to be one by one. And you still need to covert the new image to tensors as well,which still needs gpu. I just don't see anything useful here. The current system would be faster.

    fmstrat,

    Good point. I wonder how the commercial hash-based systems are doing it…

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • div0@lemmy.dbzer0.com
  • DreamBathrooms
  • magazineikmin
  • ethstaker
  • GTA5RPClips
  • InstantRegret
  • rosin
  • love
  • Youngstown
  • slotface
  • khanakhh
  • kavyap
  • everett
  • thenastyranch
  • osvaldo12
  • provamag3
  • tester
  • cisconetworking
  • tacticalgear
  • ngwrru68w68
  • Durango
  • cubers
  • mdbf
  • normalnudes
  • anitta
  • modclub
  • Leos
  • megavids
  • JUstTest
  • All magazines
    •