One last thing before I turn away to the afternoon's work: I see a lot of advice about using a robots.txt file to halt "AI"-scraper bots, which would be great if it worked.
Since I put the preferred verbiage in my own site's robots.txt, my firewall has stopped over 1800 "AI"-scraping attempts.
User-agent blocking at the firewall level is better. Sure, belt and suspenders, put the verbiage in your robots.txt, but ALSO keep your user agent blocks up to speed.
This isn't as hard as it sounds! Most firewalls have a way to add user-agent blocking. (Remember those asterisks!) Many CDNs, like Cloudflare, have the same function. Do a couple DuckDuckGo searches and protect yourself.
Also, I'd recommend getting rid of any Google Site Kit plugins etc., and anything Google-related. There's no telling what they'll slip into their code snippets next, my friends.
@jake I actually just went monkeying around on my site and found an easy way to do it with the theme I have installed, so…bonus? (I should be working, but…)
@jake I really, REALLY recommend using user-agent blocking with a firewall as well, since the bots don't seem to obey the robots.txt. (So far, my firewall has blocked over 1800 scraping attempts by bots which should have obeyed the robots.txt.) So…belt and suspenders, I'd use both.
Add comment