rrwo, 2 months ago

A website at work added the AI search engine YouBot to robots.txt and blocks all other requests from that user agent.

So now we get requests from the same IP address (in Amazon's netspace) claiming to be Googlebot.

#infosec #bot #ai