@NatureMC@beach@FediTips@feditips
Why shouldn't there be scraping bots in the Fediverse? It has an open structure that is especially well suited for scraping. And the robots.txt file is not mandatory, it's just that most search engines abide by it, but they can simply ignore it of they want.