phillycodehound, 11 months ago Google wants to change Robots.txt. Oh come on. They just want access to everything to scrape. #GoogleIsEvil
Google wants to change Robots.txt. Oh come on. They just want access to everything to scrape.
#GoogleIsEvil
reiver, 11 months ago @phillycodehound What does Google want to change about robots.txt ?
@phillycodehound
What does Google want to change about robots.txt ?
phillycodehound, 11 months ago @reiver They want to scrap it and come up with something "better" and more specific to todays landscape. I just don't trust Google.
@reiver They want to scrap it and come up with something "better" and more specific to todays landscape. I just don't trust Google.
researchbuzz, 11 months ago deleted_by_author
deleted_by_author
phillycodehound, 11 months ago @researchbuzz Yep
@researchbuzz Yep
phillycodehound, 11 months ago @rustybrick do you think #Google will just ignore Robots.txt? I mean they'd love to be able to train on everything. Though I would love more controls on stopping AI from scraping without blocking my sites from Search #seo #AI #scraping #RobotsTXT
@rustybrick do you think #Google will just ignore Robots.txt? I mean they'd love to be able to train on everything.
Though I would love more controls on stopping AI from scraping without blocking my sites from Search
#seo #AI #scraping #RobotsTXT
Add comment