ayo, 29 days ago Of course, if you make something faster the bots will just be crawling faster as well.
Of course, if you make something faster the bots will just be crawling faster as well.
wolf480pl, 29 days ago @ayo At $dayjob we have an http endpoint that an external company uses, and they scrape it as fast as we respond. When they fo that it hits our limit of conntrack table entries, and we let that happen because it's going to hit some limit anyway
@ayo At $dayjob we have an http endpoint that an external company uses, and they scrape it as fast as we respond. When they fo that it hits our limit of conntrack table entries, and we let that happen because it's going to hit some limit anyway
Add comment