99.999% of the requests coming from the usual AI suspects (Amazon, Alibaba, Microsoft, Google etc.)

Noticed it because disk was getting full from several dozens of GB of webserver logs (for something that gets like 1-2 human visits a day), the motherfuckers don't even throttle their crawlers, they're firing off requests in the dozens per second, every single one of them behaves like a berserk and of course they do not respect any robots.txt at all.

Given how often you saw normal search engines in your logs and how often you now see AI crawlers, I can only guess that it's like a death-race of who downloads the whole internet first to train their shitty eliza pro+ LLMs with.

And since their hunger for new data is so immense, they have no time to be kind to webservers. Fuck webpages anyway, in their vision people will not visit webpages anymore in the future, they will just ask some shitshippiddi for everything.

Needed some time to collect all the evil cloud networks
I traffic-shaped them down to 10kb/s
one request per minute