Hi, Nick -
Thank you for your inquiry. > I have some concerns about the way the current “AI scraper” problem is > being characterised, [...] In particular, there seems to be very > little evidence beyond assertions that the traffic being classified as > AI scraping is actually illegitimate traffic, rather than simply > automated traffic. I too have doubts about the exact nomenclature. I don't have a problem in principle with AI tools or automation accessing our services, in legitimate ways for users/developers. However, the recent access pattern is evidently not that. Receiving a firehose of O(10^6) different one-shot-IP requests per day overwhelms in terms of esp. CPU consumption of servicing bugzilla / git. So we are stuck with throttling accesses by our best guess at reputation heuristics. > [...] If we're blocking large IP ranges and putting JavaScript > challenges in front of services used by developers, CI systems and > other automated tooling [...] We're building out mechanisms for making it easier identification of IP addresses and access patterns as high-reputation. Please offer suggestions here (or better, [email protected]). - FChE
