On access of https://wiki.archlinux.org today, a dialog flashed (…)
Hello,
Many sites are currently falling victim to crawlers from LLM
companies. The bots mercilessly hammer the servers, causing enormous
load. This leads to service disruption to normal users. Since many of
them own hundreds of IP addresses, it’s not possible to throttle them
based on requests alone.
Multiple targets, Arch Wiki included, decided to deploy a client-side
proof-of-work mechanism: Anubis from Techaro.⁽¹⁾⁽²⁾ This runs a small
program, that does “expensive” operation in the browser. It’s nothing
for normal users, even if we count all users together. But it’s a
notable cost to companies doing billions requests.
Cheers
____
⁽¹⁾ https://techaro.lol/
⁽²⁾ Screenshot of what you didn’t se: https://0x0.st/8VRo.png
