[cctalk] Re: Large language model (LLM) Web Scrapers

[Paul Koning via cctalk]

> On Sep 16, 2025, at 7:53 PM, Bill Degnan via cctalk <cctalk@classiccmp.org> 
> wrote:
> 
> For those of you who run vintage computing-related info sites, have you
> noticed all of the LLM scraper activity?    AI services are using the LLM
> scrapers to populate their knowledge bases.
> 
> At any given moment 5-10 of them are active on vintagecomputer.net.  

If they are honest they will obey robots.txt and you can use that to stop them. 
 If it doesn't you can report them to the authorities... :-)

        paul