Wikipedia is suspect now too. They’ve sold the site to AI search. Hard to say that they had much of a choice, but the sure did it swiftly and gleefully.
Its all public anyways. Its more like they sold infrastructure access afaik. Which is a smart move. Money aside, unauthorized scraping is a much bigger PITA and can drain resources quickly, including financial resources. Selling access means you can regulate it and also make it go thru appropriate infra; plus it becomes an asset rather than a liability. Not a fan of the wikimedia org lately, but I don’t fault them for that one.
Exactly this. They are guaranteed to get scraped regardless. Ask any question of an AI model and the first ‘source’ it cites is almost always Wikipedia. (assuming it’s not like a super technical in-depth question)
Might as well save on server overhead costs and get paid while doing it instead of trying (and failing) to block every possible scraper.
Wikipedia is suspect now too. They’ve sold the site to AI search. Hard to say that they had much of a choice, but the sure did it swiftly and gleefully.
Its all public anyways. Its more like they sold infrastructure access afaik. Which is a smart move. Money aside, unauthorized scraping is a much bigger PITA and can drain resources quickly, including financial resources. Selling access means you can regulate it and also make it go thru appropriate infra; plus it becomes an asset rather than a liability. Not a fan of the wikimedia org lately, but I don’t fault them for that one.
Exactly this. They are guaranteed to get scraped regardless. Ask any question of an AI model and the first ‘source’ it cites is almost always Wikipedia. (assuming it’s not like a super technical in-depth question)
Might as well save on server overhead costs and get paid while doing it instead of trying (and failing) to block every possible scraper.