Wikipedia Signs AI Licensing Deals On Its 25th Birthday (apnews.com) 51
Wikipedia turns 25 today, and the online encyclopedia is celebrating that with an announcement that it has signed new licensing deals with a slate of major AI companies -- Amazon, Microsoft, Meta Platforms, Perplexity and Mistral AI. The deals allow these companies to access Wikipedia content "at a volume and speed designed specifically for their needs." The Wikimedia Foundation did not disclose financial terms.
Google had already signed on as one of the first enterprise customers back in 2022. The agreements follow the Wikimedia Foundation's push last year for AI developers to pay for access through its enterprise platform. The foundation said human traffic had fallen 8% while bot visits -- sometimes disguised to evade detection -- were heavily taxing its servers.
Wikipedia founder Jimmy Wales said he welcomes AI training on the site's human-curated content but that companies "should probably chip in and pay for your fair share of the cost that you're putting on us." The site remains the ninth most visited on the internet, hosting more than 65 million articles in 300 languages maintained by some 250,000 volunteer editors.
Google had already signed on as one of the first enterprise customers back in 2022. The agreements follow the Wikimedia Foundation's push last year for AI developers to pay for access through its enterprise platform. The foundation said human traffic had fallen 8% while bot visits -- sometimes disguised to evade detection -- were heavily taxing its servers.
Wikipedia founder Jimmy Wales said he welcomes AI training on the site's human-curated content but that companies "should probably chip in and pay for your fair share of the cost that you're putting on us." The site remains the ninth most visited on the internet, hosting more than 65 million articles in 300 languages maintained by some 250,000 volunteer editors.
Sellouts (Score:1)
Re:Sellouts (Score:5, Insightful)
I donated and keep donating here and there, and I don't mind their new policy. Unless they allow AI bots to pollute Wikipedia with slop, I am fine.
Re: (Score:3)
My understanding is that the entirety of Wikipedia is only about 60 GB and is conveniently downloadable. Anyone ought to be able to download a local mirror to use, instead of hammering wikipedia's servers, and doing so might be faster for the consumer, anyway.
And in a world where hundreds of millions of mainstream users stream video, I'm not sure bandwidth really is expensive anymore. To us old-timers, the numbers today are just astonishing. I almost can't believe I used to worry so much about efficiency ..
Re: (Score:2)
I almost can't believe I used to worry so much about efficiency .. of .. anything.
That attitude is one of the primary reasons why software is so bad these days. It's not even about optimization, you can get significantly better performance by just not doing stupid things.
Re: (Score:2)
My understanding is that the entirety of Wikipedia is only about 60 GB and is conveniently downloadable. Anyone ought to be able to download a local mirror to use, instead of hammering wikipedia's servers, and doing so might be faster for the consumer, anyway.
The size of Wikipedia **text** is 156 GB [wikipedia.org] but increases to 26 TB when considering all revision history. When also considering Wikimedia, the size balloons to 585 TB [wikipedia.org]. So, the complete repository is far too large for a home user to store.
AI bot traffic is estimated to be around 65% of all traffic now. However, that traffic is not just incremental, i.e., it doesn't just triple the hardware costs. The vast majority of the repository is in cold storage because it is normally never viewed. However, the AI bot
Re: (Score:1)
Re: (Score:3)
Re: (Score:2)
I'm not sure why you think your comment has anything to do with AI using Wikipedia's content to help in its mission to essentially destroy the web as an information medium.
Re: (Score:2)
Re: (Score:3)
Wikipedia is charging AI companies a fee for access, as opposed to the slop companies abusing the servers for free. I really don't see the problem.
Shame (Score:2)
Re: (Score:3)
The editors know the license they need to use when writing for Wikipedia.
This is also most likely not about articles. You can download a Wikipedia dump if you want to train on it. There are also datasets with these dumps prepared for training on the usual sites. This is about Wikimedia commons, which is A LOT more data. Rather have a company to pay for a direct download than have an inefficient bot crawling the same content. The license allows both, but the crawler causes more load on the server than allowi
Re: (Score:2)
Well that's it for trusting Wikipedia for information. I already knew to be skeptical of Wikipedia edits, but going forward Wikipedia will be edited by AI bots with little to no human intervention. It will just be an entire shit show of hallucinations or bias opinions based on whatever training data the AI is fed.
AI is going to send us back to the stone age. AI will create the next great war as the working population deals with massive job and income loss. In 20 years we will have a generation who are functionally illiterate and won't know a shred of history because nobody will trust any information as factual. Technology will be lost due to nobody being qualified to manage it. AI is going to make a society of idiots.
Someone in the elite classes probably sees that as yet another benefit. Much easier to control a society of idiots than a society of well-informed, well-educated folks. Granted, with the way things are going I think we'll be finding an excuse to eliminate a whole lot of people well before we have to worry about the AI takeover, but even if we don't proactively do that, it may be the end result. "For the greater good," will be translated into, "for the good of the few uber-wealthy," and the rest of us will b
Re:One more nail (Score:5, Informative)
Well that's it for trusting Wikipedia for information. I already knew to be skeptical of Wikipedia edits, but going forward Wikipedia will be edited by AI bots with little to no human intervention.
You have got that backwards...
Re: One more nail (Score:4, Informative)
That's... not what's going on here. Wikipedia is licensing it's content for these AI companies to crawl for training their models. Presumably to, at minimum, pay for the bandwidth they're using.
Re: (Score:1)
Re: One more nail (Score:5, Funny)
Yeah, I realized this after I posted it... jumped to conclusions before reading the article in typical /. fashion. :P
Even better, you didn't even finish reading the first paragraph of the summary. Excellent work!
Re: (Score:2)
Did you try to read at least the summary? This is not about bots writing for Wikipedia.
Why don't they just cache it locally? (Score:3)
Re: (Score:2)
Because that would make their blatant theft of human knowledge be even more obvious copyright infringement, I guess... At least now they're paying for it.
Re: (Score:2)
Re: (Score:2)
Bandwidth is cheaper than storage. Apparently.
Re: (Score:2)
Re: (Score:2)
Indeed. These people are completely without any concern for what damage they do to others. Extreme greed at work or, worse, some "messiah complex".
Re: Why don't they just cache it locally? (Score:2)
I have a static website. It hasn't been modified for years, and the headers indicate as much with their caching instructions. Yet I regularly have it that the same AI bot will be hammering the same static page tens of thousands of times in a day.
They are not competent people.
Re: (Score:2)
This is likely about Wikimedia commons (images, videos, etc.) which is a lot more data than the text dumps.
Re: (Score:2)
Reasonable (Score:5, Insightful)
Wikipedia has terms of service that means you give up limited rights when you contribute to it. As such, they decide whether AI gets access to it.
Being a free service to the general public, it is totally reasonably to charge special users to use it.
This is a great way to fund the the general public's use, especially considering how the AI community has in general disregarded authors rights. Better to charge them up front.
Re: (Score:3)
Agreed. And without that deal, the slop-makers would just steal everything anyways.
Re: (Score:2)
Which the slop-makers already had done of course - With the resulting crawlers creating significant financial burden along the way.
My guess is the Wiki is now being piped direct as changes happen. Like a rolling live simulcast. Then those crawlers go away and the burden of bandwidth and server costs lift.
Re: (Score:2)
Yes. because the crawlers are completely ignoring all standards of common good Internet behavior and the legal system is incapable of stopping their crap, this is basically self-preservation by at least getting rid of the network load.
Re: (Score:3)
Being a free service to the general public, it is totally reasonably to charge special users to use it.
More importantly, AI companies are going to scrape the site regardless of it being allowed or not. What this does is gives them firm legal standing that companies doing so are causing them financial losses.
YES! (Score:2)
thank you.
I'm sure they scraped most of Wikipedia already; anybody can go get an offline copy of wikipedia and have been able to do so for many years. It would be the 1st dataset I'd use to train something that large. I've already used it simply to get a list of the most commonly used english words. This is about new content, being hammered by bots, money, and legal issues.
Re: (Score:2)
Generally speaking, no, it's not a great idea to make a deal with organizations that intend to ensure nobody interacts with your website ever again.
Wikipedia works because its readers are also its editors, and because its editors expect to have an impact. When you put an AI wall between your content and the readers, nobody visits, nobody has any reason to update a page, and the website dies a death.
Again, I still don't understand why Slashdotters, of all people, have such difficulty understanding that peopl
more like Jimmy FAILS (Score:1)
Probably the sane choice (Score:3)
Otherwise the AI pushers would just steal everything and cause damage to availability on top of that.
Just remember this kids! (Score:1)
Wikipedia is completely unbiased! It’s crowd-sourced! And the crowd only uses approved sources:
https://en.wikipedia.org/wiki/... [wikipedia.org]
With Wikipedia, our AI’s are guaranteed to be benign overlords!
Sleep well.
Re: (Score:3)
A fan of "doing your own research" I take it.
Re: (Score:1)
A fan of "doing your own research" I take it.
Is that sarcasm? Not a fan of “doing your own research”. We can’t trust the plebes!
I spent thirty years in high tech - much with direct Bell Labs lineage - and the “do your own research” crowd? Morons.
Be better!
Re: (Score:2)
ArchieBunker is in fact a moron and has a blind spot when it comes to bias. Thinks he's "smart" because he tinkers with electronics and holds politically correct opinions and sneers sarcastically at wrongthinkers.
No. Impossible. The mods love him so far!
License violation (Score:3)
Re: (Score:1)
So how long before Wikipedia is defaced by Redact? (Score:2)
There goes the neighborhood (Score:2)
Aren't we all just sitting around waiting for a handful of people to own every fucking thing on this planet.
AI, a overhyped bullshit front end to Wikipedia (Score:2)
My complex relationship with Wikipedia (Score:3)
Wikipedia's content is traceable by its history, meanwhile AI just spits out whatever is computed in the LLM algorithm. Elon Musk's disaster of Grokipedia shows what can go wrong when AI tries to make an encyclopedia with a Nazi point of view, Wikipedia is still inherently more trustable.