Reddit has long been a hot spot for conversation on the internet. About 57 million people visit the site every day to chat about topics as varied as makeup, video games and pointers for power washing driveways.
In recent years, Reddit’s array of chats also have been a free teaching aid for companies like Google, OpenAI and Microsoft. Those companies are using Reddit’s conversations in the development of giant artificial intelligence systems that many in Silicon Valley think are on their way to becoming the tech industry’s next big thing.
Now Reddit wants to be paid for it. The company said on Tuesday that it planned to begin charging companies for access to its application programming interface, or A.P.I., the method through which outside entities can download and process the social network’s vast selection of person-to-person conversations.
“The Reddit corpus of data is really valuable,” Steve Huffman, founder and chief executive of Reddit, said in an interview. “But we don’t need to give all of that value to some of the largest companies in the world for free.”
The move marks one of the first significant examples of a social network’s charging for access to the conversations it hosts for the purpose of developing A.I. systems like ChatGPT, OpenAI’s popular program. Those new A.I. systems could one day lead to big businesses, but they aren’t likely to help companies like Reddit very much. In fact, they could be used to create competitors — automated duplicates to Reddit’s conversations.
A New Generation of Chatbots
A brave new world. A new crop of chatbots powered by artificial intelligence has ignited a scramble to determine whether the technology could upend the economics of the internet, turning today’s powerhouses into has-beens and creating the industry’s next giants. Here are the bots to know:
Reddit’s move also comes as it prepares for a possible initial public offering on Wall Street later this year. The company, which was founded in 2005, makes most of its money through advertising and e-commerce transactions on its platform. Reddit said it was still ironing out the details of what it will charge for A.P.I. access and will announce prices in the coming weeks.
Reddit’s conversations — or subreddits, as the company calls them — have become valuable commodities as large language models, or L.L.M.s, have become an essential part of creating new A.I. technology.
L.L.M.s are essentially sophisticated algorithms developed by companies like Google and OpenAI, which is a close partner of Microsoft. To the algorithms, the Reddit conversations are data, and they are among the vast pool of material being fed into the L.L.M.s. to develop them.
The underlying algorithm that helped to build Bard, Google’s conversational A.I. service, is partially trained on Reddit data. OpenAI’s Chat GPT cites Reddit data as one of the sources of information it has been trained on.
Other companies are also beginning to see value in the conversations and images they host. Shutterstock, the image hosting service, also sold image data to OpenAI to help create DALL-E, the generative A.I. program that creates new, vivid graphical imagery with only a text-based prompt required.
Last month, Elon Musk, the owner of Twitter, said he was cracking down on use of Twitter’s A.P.I., which is used by thousands of outside companies and independent developers to track the millions of conversations that occur across the network. Though he did not cite L.L.M.s as a reason for making the change, the new fees could go well into the tens or even hundreds of thousands of dollars.
To keep improving their models, artificial intelligence makers need two significant things: An enormous amount of computing power and an enormous amount of data. Some of the biggest A.I. developers have plenty of computing power, but still look outside their own networks for the data needed to improve their algorithms. That has included sources like Wikipedia, millions of digitized books, academic articles and Reddit.
Reddit has long had a symbiotic relationship with the search engines of companies like Google and Microsoft. The search engines “crawl” Reddit’s web pages in order to index information and make it available for search results. That crawling, or “scraping,” isn’t always welcome by every site on the internet. But Reddit has benefited by appearing higher in search results.
The dynamic is different with L.L.M.s — they gobble as much data as they can to create new A.I. systems like the chatbots.
Reddit believes its data is particularly valuable because it is continuously updated. That newness and relevancy, Mr. Huffman said, is what large language modeling algorithms need to produce the best results.
“More than any other place on the internet, Reddit is a home for authentic conversation,” Mr. Huffman said. “There’s a lot of stuff on the site that you’d only ever say in therapy, or A.A., or never at all.”
Mr. Huffman said Reddit’s A.P.I. will still be free to developers who want to build applications that help people use Reddit. They could use the tools to build a bot that automatically tracks whether users’ comments adhere to the rules of a subreddit, for instance. Researchers who want to study Reddit data for academic or noncommercial purposes will continue being allowed free access to it.
Reddit also hopes to incorporate more so-called machine learning into how the site itself operates. It could be used, for instance, to identify the use of A.I.-generated text on Reddit, and add a label that notifies users that the comment came from a bot.
The company also promised to improve software tools that can be used by moderators — the users who volunteer their time to keep the site’s forums operating smoothly and improve conversations between users. And third-party bots that help moderators monitor the forums will continue to be supported.
But for the A.I. makers, it’s time to pay up.
“Crawling Reddit, generating value and not returning any of that value to our users is something we have a problem with,” Mr. Huffman said. “It’s a good time for us to tighten things up.”
“We think that’s fair,” he added.
