Reddit CEO Steve Huffman announced that it will block Microsoft from using data from its site. Huffman insisted that Reddit, which is a social media platform, will continue to block AI companies, including Microsoft, from scraping data on its site until it is paid and has an explanation on how the content is used.
As reported by The Verge, Steve Huffman further added that Reddit will not compromise on the unlicensed use of its data to train AI models.
Reddit blocks Microsoft
According to sources Reddit has made changes to its policy in an attempt to prevent AI developers from scrapping its user data, posts, and communities without consent or payment. Huffman said that Microsoft has been using Reddit’s data to train its AI and summarizing its content in Bing results “without telling us”. He also added that Reddit’s data has also been sold through the Bing API to other search engines.
However, in response to the allegations made by Reddit, a Microsoft spokesperson said that the company “respects” the robot.txt protocol and had stopped crawling Reddit on July 1, 2024.
A few days earlier, Reddit had blocked several other AI websites from using its data for free. According to Reuters, the social media platform Reddit (RDDT.N) said on Tuesday that it will update a web standard to block automated data scraping from its website. The company also said it will maintain rate-limiting, a technique used to control the number of requests from one particular entity. In addition to this it will also block unknown bots and crawlers from data scraping, collecting and saving raw information on its website.
Other updates
During a recent interview with the Verge, Huffman said that “We’ve had Microsoft, Anthropic, and Perplexity act as though all of the content on the internet is free for them to use. That’s their real position.”. He further said that other AI companies have refused to negotiate the payment for using data from Reddit. Earlier the company had concluded a deal worth $60 million with Google, allowing the tech giant to use its content. Reddit had also made a similar agreement with ChatGPT-maker OpenAI in May.
Furthermore, Reddit will be using Robots Exclusion Protocol or robots.txt, a tool used by websites to identify web crawlers extracting data from websites without their knowledge. Such data is usually used for illegal uses. The move comes at a time when artificial intelligence firms have been accused of plagiarizing content from publishers to create AI-generated summaries without giving credit or asking for permission.
Follow FE Tech Bytes on Twitter, Instagram, LinkedIn, Facebook
