(source-appleinsider.com_.jpg)
Website owners can easily instruct Apple’s AI not to scrape their sites for training purposes using a simple mechanism, and major platforms like Facebook and The New York Times are reportedly utilizing this feature. Unlike Google, which maintains that data should be freely accessible for training AI models, Apple has offered publishers millions of dollars for permission to scrape their content. To facilitate this, Apple respects a system that allows websites to specify in a file that they do not wish to be scraped.
Publishers Use robots.txt to Restrict Apple’s Scraping Controversy This file is known as robots.txt, a basic text document that many major publishers are using to block Apple’s AI training.
According to Wired, this method is widely adopted to prevent Apple’s Scraping Controversy AI from accessing their sites.
The robots.txt file does not constitute a technical or legal barrier to scraping, and some companies disregard these blocking efforts. Wired reports that recent studies show about 6% to 7% of high-traffic websites are blocking Apple’s search tool, Applebot-Extended. Another study by Ben Welsh found that over 25% of checked sites are blocking it. The variation in these figures is due to differences in the high-traffic sites studied. For comparison, OpenAI’s bot is blocked by 53% of news sites, and Google’s equivalent is blocked by nearly 43%.
Discrepancies in Blocking Rates Highlight Apple’s Scraping Controversy Bot Obscurity and Legal Battles
Wired suggests that the low blocking rates for Apple’s AI might be due to its relative obscurity compared to other bots. However, AppleBot-Extended, a more advanced version of AppleBot, has been active since it was first detected by sites in November 2014 and officially disclosed by Apple in May 2015.
Given that AppleBot has been used for ten years to support Siri and Spotlight searches, it’s unlikely that website owners are unaware of Apple’s AI initiatives. Rather, the issue may be that some sites are actively choosing to block Apple’s AI, particularly as negotiations continue or might begin. This includes The New York Times, which is also pursuing legal action against OpenAI for copyright infringement related to AI Apple’s Scraping Controversy.
“The law and The Times’ own terms of service clearly prohibit scraping or using our content for commercial purposes without prior written permission,” says Charlie Stadtlander of The New York Times. “Importantly, copyright law still applies regardless of whether technical blocking measures are in place.”
Also Read : The Enterprise world
