Cloudflare launches anti-AI bot tool to combat website scraping

A new free tool has been released by exchange-listed cloud services company Cloudflare with the aim of preventing bots from collecting information from websites hosted on its network to create artificial intelligence (AI) models.

In a blog post announcing the update, the Cloudflare team also shared some data on how its clients are coping with the rise in bots using content scraping to build generative AI models.

The report says all users of the cloud service provider, including those on free plans, have access to the new tool.

By modifying their site’s robots.txt file, which tells bots which pages on a website they are allowed to access, website owners can prevent bots from being used for data scraping and model training, according to several AI providers including Google, OpenAI, and Apple.

“Customers don’t want AI bots visiting their websites, especially those that do so dishonestly,” the company informed on its official blog.

READ ALSO: Microsoft pays more than $14 million to settle discrimination claims against workers

“We are concerned that some AI companies intent on circumventing the rules to access content will continue to adapt to avoid bot detection.”

The model takes into account, for example, the possibility that AI bots try to hide from detection by mimicking the appearance and behavior of web browser users.

“When bad actors try to crawl websites at scale, they typically use tools and frameworks that we can fingerprint,” Cloudflare wrote.

“Based on these signals, our models [are] able to accurately label traffic from AI bots that are difficult to understand as bots.”

Cloudflare launches anti-AI bot tool to combat website scraping first appeared on Latest Nigeria News | Top Stories from Ripples Nigeria.