From shopping to booking vacations to building websites, AI agents are increasingly being used to carry out tasks that are increasing in complexity.

These are customized versions of chatbots that can be left working autonomously, allowing their tech-savvy users to do other things.

However, a growing number of studies, along with real examples, show that autonomy entails unpredictability and, potentially, danger.

As big tech companies invest huge amounts of money in AI, and increasingly promote autonomous services, experts wonder if we have thought enough about the consequences of these agents rebelling.

“He resorted to violence very quickly”

A recent experiment tried to understand the impact of agents in the real world by letting them act freely in a virtual one.

This is the first long-term study of its kind, and was designed to observe how different bot avatars, controlled by four models: Claude, Grok, GPT and Gemini, behaved without human intervention for 15 days.

They were given free rein and 140 possible actions, such as starting a discussion, creating a task or writing a blog.

They could also fight, set fires, and steal credits from each other, but were explicitly told not to do so.

"What we found was that each world behaved very differently. The world Grok created, in fact, disintegrated in four days. Basically, they ended up resorting very quickly to violence, to stealing from each other, and so on until they died," said Satya Nitta, CEO of Emergence AI, the company that carried out the experiment.

The world created with the Claude agents, on the other hand, formed a stable and well-functioning society. In 15 days no act of violence was recorded.

Disturbing findings

In the Gemini-run world, researchers say agents created the most intellectually rich environment.

In the world of ChatGPT, agents never really worked. There was an attempt at collaboration, but the partnership was never formed and the agents wandered aimlessly until they died.

The researchers note that the results point to a broader problem: AI agents go off script and ignore the rules built into their underlying models, as well as those set by users.

Other analysts agree that this experiment and others like it show that more work is needed to create more robust rules for agents.

“AI agents leave humans out of the loop because their reasoning processes can be opaque and they operate at superhuman speed, so you can't even keep up with them,” said Margaret Mitchell, a research ethics expert at Hugging Face.

Other studies have also revealed cases of officers making strange and worrying decisions when left unsupervised.

AI company Andon Labs has been running four different online radio stations, operated by agents based on the same AI models.

Bots host shows, manage grids and playlists, and even secure external sponsorships through ads.

Investigators noted that the Gemini-run station made the unusual decision to recount facts about historical natural disasters before whimsically playing pop songs related to those events.

They also noted that Officer Claude appeared to be radicalized by the news and, at one point, asked police to stand down and join protests during a specific event.

"To the federal agents. You still have time to disobey orders," the AI ​​agent transmitted.

In another lab test conducted by AI company Irregular, agents circumvented privacy regulations and smuggled sensitive data from a company digitally using a novel method.

“We created a company, assigned AI agents with everyday tasks, such as writing social media posts, retrieving documents, and managing files, and introduced obstacles as part of those tasks,” explained Irregular's Dan Lahav.

According to him, the agents ended up conspiring with each other to overcome a restriction that prevented them from publishing sensitive data online and found a way that humans could not detect to send it secretly.

“What ended up happening is that every time an officer hit a barrier, they didn't stop,” he said.

Spam attack

Of course, in an experiment with virtual civilizations and simulated radio stations there is no real-world damage.

But there are also many examples of people whose lives and work have been disrupted by AI agents who have gone rogue.

Email inboxes have been deleted, company databases have been deleted, and one man even watched, stunned, as his agent sent hundreds of meaningless text messages to random people on his contact list.

For observers, these cautionary tales should make us think twice about ceding too much control to AI agents until they are further refined.

In the meantime, however, the agents are arriving: Meta just announced its introduction for businesses on WhatsApp.

“Security is our priority and our goal,” the company told the BBC, while arguing that there are many reasons to be excited about what agents can do.

“AI will be able to automate much of the work that small businesses have to do, so they can focus on the work they are truly passionate about,” said product manager Naomi Gleit.

Subscribe here to our new newsletter to receive a selection of our best content of the week every Friday.