Breaking: Autonomous Agents are a Shitshow

Remember my essay back in August with Nathan Hamiel LLM + Coding Agents = Security Nightmare? We were not wrong.A new study from researchers from Stanford, MIT CSAIL, Carnegie Mellon, ITU Copenhagen, and NVIDIA and Elloe AI Labs, examining 847 autonomous agent deployments drawn from healthcare, finance, customer service and code-generation, showed that 91% were vulnerable to subtle but dangerous tool-chaining attacks (see diagram below), Sample tool chain attack, from a separate paper from AWS and Berkeley that is mentioned below. The gist is that seemingly innocuous calls can combine to cause serious problems that “reasoning” models miss.Additionally, in the new study 89.4% of agents showed drift relative to their goals after...

Read the full article at the source.