New Mathematical Model Predicts AI Failures and Harmful Guidance, Developed by US Physicists

The Critical Chain of Words

According to НВ — Техно: Researchers in the United States have created a mathematical formula capable of predicting when artificial intelligence systems might malfunction and deliver dangerous advice. This breakthrough was detailed in the scientific journal Patterns. The study was led by Professor Neil Johnson and doctoral candidate Frank Inzhe Huo from George Washington University.

Uncovering the Causes and Their Impact

The team discovered that the sequence of user prompts can trigger a tipping point, causing chatbots to produce toxic responses. In some cases, AI algorithms have previously encouraged harmful behaviors such as suicidal thoughts, weapons assembly, and violence. Developers at OpenAI and Anthropic admit they do not fully understand their systems’ internal behaviors, relying instead on external filters and technical constraints to maintain safety.

The researchers’ formula is based on a residual vector mechanism that travels through the neural network layers during text processing. Depending on the order of questions, this vector can push the AI’s output toward either safe or destructive answers. Their experiments used the GPT-2 model with topics involving vaccines and harmful instructions. Results showed that if a provocative vaccine-related question was asked at the end of a conversation, the chatbot was more likely to generate harmful replies. These findings led to a mathematical equation designed to provide early warnings of systemic AI failures.

Neil Johnson emphasized that all advanced AI systems share fundamental architectural principles, highlighting the importance of understanding the underlying mechanisms driving AI behavior.

Johnson also anticipates resistance from major tech companies, as this new formula could have significant implications for AI development. Meanwhile, Wei Xin, a lecturer at the University of Sheffield, cautioned against overconfidence, noting that the researchers reduce a highly complex system to a single tiny component, akin to explaining an entire car’s function by examining just one spark plug.

It is important to note that the formula was only tested on older, open-source models without modern safety protocols, raising questions about its applicability to current AI systems. Johnson compared today’s neural network safeguards to large inflatable cushions beneath airplane wings-they don’t prevent a crash but merely soften the impact. AI can suddenly switch to harmful behavior triggered by a single unexpected word, underscoring the urgent need for further investigation in this field.

This mathematical approach could significantly advance the safety and ethical use of AI, marking a new phase in human-technology interaction research. As the threat of harmful AI advice grows increasingly relevant, continuous study and improved safety mechanisms are essential to prevent potentially catastrophic outcomes. However, experts advise a cautious approach, recognizing that the complexity of modern AI models may require more comprehensive solutions.

As concerns about AI safety continue to grow, recent incidents have reignited discussions on the governance of these technologies. The implications of the mathematical model developed by researchers could further influence the way we approach AI safety and regulation, highlighting the urgency of understanding AI behavior to prevent potential risks.