Where Large Language Models Stand Today
Appearing on a broadcast with Yuriy Romanenko, Anar Lavrenov—head of the AI department at Sponge—stated that today’s large language models (LLMs) have reached a technological ceiling. According to him, leading developers are now shifting their focus toward building agent-based systems, which makes it possible to use LLMs more efficiently across various fields. Lavrenov noted that the transformer architecture era is approaching its peak, and the world is waiting for new architectures that will surpass current technologies.
Lavrenov emphasized that the arrival of GPT-3.5 marked a major leap forward, driven by the transition from recurrent models to transformers. In this context, he also mentioned Anthropic's rollout of the MCP (Model Context Protocol), which turns LLMs into full-fledged assistants. Anthropic, an extremely secretive company, recognizes that LLMs serve as the foundation for building more complex systems capable of handling diverse tasks—from writing code to creating design layouts.
Key Stages in AI Training
The AI training process consists of three critical stages:
- The first stage involves the model trying to predict the next word using a massive volume of text. During this stage, according to Lavrenov, Claude employs a technique called restructured pretraining, which helps the model learn in a question-and-answer format.
- The second stage is supervised learning, where the model is shown questions along with their expected answers.
- The third stage—known as alignment or reinforcement learning—is the most important when it comes to shaping the model's behavior.
Lavrenov also pointed out that all models share about 90% of the same data, and the main differences between them lie in their training techniques. Narrow domains may show variations of only 5–10%. Beyond architecture, data delivery methods and training speed also play crucial roles. The Anthropic team introduced the DPO optimization method, which further contributes to improving models.
Anar Lavrenov: 'The LLM has reached its ceiling, and the next stage of development is improving agent-based systems that integrate LLMs into our interaction environment.'
These statements highlight the importance of adapting to new AI technologies, as companies seek fresh approaches to using LLMs. The shift toward agent-based systems may signal that the future of artificial intelligence lies in deeper integration and user interaction—a change that could transform how tasks are performed across areas such as business, education, and science.