It now seems as if we have reached a plateau in AI development. Improvements in LLM are slowly coming to a halt, and the difference between GPT 4.5 and GPT 5 appears to be minimal. I don’t believe that AGI can be achieved by only training with more and more data. I don’t even believe that the LLM era will last¹. I have always believed that this is not the way to create true (artificial) intelligence. The recent article “Sakana AI’s CTO says he’s ‘absolutely sick’ of transformers, the tech that powers every major AI model” gives me a similar feeling. Especially the following paragraph made me think:

Jones recalled the period just before transformers emerged, when researchers were endlessly tweaking recurrent neural networks — the previous dominant architecture — for incremental gains. Once transformers arrived, all that work suddenly seemed irrelevant. “How much time do you think those researchers would have spent trying to improve the recurrent neural network if they knew something like transformers was around the corner?” he asked.

What, if we’re just on another plateau, a local minimum between real improvements compared to the changes from one LLM generation to the next?

It is sad that people who try different approaches and don’t succeed don’t get the acknowledgement and recognition that people who make these breakthrough discoveries get. Just like with transformers.