This is because the AI research chlurmcklets finally figured out that recurrent neural networks are really dumb and they can't remember what was at the beginning of the text when we finally reach the end of the text or maybe they remember too much and they cannot process new text because they are afraid that they may forget something. Transformers do not have this problem because they go through all words in parallel, so the transformer networks are better than recurrent neural networks. Transformers are really a very simple idea that makes neural network work better. Oh. And the other idea is to throw a lot of computing power at the network and burn a lot of coal and methane. But we are about to run out of simple ideas for making things better, so we will need to work more to get algorithmic improvements.

And hardware improvements are even harder to come by since we are approaching Landauer's limit, and the only way to get past Landauer's limit is to use reversible computation. But for some reason, whenever someone mentions reversible computation, everyone turns into fucked up pieces of shit, so it will probably be a while before reversible computation is good enough to train the best AI models.

-Regards,

-Joseph Van Name Ph.D.