Breakthroughs are not a myth. They still happen even when the process is iterative. That page even explains it. The advent of the GAN (2014-2018), which got overtaken by transformers in around 2017 for which GPTs and Diffusion models later got developed on. More hardware is what allowed those technologies to work better and bigger but without those breakthroughs you still wouldnt have the AI boom of today.
Breakthroughs are more or less of a myth. Everything is iterative.
Breakthroughs are not a myth. They still happen even when the process is iterative. That page even explains it. The advent of the GAN (2014-2018), which got overtaken by transformers in around 2017 for which GPTs and Diffusion models later got developed on. More hardware is what allowed those technologies to work better and bigger but without those breakthroughs you still wouldnt have the AI boom of today.
Agreed. The only thing that has really changed is how much hardware we can throw at it. ML has existed more or less since the 60s.