When the article was written, almost nobody cared about GPUs or similar architectures being really good at parallel computation.
So while he wrote about MIPS, and those have indeed basically stopped significantly improving, FLOPS have continued to improve. And for AI in particular, for inference at least, we can get away with 8 bit floats, which is why my phone does 1.58e13/second, only a factor of x60 from a million-billion.
So while he wrote about MIPS, and those have indeed basically stopped significantly improving, FLOPS have continued to improve. And for AI in particular, for inference at least, we can get away with 8 bit floats, which is why my phone does 1.58e13/second, only a factor of x60 from a million-billion.