"On the other hand, if this claim were actually true, it would invalidate basically the entire field of mathematical optimization and decision theory."
I might be able to help by pointing out that a technology that depends on random sampling of small components of "intelligence" or whatever is going to be totally screwed up if the sampling is no longer random and now the outcome of the test depends mostly on the amount of money spent on prep classes.
From manufacturing I can predict a lot about the quality of something in general off just one graph of the distribution of length of one part compared to other manufacturers graphs of the same single measurement on the same part, but if the mfgr "cooks the books" by optimizing one measurment of that part to perfection while taking away labor from other areas of part quality, then my comparisons will be useless, especially when comparing optimizers vs non-optimizers.
I might be able to help by pointing out that a technology that depends on random sampling of small components of "intelligence" or whatever is going to be totally screwed up if the sampling is no longer random and now the outcome of the test depends mostly on the amount of money spent on prep classes.
From manufacturing I can predict a lot about the quality of something in general off just one graph of the distribution of length of one part compared to other manufacturers graphs of the same single measurement on the same part, but if the mfgr "cooks the books" by optimizing one measurment of that part to perfection while taking away labor from other areas of part quality, then my comparisons will be useless, especially when comparing optimizers vs non-optimizers.