I absolutely agree that latency is the real thing to optimize for. In my case, I only leave the application to access the db, and my applications tend not to be write-heavy. So in my case latency-per-request == how much work the computer has to do, which is constrained to one core because the overhead of parallelizing any part of the pipeline is greater than the work required. See, in that sense, we’re already close to the performance ceiling for per-request processing because clock speeds aren’t going up. You can’t make the processing of a given request faster by throwing more hardware at it. You can only make it faster by creating less work for the hardware to do.

(Ironically, HN is buckling under load right now, or some other issue.)