Actually, how do you debug those? If you put in a breakpoint, use DTrace & co or do logger output, you won't likely hit the issue. I did this with super low-level distributed systems and they were mostly resolved by hard thinking and modeling execution path in my head...
Once when developing with the 8051 family (using a development board that included a debugger), the program worked perfectly on the dev board, but when I burned it onto EEPROM, a subtle timing issue with a peripheral caused the program to fail. Much frustration ensued; had to develop that section with a long winded code->compile->burn->test cycle. Then we found out the particular 8051 variant we were using had been discontinued :(
Reason through the code. Sometimes, reason through the output assembler (in case the C code is making invalid assumptions). Stare at oscilloscope traces. Think really hard about what's happening, come up with a hypothesis, and test that hypothesis.
The hypothesis testing part is key. I've worked with embedded developers who "fix" a bug by making a code change, but can't at all explain why the change is likely to fix the bug. In many cases, the reason it seems to fix the bug is that it changes the timing just enough so that the bug happens less frequently, but without having a really solid causal explanation, it likely just means that it's going to be even harder to find the bug when it pops back up again.
