Of course somebody is paying your 3%; there's no free lunch. To a large extent it's yourself (via interchange paid as part of the merchant's fees); part of it is however also other people paying cash and often not receiving a discount for doing so.
Regardless of all that, at the individual level, not using a credit card is still irrational. Not quite the tragedy of the commons, but it's pretty close.
The only way out is probably outright regulating the entire thing away, like the EU has successfully done; the market seems too inefficient and consumer behavior too sticky for card-specific surcharges (i.e. "you pay for your own points") alone to solve this.
not an argument for or against cards, but they are becoming closer to neutral than a discount.