Imagine applying this same non-logic to car seat belts, motorcycle helmets, rock climbing harnesses, et multa alia. If you don't want to wear a helmet, that's fine, but don't bother trying to justify it with anything other than "I don't want to", because you really can't.

Even strong and careful skiers wipe out occasionally. A helmet is the difference between shaking it off and brain injury.