How does such a "private cloud" differ from using the "public cloud"? The two seem identical to me.
It's not even a binary at many of the large providers: you can have dedicated servers ("my own server") in a public cloud at AWS, for example.
Personally, I keep it simple. If I can physically hit a box and no-one can get mad at me, then it's private and mine.