A private cloud can be configured as though it were an extension of your LAN or datacentre, and not directly connected to the Internet. That is, they can be addressed with private IPv4 addresses, making them unreachable from the Internet. Via a VPN, the private cloud operates as though it were an extension of your LAN or co-lo facility.
Even in a private cloud you're typically dealing with multi-tenancy. Other customers have VPSs running on the same hardware you're running on. However, unless they're able to take advantage of some unknown virtualisation exploit, they might as well be running on isolated hardware... they cant get to your VPS on 192.168.1.1 (or whatever address you've configured).
A public cloud is inherently reachable from the Internet (at least one interface has a public IP address). Virtual machines in a private cloud can only access (or be accessed) the Internet via a NAT instance or via VPN (they can be configured to only have a private IP address).
- Thanks to TomTom for the correction re: multi-tenancy. A private cloud should not be thought of as multi-tenant. I was thinking of Amazon Virtual Private Cloud when answering, and while that's multi-tenant, I think it's more accurate to say that Amazon VPC is not strictly speaking a private cloud rather than saying because of Amazon VPC, multi-tenant hosts can be thought of as private cloud.