How worried should I be about unsecured HTTP on a "private" network?

Question

So I've joined a new team, which develops and operates a service available on the Internet. It's aimed at B2B use rather than consumer, although anyone can sign up for a low-tier account to check it out (you don't get anywhere if your potential customers' developers can't have a play before the suits sign a deal!).

I've recently learned, to my surprise, that some things I would expect to be served over HTTPS to a tightly controlled set of client certs are instead going over open HTTP with no authentication of any kind. Things like a Consul key/value store, in which some of the values are passwords, or a Docker private registry in which some of the images contain private keys (there's a project under way to remove keys from the images and inject them at runtime, but the old images are still in the registry and I wouldn't like to bet on the keys having been changed). There are probably other services in a similar position that I haven't found yet.

The colleague I asked about this agreed that it wasn't ideal, but was fairly unconcerned because these services are only exposed on the private network within the (third party hosted) datacenter. They're not (if everything's working right) routable from the Internet, thank god. Nevertheless, that seems like a lot of trust to put in the network routing config, not to mention if one of the servers does get compromised, its access to the internal network means the rest are sitting ducks.

This is my first gig running a public service, until now I've worked in the "shrinkwrap" world where we sell software but our customers install and run it. So I don't have a feel for quite how bad this is. I'm going to be raising it and trying to get it fixed, but I wanted to run it by a community with more experience running production services in order to calibrate how loud I should be shouting. Is this really terrible and we should drop everything till it's fixed, or not good but actually not that uncommon in reality?

Posting as a guest since I don't want to give any clues about whose service this is :)

Answer 1

I don't think this is much of a problem, with one caveat: provided the network that connects the private servers (whether virtual or physical) is switched, this isn't much of an issue.

My usual mantra is that there is no such thing as security in the abstract, only threats and responses thereto, appropriate and otherwise. So what's your threat model? An attacker compromises the internet-facing server; what can (s)he now do?

SSL (including HTTPS) on the wire provides two services, encryption and authentication. Encryption provides no protection against this threat model: provided the back-end network is switched, the attacker can only see traffic to and from the compromised server. Since that's an SSL endpoint, (s)he'd be able to read all that traffic anyway. Authentication provides a slight benefit, but it's only trace: even if they didn't use the usual self-signed certificates (which tend to gut authentication) you can be fairly confident that you're really talking to the back-end servers, because you control the internal network, and the threat model doesn't specify penetration of the network infrastructure.

In short: you write that "if one of the servers does get compromised, its access to the internal network means the rest are sitting ducks". This is true, but it's no less true just because the internal crosstalk is encrypted.

When you comment to Ryan above that "access from the office to the hosted private LAN is over a VPN, and operations on the production service are done from dedicated Linux laptops rather than devs' main machines", I see a company that's actually thinking about threat models and responses, rather than waving crypto around like some kind of shiny security blanket, and assuming that they're now safe, because crypto. It sounds like they work hard to secure the bits that benefit from security, and don't sweat the bits that don't.

Answer 2

I had a long conversation about this with a friend of mine. It boils down to what you're doing, and what your network looks like.

Generally: It's bad.

Cryptography has become much more simple to implement over the years, thus there really shouldn't be too much excuse to not have it implemented.

It really depends what's going over your wire though. Does that information need to be confidential/authenticated? Remember that although the destination may not be able to route to the Internet - your machine can. Technically if someone tapped your machine or any routing/switching device between you and your destination, your data is compromised. You should always assume that if your machine can reach the Internet, then someone out there could have all of the power you have. Just because the Internet can't reach your private LAN directly doesn't mean it can't reach it through compromising your machine.

This is an easily debatable topic, but generally not having encryption will lower your security posture. If you can do it, then do it. I would agree that your situation is pretty minimal risk, but still - just do it!