Tor relay/exit nodes under preemptible VMs

Question

I have a paid subscription to the Google Cloud Platform, and I want to run Tor relay and exit nodes. I use Docker image quantumobject/docker-tor-exit-relay, just plug&play.

In a nutshell, per the answers received:

• g1-micro VM configuration is the cheapest alternative, but may not perform as at high performance I'm want to offer.
• Custom VM with 1 vCPU and 1GB RAM, with commited usage for 3 years, seems to be the most convenient and affordable.
• Preemptible VM class is useless.
• Several instances behind a Load balancer is a bad idea.

My original rationale

As the preemptible VMs are much cheaper, I wanted to use instances groups with preemptible VMs. Then, in order to have a high availability nodes, I'm planning to use GCP load balancer, in order to keep only one IP address and at least one VM running at any moment.

Also, I've originally planned to use the g1-small VM type (1/2 shared vCPU and 1.7 GB RAM) for costs, and I'm comparing with other alternatives like regular class VMs with "commited usage" (that apply discounts).

Investment

Following are the estimated costs for VMs (according to the calculator):

Config.     | Class       | vCPU | RAM    | C. usage | Price/Mo
----------------------------------------------------------------
g1-micro    | Regular     | 0.2  | 600MB  | Not set  | US$3.88
g1-small    | Preemptible | 0.5  | 1.70GB | Not set  | US$5.11
g1-small    | Regular     | 0.5  | 1.70GB | Not set  | US$13.80
n1-standard | Regular     | 1    | 3.75GB | 3 years  | US$15.60
Custom      | Regular     | 1    | 1GB    | 3 years  | US$11.78
Custom      | Regular     | 1    | 2GB    | 3 years  | US$13.17

vCPUs are Intel Xeon @ 2.3 GHz, Haswell microarchitecture.

Additionally, we need the estimated egress traffic (montly consummed bandwith) costs (useful for groups rather than single VMs):

BW    | Zone 1 | Zone 2 | Zone 3 | Zone 4 | Price
-----------------------------------------------------
70GB  | 40GB   | 15GB   | 5GB    | 10GB   | US$9.42
80GB  | 80GB   | 30GB   | 10GB   | 20 GB  | US$19.27
350GB | 200GB  | 75GB   | 25GB   | 50 GB  | US$48.82
700GB | 400GB  | 150GB  | 50GB   | 100 GB | US$98.07

Zone 1: Americas/EMEA
Zone 2: Asia/Pacific
Zone 3: Australia
Zone 4: China

VM specs

Currently, I'm testing a VM with the folowing configuration:

• 1 vCPU Inten Xeon @ 2.30GHz (Haswell)
• 1GB RAM
• OS image: cos-stable-71-11151-71-0
  • Kernel: ChromiumOS-4.14.74
  • Kubernetes: 1.11.5
  • Docker: 18.06.1
  • Family: cos-stable
• Docker image: quantumobject/docker-tor-exit-relay
• Startup script: run -d -p 2222:22 -p 80:80 -p 9050:9050 -p 9001:9001 quantumobject/docker-tor-exit-relay
• VM class: regular
• Region: us-central1

Tip: Go to VPC network > Firewall and create a rule called "allow-tor", and set:

• Priority: 500 (anything lower than the default 1000)
• Targets: Specified target tags
• Target tags: allow-tor
• Protocols and ports: Specified protocols and ports: "tcp:22,80,443,9001,9050"

When creating the VM/Template, go to Management, security, disks, networking, sole tenancy, select Networking, and set the tag you set in Firewall settings.

Benchmark

Also, I ran an openssl benchmark (openssl speed aes for 3 seconds each cbc) in both g1-small and Custom 1 GB RAM 1 vCPU instance, and I got the following results:

g1-small:

type         16 bytes    64 bytes    256 bytes   1024 bytes  8192 bytes  16384 bytes
aes-128 cbc  102714.85k  114937.26k  118729.12k  119713.45k  120548.01k  120684.54k
aes-192 cbc  87781.72k   96917.15k   99274.67k   100145.83k  100463.96k  100515.84k
aes-256 cbc  76597.38k   83198.89k   84709.38k   85080.02k   85516.29k   85859.83k

Custom:

aes-128 cbc  100811.34k  114144.42k  116054.50k  117985.42k  119586.42k
aes-192 cbc  85914.78k   95220.79k   98367.39k   98994.68k   99095.15k
aes-256 cbc  76171.38k   82969.05k   84497.50k   85145.08k   84728.69k

Performance is nearly the same, maybe because the shared vCPU receives full power at certain moments when needed, but not always, compared with a dedicated 1 vCPU.

My questions:

• Which bandwidth amount is recommended for every instance?

• Is safe to run one or more Tor relay/exit nodes behind one Load balancer in the GCP? Nope.

• Is safe/useful to run preemptible nodes that shuts down at 24 hours of running or less? (IP address assignment are dynamic and the same IP for a new node is not guarranteed) Safe but useless.

• Are the g1-micro or g1-small VM specs enough to run a decent performance node? Custom 1GB RAM VM is cheaper and more powerful.

Thanks in advance.

Answer 1

Running multiple preemptible nodes for redundancy is a bad plan. If the preemptible nodes are running in the same region they are quite likely to be shut down simultaneously due to increasing demand in the region.

Preemptible nodes in different regions are less likely to be shut down simultaneously, but the risk still exists and can depend on what other GCE customers are doing, which is entirely outside your control.

Don't use preemptible nodes if you care about availability. They are suitable for batch processing where being preempted isn't much of a problem and you can just do the processing at a later time.

Answer 2

Is safe to run preemptible nodes that shuts down at 24 hours of running or less? (IP address assignment are dynamic and the same IP for a new node is not guarranteed)

Safe? Sure. But you'll never see any traffic to them. There is an excellent writeup on the way Tor vets new relays and exit nodes here: The lifecycle of a new relay

Essentially any new relay is "tested out" for 60+ days before it sees any real traffic. I can vouch that this is accurate as I've been running a Tor relay for about 2 years. You're unlikely to see any real traffic unless your nodes keep the same IDs each time they are restarted, and if your node cannot prove to be reliable it will likely be placed lower down in the food chain.

Additionally if the node goes offline for a long time, you may have to re-sit those waiting periods. My relay was offline for 3 months as I moved countries and I'm yet to see traffic return to what it used to be.

You can see what Tor believes about your relay by looking it up on their metrics page. For example, here is mine.

Are the g1-small VM specs enough to run a high performance node?

Tor's idea of high performance is different to the general internet at large. For example I am running mine on a pretty old Intel NUC with a pretty old Intel Celeron processor and a sad amount of RAM and it caps out at about 70Mbps - because that's all that OpenSSL can push through the CPU. 70Mbps is very large for a Tor relay (maybe not an exit node). I'd imagine even a basic semi-modern small VM would be able to outpace it.