0

I was able to successfully deploy BOSH and CF on GCP. I was able to install the cf cli on my worker machine and was able to cf login to the api endpoint without any issues. Now I am attempting to deploy a python and a node.js hello-world style application (cf push) but I am running into the following error:

Python:

**ERROR** Could not install python: Get https://buildpacks.cloudfoundry.org/dependencies/python/python-3.5.4-linux-x64-5c7aa3b0.tgz: dial tcp: lookup buildpacks.cloudfoundry.org on 169.254.0.2:53: read udp 10.255.61.196:36513->169.254.0.2:53: i/o timeout
   Failed to compile droplet: Failed to run all supply scripts: exit status 14

NodeJS

-----> Nodejs Buildpack version 1.6.28
   -----> Installing binaries
          engines.node (package.json): unspecified
          engines.npm (package.json): unspecified (use default)
          **WARNING** Node version not specified in package.json. See: http://docs.cloudfoundry.org/buildpacks/node/node-tips.html
   -----> Installing node 6.14.3
          Download [https://buildpacks.cloudfoundry.org/dependencies/node/node-6.14.3-linux-x64-ae2a82a5.tgz]
          **ERROR** Unable to install node: Get https://buildpacks.cloudfoundry.org/dependencies/node/node-6.14.3-linux-x64-ae2a82a5.tgz: dial tcp: lookup buildpacks.cloudfoundry.org on 169.254.0.2:53: read udp 10.255.61.206:34802->169.254.0.2:53: i/o timeout
   Failed to compile droplet: Failed to run all supply scripts: exit status 14

I am able to download and ping the build pack urls manually on the worker machine, jumpbox, and the bosh vms so I believe DNS is working properly on each of those machine types.

As part of the default deployment, I believe a socks5 tunnel is created to allow communication from my worker machine to the jumpbox so this is where I believe the issue lies. https://docs.cloudfoundry.org/cf-cli/http-proxy.html When running bbl print-env, export BOSH_ALL_PROXY=ssh+socks5://jumpbox@35.192.140.0:22?private-key=/tmp/bosh-jumpbox725514160/bosh_jumpbox_private.key , however when I export https_proxy=socks5://jumpbox@35.192.140.0:22?private-key=/tmp/bosh-jumpbox389236516/bosh_jumpbox_private.key and do a cf push I receive the following error:

Request error: Get https://api.cloudfoundry.costub.com/v2/info: proxy: SOCKS5 proxy at 35.192.140.0:22 has unexpected version 83 TIP: If you are behind a firewall and require an HTTP proxy, verify the https_proxy environment variable is correctly set. Else, check your network connection. FAILED

Am I on the right track? Is my https_proxy variable formatted correctly? I also tried https_proxy=socks5://jumpbox@35.192.140.0:22 with the same result.

glux
  • 532
  • 7
  • 23
  • Can you check your application security groups? They are only applied to your staging and/or runtime containers, so it's an additional access control which is not applied when you test network access from your Cell or from your jump box. The default is to not allow anything and the ASGs allow out specific traffic. https://docs.cloudfoundry.org/concepts/asg.html – Daniel Mikusa Aug 28 '18 at 11:42
  • Below is the content of my default security_groups in my cf-deployment.yaml. If I am reading this correctly, It seems the rules should allow dns and public network traffic. https://pastebin.com/5pjhXd0G . – glux Aug 28 '18 at 15:53
  • yes, those look like they are defined OK. Can you also make sure they are applied properly? Run `cf staging-security-groups` and make sure it lists `public_networks` and `dns`. – Daniel Mikusa Aug 29 '18 at 11:32
  • The groups are present. https://pastebin.com/pc0fkY4n. Am I far off with my https_proxy hypothesis? – glux Aug 29 '18 at 15:52
  • OK, that's good. There are many possibilities, just trying to check off some of them. There definitely seems like a proxy issue somewhere. Where are you seeing the `unexpected version 83` error? Is that from your worker, trying to use the cf cli? There appears to be a different issue in the staging container. It is unable to make outbound connections. Normally that's unrestricted, so you might want to review your setup for outbound traffic. Could be blocked by firewall or your NAT gateway, if using one. – Daniel Mikusa Aug 30 '18 at 12:03
  • I did some further testing. I am able to download the build pack manually from my worker which is local, jumpbox and the bosh vms. I am unable to download the buildpack from the director. Looks like DNS is not resolving on the director only (https://pastebin.com/nqr2h0Be). I get the `unexpected version 83` only when I set the `https_proxy` environment variable to the same as `BOSH_ALL_PROXY`. If `https_proxy` is not set, I receive the `i/o timeout` error. See initial post for detailed error output. – glux Aug 30 '18 at 16:02
  • The tests you're running from the worker and bosh VMs are not relevant. The downloads from `buildpacks.cloudfoundry.org` that are failing are happening inside the staging container. A test from the director, a bosh deployed VM or from your worker is *not* the same. Unfortunately, it is really hard to test from inside a staging container because they are short lived. My suggestion would be to push a demo app that doesn't need to download anything, like something with the binary buildpack, then `cf ssh` into the container and test networking. It's not exactly the same but very close. – Daniel Mikusa Aug 31 '18 at 13:09

0 Answers0