5

I have an application running on my host which has the following features: it listens to port 4001 (configurable) and only accepts connections from a whitelist of trusted IP addresses (127.0.0.1 only by default, other addresses can be be added but one by one, not using a mask).

(It's the interactive brokers gateway application which is run in java but I don't think that's important)

I have another application running inside a docker container which needs to connect to the host application.

(It's a python application accessing the IB API, but again I don't think that matters)

Ultimately I have will multiple containers on multiple machines trying to do the same thing, but I can't even get it working with one running on the same machine.

sudo docker run -t  myimage

Error: Couldn't connect to TWS.  Confirm that "Enable ActiveX and Socket Clients" is enabled on the TWS "Configure->API" menu.

(No response from IB Gateway on host machine)

IDEALLY I'd be able to set up the docker containers / bridge so that all the docker containers appear as if they are on a specific IP address, add it to the whitelist, and voila.

What I've tried:

1) using -p and EXPOSE

sudo docker run -t -p 4001:4001 myimage

Bind for 0.0.0.0:4001 failed: port is already allocated.

(No response from gateway)

This eithier doesn't work or leads to a "port already in use" conflict. I gather that these settings are designed for the opposite problem (host can't see a particular port on the container).

2) setting --net=host

sudo docker run -t --net=host myimage

Exception caught while reading socket - Connection reset by peer

(no response from gateway)

This should work since the docker container should now look like it's 127.0.0.1... but it doesn't.

3) setting --net=host and adding the local host's real IP address 192.168.0.12 (as suggested in comments) to the whitelist

sudo docker run -t --net=host myimage

Exception caught while reading socket - Connection reset by peer

(no response from gateway)

4) adding 172.17.0.1, ...2, ...3 to the whitelist on the host application (the bridge network is 172.17.0.0 and subsequent containers get allocated in this range)

sudo docker run -t  myimage

Error: Couldn't connect to TWS.  Confirm that "Enable ActiveX and Socket Clients" is enabled on the TWS "Configure->API" menu.

(no response from host)

This is horribly hacky but doesn't work eithier.

PS Note this is different from the problem of trying to run the host application IB Gateway inside a container - I am not doing that.

I don't want to run the host application inside another container, although in some ways that might be a neater solution.

halfer
  • 19,824
  • 17
  • 99
  • 186
robcarver
  • 76
  • 1
  • 5
  • So, your question is how to add the containers ips to the whitelist? – mayid Jan 19 '17 at 15:58
  • I can add them to the whitelist, but (a) it doesn't work (There is something else going wrong: the containers can't access port 4001 on the host computer) and (b) it's a very unsatisfactory solution as there is no guarantee what the IP addresses are and it could mean adding a hundred individual IP addresses – robcarver Jan 19 '17 at 16:00
  • Could you explain this more? "IDEALLY I'd be able to set up the docker containers / bridge so that all the docker containers appear as if they are on a specific IP address, add it to the whitelist, and voila." – Mohammed Noureldin Jan 19 '17 at 16:13
  • @MohammedNoureldin If I run my containers with *docker run* they will be on the bridge network. *docker network inspect bridge* then I can see the IP addresses of the containers I am running; 172.17.0.1, ...172.17.0.2 and so on. I want to avoid adding these seperately to the white list (which doesn't work anyway!) Running *docker run --net=host myimage* should mean that all the containers are on the host network and look like they are 127.0.0.1 and on the whitelist. But this doesn't seem to work eithier. Ideal would all containers appearing as 172.17.0.0 (docker bridge IP address). – robcarver Jan 19 '17 at 16:35
  • Is there no verbose logging option in the IB app? localhost is an alias, not a host. Have you tried -net=host and using the hosts IP address? – user2105103 Jan 19 '17 at 17:02
  • Note that once you have found your solution, there is no need to substantially amend your question. We also put answers below, rather than adding them as an addendum to the question. Thanks! – halfer Jan 20 '17 at 17:16

4 Answers4

4

Running the IB gateway is tricky on a number of different levels, including connecting to it, and especially if you want to automate the process.

We took a close look at connecting to it from other IPs, and finally gave up on it--gateway bug as far as we could tell. There is a setting to white IPs that can connect to the gateway, but it does not work and can not be scripted.

In our build process we create a docker base image, then add the gateway and any/all of the gateway's clients to that image. Then we run that final image.

Pablo Rodriguez Bertorello
  • 260
  • 3
  • 8
3

(Posted on behalf of the OP).

Setting --net=host and changing the port from 4001 so it doesn't conflict with a live version of the gateway on the same network. The only IP address required in the whitelist is 127.0.0.1.

sudo docker run -t --net=host myimage
halfer
  • 19,824
  • 17
  • 99
  • 186
3

Use socat to forward port from the gateway to a new port which can listen on any address. For example, set the gateway to listen on port 4002 (localhost only) and use command in the container

socat tcp-listen:4001,reuseaddr,fork tcp:localhost:4002

to forward the port to 4001. Then you can connect to the gateway from outside of the container using port 4001 when running the container with parameter -p 4001:4001.

Pavel
  • 121
  • 4
1

In case this one is useful for another person. I tried a couple suggestions that were put here to connect from my python app running on a Docker container to a TWS IBGateway instance running on another server and none of them were 100% working. The socat option was connecting, but then the connection was being drop due an issue with the socat buffer that we couldn't fix. The solution we found was to create an ssh tunnel from the machine that is running the Docker container to the machine that is running the TWS IBGateway.

ssh -i ib-gateway.pem <ib-gateway-server-user>@<ib-gateway-server-ip> -f -N -L 4002:127.0.0.1:4001

After you establish this ssh tunnel, you can test it running

telnet 127.0.0.1 4002

If this command run successfully, your ssh tunnel is ready. The next step would be to configure your python application to connect to 127.0.0.1 on port 4002 and start your docker container with --net=host to be able to access the ssh tunnel running on Docker host machine.

Fabri Pautasso
  • 485
  • 6
  • 17