1

I'm using Open MPI 1.6.5 to run OpenFOAM in parallel on 3 nodes. I'm allowed to open only a few TCP ports for security reasons. So I opened ports 49990-50009 for Open MPI and set the values in openmpi-mca-params.conf as follows:

btl_tcp_port_min_v4=49990    
btl_tcp_port_range_v4=10   
oob_tcp_static_ports=50000-50009

When I ran mpirun, I got a message:

mca_oob_tcp_init: unable to create IPv4 listen socket: Unable to open a TCP socket for out-of-band communications.

Did I miss something? How can I set MPI to run a range of ports?

Cel Skeggs
  • 1,827
  • 21
  • 38
Ping Wang
  • 11
  • 1
  • 2

1 Answers1

0

The value of oob_tcp_static_ports should be a comma-separated list of specific ports to be used, not to a range of ports. To set a port range for the tcp OOB, assign it to oob_tcp_dynamic_ports instead.

Note that all those port numbers (also for the tcp BTL) affect the listening sockets, i.e. the incoming connections. The connection initiator side uses whatever port number the OS binds the socket to.

Reference - the Open MPI user's mailing list.

Hristo Iliev
  • 72,659
  • 12
  • 135
  • 186
  • Hi Hristo, I assigned the port range to oob_tcp_dynamic_ports=50000-50009. When I try to start the simulation with the command: mpirun -np 4 --hostfile machines simpleFoam -parallel. The simulation hangs and there is no error message. It seems the communication between nodes is still blocked. What else should I set? – Ping Wang May 30 '16 at 09:20
  • 1
    Add `--mca btl_base_verbose 20 --mca oob_base_verbose 10` to the `mpiexec` command line in order to get plenty of debug information from both frameworks, including the addresses and the ports the library is using. Also, make sure that the firewall is filtering connections based on the destination port and not on the source one. By the way, if the cluster uses a high-speed network such as InfiniBand, you don't have to assign any ports to the `tcp` BTL as it won't be used at all and you can give the entire port range to the `tcp` OOB. – Hristo Iliev May 30 '16 at 11:20
  • Hi Hristo, thanks for your reply. How can I set the firewall to filter connection based on the destination port? I'm using ubuntu 14.04. – Ping Wang May 31 '16 at 21:00
  • It depends on the firewall used. With `iptables` something like this: `iptables -A INPUT --match multiport --dports 49990:50009 -j ACCEPT` – Hristo Iliev Jun 01 '16 at 13:25