2

I have a mpirun version of mpirun (Open MPI) 1.8.7 and using a Centos7 operated cluster.

To set my firewall configs between the nodes, I need to know which ports does MPI use? Or set specific ports range to mpirun commands?

Looking at the man page, there is no such option. I tried to set some environment variables but didn't work.

east.charm
  • 480
  • 1
  • 7
  • 17
  • 1
    Do you really want a firewall between nodes? Most people just have one towards the outside world. Packet filtering between your nodes might impact your performance. – haraldkl Sep 23 '15 at 21:13

1 Answers1

4

You can limit the port range in the openmpi-mca-params.conf (from that link:)

By default, two files are searched (in order):

$HOME/.openmpi/mca-params.conf: The user-supplied set of values takes the highest precedence.

$prefix/etc/openmpi-mca-params.conf: The system-supplied set of values has a lower precedence.

The following settings appear to be relevant for the used ports:

         MCA btl: parameter "btl_tcp_port_min_v4" (current value:
                  "1024", data source: default, level: 2 user/detail,
                  type: int)
                  The minimum port where the TCP BTL will try to bind
                  (default 1024)
         MCA btl: parameter "btl_tcp_port_range_v4" (current value:
                  "64511", data source: default, level: 2
                  user/detail, type: int)
                  The number of ports where the TCP BTL will try to
                  bind (default 64511). This parameter together with
                  the port min, define a range of ports where Open
                  MPI will open sockets.

         MCA oob: parameter "oob_tcp_static_ipv4_ports" (current
                  value: "", data source: default, level: 9 dev/all,
                  type: string)
                  Static ports for daemons and procs (IPv4)
         MCA oob: parameter "oob_tcp_dynamic_ipv4_ports" (current
                  value: "", data source: default, level: 9 dev/all,
                  type: string)
                  Range of ports to be dynamically used by daemons
                  and procs (IPv4)

See ompi_info --param btl tcp --level 9 and ompi_info --param oob tcp --level 9. Setting these values should allow you to limit the port range, that OpenMPI will attempt to use. But I am not quite sure how this would improve your security? Most attackers would be able to find any open port range? It usually is more important, what is actually listening on those ports, isn't it? Better move your firwall outside your cluster and make be sure of which services are listening on the open ports.

haraldkl
  • 3,809
  • 26
  • 44
  • I am not a security expert. I'm using firewalld (instead of iptables)Turning off the F Firewall would be great if it doesn't cause security threat or if there is another way to know who is listening on the ports. I added a rule to allow all traffic between nodes instead. Thank you for your insightful answer. – east.charm Sep 23 '15 at 22:21
  • 1
    @east.charm I am neither a security expert, but I know that HPC systems usually do not employ a firewall on their nodes. Also getting security right is hard. But you need to think about the threats you want to counter and what to do about. Just applying some random security measure will only give you the feeling of raised security. Usually the nodes should not be reachable from the outside world, only the frontends. Access to those frontends you should probably lock down as much as possible, but blocking ports between nodes? Which benefit does it offer? – haraldkl Sep 24 '15 at 04:54