5

I would like to reach my NFSv4 servers though port forward. The big plan will be a cluster of NFSv4 servers loadbalanced with HAProxy running on localhost. But this isn't really important now.

At the server the /etc/exports looks like this

/mnt/x  192.168.0.0/16(rw,sync,no_subtree_check,no_root_squash,fsid=1)

I can connect from my client to the server on TCP 2049 and mount the share like this

mount -t nfs4 -o proto=tcp,port=2049 192.168.2.25:/mnt/x /mnt

I tested that NFVs4 is happy with only this one TCP port open by filtering all other communication between the two machines.

So I think NFS works well.

But when I forward a port for example with redir on the client to the server like

redir --lport=3049 --cport=2049 --caddr=192.168.2.25

and want to mount it as follows

mount -t nfs4 -o proto=tcp,port=3049 127.0.0.1:/mnt/x /mnt

i get

mount.nfs4: Operation not permitted

What I miss? I can't see any relevant information in the server logs.

Update: I captured both the good and the bad connection attempt, at the beginning they are the same then the client sends a

PUTROOTFH,GETFH,GETATTR

command. In the good case the servers responds

PUTROOTFH-NFS4_OK,GETFH-NFS4_OK,GETATTR-NFS4_OK

in the bad (forwarded) case it responds

PUTROOTFH-NFS4_OK,GETFH-NFS4_OK,GETATTR-NFS4ERR_PERM

This point I changed the export to

/mnt/x  0.0.0.0/0.0.0.0(rw,sync,no_subtree_check,no_root_squash,fsid=1)

but the error is the same.

In the good case the server logs are

rpc.mountd[1711]: nfsd_export: inbuf '0.0.0.0/0.0.0.0 /'
rpc.mountd[1711]: nfsd_export: found 0x12dfeb0 path /
rpc.mountd[1711]: nfsd_export: inbuf '0.0.0.0/0.0.0.0 /mnt'
rpc.mountd[1711]: nfsd_export: found 0x12e2810 path /mnt

in the bad case

rpc.mountd[1711]: nfsd_export: inbuf '0.0.0.0/0.0.0.0 /'
rpc.mountd[1711]: nfsd_export: found 0x12dfeb0 path /
Stone
  • 7,011
  • 1
  • 21
  • 33
  • Forwarding `2049/tcp` is not enough, there are several ports involved in NFSv4 protocol operation. Check the `/etc/sysconfig/nfs` file (or equivalent if not in RHEL) for the default values. – dawud Feb 21 '14 at 13:23
  • That will almost certainly be an SELinux issue. But it's also not a good test, since in production you'll be forwarding port 2049 from one machine to port 2049 of another machine. Set up a test with exactly those parameters instead. – Michael Hampton Feb 21 '14 at 13:24
  • @dawud A pure NFSv4 environment only needs ports 2049 and maybe 111. Though I think maybe even that is unused as well; my firewall on my NFsv4 server records no traffic on that port. – Michael Hampton Feb 21 '14 at 13:26
  • In my tests I dropped everything between the two machines but 2049 and the mount was successful. See the top of my question. – Stone Feb 21 '14 at 13:28
  • 1
    There is no SELinux or any other MAC on the machines. – Stone Feb 21 '14 at 13:30
  • @Stone does `redir` keep the source IP intact before redirecting? Or IOW, is the client using the 192.x.x.x IP address to contact the server? anything useful in the NFS server logs? Can you capture (`tcpdump`) a connection attempt? – dawud Feb 21 '14 at 13:35
  • @dawud `redir` terminates the TCP in it's one side and opens a new TCP to the other side. So in both cases the client address is the same. – Stone Feb 21 '14 at 14:06

1 Answers1

6

My colleague spotted the solution in the tcpdump, that the only other difference between the good and bad connection attempts is the source port.

Then Google told me that I need the insecure option to the export, because after the forward the source port is above 1024 and in the normal case it is below 1024 and insecure tells to the server that it can accept clients with source port above 1024.

Stone
  • 7,011
  • 1
  • 21
  • 33
  • Huh, that did the trick. That's one hell of a braindead default option! I guess some guy really thought that clients that use a privileged port number are somehow more "secure" ¯\\_(ツ)_/¯ – Navin Jul 03 '17 at 00:32
  • Thanks for sharing. It solved this problem for me also. – Arkan Sep 15 '20 at 17:33