FreeBSD pf with NFS

Question

I'm having trouble establishing new NFS connections when my pf is turned on.

When I turn off pf on the server and client I'm able to mount with no trouble.

I tried keep state and flags S/SA on the server config, but that gave the same results. Am I setting up the flags wrong?

Client pf.conf:

pass in on $int_if inet proto { tcp udp } from $nfs_int_ip to ($int_if) port { sunrpc nfsd-status nfsd-keepalive nfsd lockd }
pass out on $int_if inet proto { tcp udp } from ($int_if) to $nfs_int_ip port { sunrpc nfsd-status nfsd-keepalive nfsd lockd }

Server pf.conf:

pass in on $int_if inet proto { tcp udp } from $int_subnet to ($int_if) port { sunrpc nfsd-status nfsd-keepalive nfsd lockd }
pass out on $int_if inet proto { tcp udp } from ($int_if) to $int_subnet port { sunrpc nfsd-status nfsd-keepalive nfsd lockd }

Below is the output of sudo tcpdump -tttnnnvvv -i pflog0 from the server for when I try to mount on the client:

00:00:27.567834 IP (tos 0x0, ttl 64, id 13093, offset 0, flags [none], proto UDP (17), length 84)
    10.132.42.85.817 > 10.132.42.148.111: [udp sum ok] UDP, length 56
00:00:00.009245 IP (tos 0x0, ttl 64, id 54776, offset 0, flags [none], proto UDP (17), length 56)
    10.132.42.148.111 > 10.132.42.85.817: [udp sum ok] UDP, length 28
00:00:00.003862 IP (tos 0x0, ttl 64, id 13094, offset 0, flags [DF], proto TCP (6), length 60)
    10.132.42.85.871 > 10.132.42.148.2049: Flags [S], cksum 0x97b8 (correct), seq 1435157682, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 1292227501 ecr 0], length 0
00:00:00.000283 IP (tos 0x0, ttl 64, id 54777, offset 0, flags [DF], proto TCP (6), length 60)
    10.132.42.148.2049 > 10.132.42.85.871: Flags [S.], cksum 0x2967 (correct), seq 3548396325, ack 1435157683, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 1375604124 ecr 1292227501], length 0
00:00:00.001210 IP (tos 0x0, ttl 64, id 13095, offset 0, flags [DF], proto TCP (6), length 52)
    10.132.42.85.871 > 10.132.42.148.2049: Flags [.], cksum 0x541f (correct), seq 1, ack 1, win 1040, options [nop,nop,TS val 1292227504 ecr 1375604124], length 0
00:00:00.000260 IP (tos 0x0, ttl 64, id 13096, offset 0, flags [DF], proto TCP (6), length 96)
    10.132.42.85.2847810760 > 10.132.42.148.2049: 40 null
00:00:00.000608 IP (tos 0x0, ttl 64, id 54778, offset 0, flags [DF], proto TCP (6), length 52)
    10.132.42.148.2049 > 10.132.42.85.871: Flags [.], cksum 0xe635 (correct), seq 1, ack 45, win 29127, options [nop,nop,TS val 1375604130 ecr 1292227504], length 0
00:00:00.000139 IP (tos 0x0, ttl 64, id 54779, offset 0, flags [DF], proto TCP (6), length 80)
    10.132.42.148.2049 > 10.132.42.85.2847810760: reply ok 24 null
00:00:00.001038 IP (tos 0x0, ttl 64, id 13097, offset 0, flags [DF], proto TCP (6), length 52)
    10.132.42.85.871 > 10.132.42.148.2049: Flags [F.], cksum 0x53d0 (correct), seq 45, ack 29, win 1040, options [nop,nop,TS val 1292227504 ecr 1375604130], length 0
00:00:00.000062 IP (tos 0x0, ttl 64, id 54780, offset 0, flags [DF], proto TCP (6), length 52)
    10.132.42.148.2049 > 10.132.42.85.871: Flags [.], cksum 0xe618 (correct), seq 29, ack 46, win 29127, options [nop,nop,TS val 1375604130 ecr 1292227504], length 0
00:00:00.000183 IP (tos 0x0, ttl 64, id 54781, offset 0, flags [DF], proto TCP (6), length 52)
    10.132.42.148.2049 > 10.132.42.85.871: Flags [F.], cksum 0xe617 (correct), seq 29, ack 46, win 29127, options [nop,nop,TS val 1375604130 ecr 1292227504], length 0
00:00:00.000332 IP (tos 0x0, ttl 64, id 13098, offset 0, flags [none], proto UDP (17), length 84)
    10.132.42.85.895 > 10.132.42.148.111: [udp sum ok] UDP, length 56
00:00:00.000557 IP (tos 0x0, ttl 64, id 13099, offset 0, flags [DF], proto TCP (6), length 52)
    10.132.42.85.871 > 10.132.42.148.2049: Flags [.], cksum 0x53cf (correct), seq 46, ack 30, win 1040, options [nop,nop,TS val 1292227504 ecr 1375604130], length 0
00:00:00.000555 IP (tos 0x0, ttl 64, id 54782, offset 0, flags [none], proto UDP (17), length 56)
    10.132.42.148.111 > 10.132.42.85.895: [udp sum ok] UDP, length 28

Let me know if I'm missing any information.

I don't any problem But i ave a question , Do you use RDMA? if yes, add `nfsrdma` to port list — PersianGulf, May 04 '15 at 19:32
I'm not sure how to get that. When I start `nfsd` it says `NFSv4 is disabled` — Samer, May 04 '15 at 19:32
@MohsenPahlevanzadeh I don't think so, I haven't explicitly enabled it. — Samer, May 04 '15 at 19:34
I have another suggesstion: disable all of pf rule set in two side(client and server) then at first test nfs — PersianGulf, May 04 '15 at 19:40
I updated my question: When I turn off `pf` on the server and client I'm able to mount with no trouble. — Samer, May 04 '15 at 19:44
Let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/23436/discussion-between-mohsen-pahlevanzadeh-and-samer). — PersianGulf, May 04 '15 at 19:45
Try tcpdumping it on $int_if, not pflog0. But i kind of suspect it's not pf blocking nfs itself, but something else.... btw. not sure if you have it, but it's good (at least for debugging) to have `set block-policy return`. — Fox, May 04 '15 at 22:26

PersianGulf · Answer 1 · 2015-05-04T22:23:26.467

1

NOTE: 192.168.1.9 is client and 14 is server and is work fine in my two server ....
Client pf.conf :

block all
pass in on em0 inet proto { tcp udp } from 192.168.1.14 to em0 port { sunrpc nfsd-status nfsd-keepalive nfsd lockd }
pass out on em0  inet proto { tcp udp } from em0 to 192.168.1.14 port { sunrpc nfsd-status nfsd-keepalive nfsd lockd }

Server pf.conf :

block all
pass in on em0 inet proto { tcp udp } from 192.168.1.9 to em0 port { sunrpc nfsd-status nfsd-keepalive nfsd lockd }
pass out on em0 inet proto { tcp udp } from em0 to 192.168.1.9  port { sunrpc nfsd-status nfsd-keepalive nfsd lockd }

edited May 04 '15 at 22:23

answered May 04 '15 at 19:55

PersianGulf

602
8
21

Now, i set above rule set with my eyes, but i write with logic. wait. – PersianGulf May 04 '15 at 20:02
lol what? sorry I just saw this. – Samer May 04 '15 at 20:11
So those rules are similar to what I have. I know in Linux `iptables` there is the `NEW, ESTABLISHED` that we can pass, is that the problem that is happening to me? – Samer May 04 '15 at 20:21
@Samer , i updated my answer, They work fine, – PersianGulf May 04 '15 at 22:19
i reupdate and add `block all` – PersianGulf May 04 '15 at 22:23
Yeah I already have block all and lots more in my file, I only pasted the NFS stuff. – Samer May 04 '15 at 22:24
Yeah , But i didn't post for this answer, i know you did it. – PersianGulf May 04 '15 at 22:26

score 0 · Answer 2 · edited Apr 11 '23 at 21:44

These pf rules work for me. Port 797 was added for mountd. There is a usefull guide PF: Testing Your Firewall

/etc/rc.conf:

mountd_flags="-r -S -p 797"

NFS clients:

- nfs_ports="{ sunrpc, nfsd-status, nfsd-keepalive, nfsd, lockd, 797 }"
- nfs_server="10.132.0.2"
- pass in on $ext_if proto tcp from $nfs_server to $ext_if port $nfs_ports keep state flags S/SA
- pass in on $ext_if proto udp from $nfs_server to $ext_if port $nfs_ports                                                                                                                  
- pass out on $ext_if proto tcp from $ext_if to $nfs_server port $nfs_ports keep state flags S/SA
- pass out on $ext_if proto udp from $ext_if to $nfs_server port $nfs_ports

NFS server:

- nfs_ports="{ sunrpc, nfsd-status, nfsd-keepalive, nfsd, lockd, 797 }"
- nfs_clients="{ 10.132.0.3, 10.132.0.4 }"
- pass in on $ext_if proto tcp from $nfs_clients to $ext_if port $nfs_ports keep state flags S/SA
- pass in on $ext_if proto udp from $nfs_clients to $ext_if port $nfs_ports                                                                                                                  
- pass out on $ext_if proto tcp from $ext_if to $nfs_clients port $nfs_ports keep state flags S/SA
- pass out on $ext_if proto udp from $ext_if to $nfs_clients port $nfs_ports

FreeBSD pf with NFS

2 Answers2