Background: we've been seeing incredibly slow performance allocating files, on a system that creates many small files on a 25T disk. We found from http://xfs.9218.n7.nabble.com/1B-files-slow-file-creation-only-AG0-used-td1606.html that indeed it seemed like all files were getting created in the first AG.
for x in {0..38}; do echo -n "${x}: "; xfs_db -c "agi ${x}" -c "print" -r /dev/md1 | grep "^count"; done
0: count = 194711488
1: count = 0
2: count = 0
3: count = 0
4: count = 0
5: count = 0
6: count = 0
7: count = 0
8: count = 0
9: count = 0
10: count = 0
11: count = 0
12: count = 0
13: count = 0
14: count = 0
15: count = 0
16: count = 0
17: count = 0
18: count = 0
19: count = 0
20: count = 0
21: count = 0
22: count = 0
23: count = 0
24: count = 0
25: count = 0
26: count = 0
27: count = 0
28: count = 0
29: count = 0
30: count = 0
31: count = 0
32: count = 0
33: count = 0
34: count = 0
35: count = 0
36: count = 0
37: count = 0
38: count = 0
So, we remounted with the inode64 option, expecting that it would start using the other AGs and improve performance.
Performance did seem to improve a bit but was still unreliable, it seemed to vary. And further, rerunning the above command, I still see count = 0 for all AGs except the first one.
In further weirdness, it looks like AGs beyond the first only have 5 blocks.
# xfs_db -c "freesp -a 0" -c "print" -r /dev/md1
from to extents blocks pct
1 1 58586 58586 0.02
2 3 40188 95811 0.04
4 7 119478 504965 0.20
8 15 51112 490250 0.19
16 31 22071 456215 0.18
32 63 14025 634223 0.25
64 127 474758 54537529 21.43
128 255 28449 6001372 2.36
256 511 13115 5007894 1.97
512 1023 4764 3441878 1.35
1024 2047 1952 2783230 1.09
2048 4095 883 2536350 1.00
4096 8191 477 2742390 1.08
8192 16383 208 2330731 0.92
16384 32767 121 2719892 1.07
32768 65535 37 1518146 0.60
65536 131071 24 2023965 0.80
131072 262143 9 1595257 0.63
262144 524287 1 349393 0.14
1048576 2097151 1 1345705 0.53
134217728 268435328 1 163286552 64.17
no current type
# xfs_db -c "freesp -a 1" -c "print" -r /dev/md1
from to extents blocks pct
1 1 5 5 100.00
no current type
Having almost no blocks would explain why nothing is getting allocated there, but why would these have so few blocks? Or am I misreading this data? (I'm not an XFS expert)
Any insight would be appreciated.
