I manage a small network of windows clients and a BSD file server running Samba 4.6.x. We had some odd issues which led to the discovery that when users save files to the server, about 12% of the files as saved aren't faithful copies of the Windows original. (Tested by copying 2000 files of 1 - 5 MB and hashing Windows originals and BSD copies: about 245 differed).
I tested a bunch of things: copying from multiple clients (same happened on all clients), server hardware checks (ECC fine, ZFS no errors), network data corruption (no issues end to end), directionality (copying client to server corrupted about 12%, server to client was faithful), consistency (copy same folder 3 times one after the other in a session and compare: in each copy, the corrupted files differed; one copy had no corruption), long path issues (no long filenames, paths, or odd chars in filenames).
I also copied using SCP but got "server aborted connection" errors after a second or two, which might mean something or nothing, so I couldn't check if it was Samba-specific. SSH which I think SCP uses is rock solid so I'm not sure what to make of that. The NICs are good quality - Intel 1G + Chelsio 10G. Nobody else has logged in, the server is locked down and firewalled, and no system tweaking has gone on - it's pretty much FreeBSD 11 + Samba.
I've always assumed (naively?) that file server issues were almost always down to access issues (config, permissions and authentication) and provided users can actually write files then, barring hardware faults, it "just works". So this random "file saves on server but saved version not same as original" has really got me foxed.
Any suggestions what kind of issue could cause this, and how to troubleshoot it?
