How to overcome Centos to QNAP strange FTP file size limit of 150114776 bytes?

Question

I'm sure there is a simple explanation but I cannot seem to figure it out. I have a Centos server which needs to do a daily FTP upload of a database to external back-up provided by a QNAP NAS. The server also puts a copy of the DB on to a second Centos server. The file is >800MB and growing.

I have a script which handles the FTP put of the file to the second server and this is called by crontab daily and works every time.

I have an almost identical script also called by crontab for the FTP to the QNAP and it always truncates the file at exactly 150114776 bytes. Strangely, if I run this same script from the CLI it always works perfectly delivering the entire file to the QNAP which suggests that there is no QNAP limit on filesize coming in to play.

The problem is consistent. Invoke the transfer with crontab and the file is truncated. Invoke with CLI and the whole file is transferred. No error is ever reported; FTP thinks it has done the whole job.

Sample log of transfer by crontab:

    Connected to 172.172.1.1 (172.172.1.1).
    220 NASFTPD Turbo station 1.3.4e Server (ProFTPD) [::ffff:172.172.1.1]
    Remote system type is UNIX.
    Using binary mode to transfer files.
    331 Password required for fred
    230 User fred logged in
    250 CWD command successful
    local: DATA_bk.sql.1.gz remote: DATA_bk_20150811_071501.sql.gz
    227 Entering Passive Mode (172,172,1.1,217,232).
    150 Opening BINARY mode data connection for DATA_bk_20150811_071501.sql.gz
    226 Transfer complete
    150114776 bytes sent in 23 secs (6.4e+03 Kbytes/sec)
    221 Goodbye.

And a manual invocation:

    Connected to 172.172.1.1 (172.172.1.1).
    220 NASFTPD Turbo station 1.3.4e Server (ProFTPD) [::ffff:172.172.1.1]
    Remote system type is UNIX.
    Using binary mode to transfer files.
    331 Password required for fred
    230 User fred logged in
    250 CWD command successful
    local: DATA_bk.sql.1.gz remote: DATA_bk_20150811_120117.sql.gz
    227 Entering Passive Mode (172,172,1.1,217,189).
    150 Opening BINARY mode data connection for DATA_bk_20150811_120117.sql.gz
    226 Transfer complete
    879067272 bytes sent in 182 secs (4.5e+03 Kbytes/sec)
    221 Goodbye.

Can anyone point me to some rule I've overlooked or suggest a way to debug this?

Thanks

score 2 · Accepted Answer · answered Dec 01 '15 at 07:15

It turns out I made a simple error. The cron tab was executing in the wrong directory where there happened to be an old copy of the source file which just happened to be 150114776 bytes big. Some times, the simplest causes are the toughest to see.

Data transfer to the QNAP now works perfectly every time.

score 0 · Answer 2 · edited Sep 17 '15 at 07:06

0

We have the same problem. We opened a ticket with QNAP and this is the reply:

This a known issue (bug) what you can do is downgrade to the former firmware version or wait to the next one is released where hopefully this issue is solved. Sorry for the inconvenience.

So.... downgrade or wait...

edited Sep 17 '15 at 07:06

Matt

74,352
26
153
180

answered Sep 10 '15 at 15:42

Dirk

1

Thank you Dirk/Matt. It's good to know that my experience is not 100% unique. I've just initiated a firmware update for a recent release (last few weeks) so fingers crossed. Will update you once this new firmware has run for a few days. – CJF123 Sep 18 '15 at 20:18
I'm at QTS 4.1.4 Build 0804 with similar problem, in my case Plesk fails to upload file above 1GB . Trying to upgrade to QTS 4.1.4 Build 0820. – brablc Sep 21 '15 at 07:54
Update: Since that firmware upgrade, and a subsequent one to v4.1.4 Build 20150910 every FTP transfer of this large file is truncated. I suppose the new consistency of results makes this a less puzzling issue and highlights that the issue is with the NAS not the server. I have re-opened my ticket with QNAP. – CJF123 Sep 24 '15 at 07:25

How to overcome Centos to QNAP strange FTP file size limit of 150114776 bytes?

2 Answers2