1

I have to generate large csv files that export data from database with some minimal processing of the raw data.

The file is downloaded to the user, writing chunks with putcsv. My problem is that the download can stop before is done and the user won't notice that the csv if truncated.

Question: is there a way to add integrity hints to cvsfile so the user can check if the file contains all the data was supposed to contain?

One solution would be to add at the last line count of all records and the client should check that, but is seems more like a hack

user237329
  • 809
  • 1
  • 10
  • 27
  • Maybe print an error when an error occurred: `if(!fputcsv($fp, $row)) die('An error occured when downloading csv.');` – reinierkors Jun 06 '17 at 21:43
  • That would add a new line to the csv file. A csv client might ignore that line. I was thinking it might exist already a mechanism like that – user237329 Jun 06 '17 at 21:44

1 Answers1

2

The CSV specification doesn't leave room for any sort of file integrity check. So if you are hoping to so something that a CSV parser will pick up on automatically, you can't. Part of the issue is that there isn't really a CSV specification, and in my experience different parsers have slightly different definitions of what CSV means. Either way, there is no integrity check built into the CSV "specification" that you can expect any CSV parser to automatically notice. Adding something to the end would give the end user a chance to verify the file themselves, but then again I doubt anyone would actually bother checking.

One possible option would be to send the content length along with the file. Then the browser may detect an interrupted download when the total content doesn't match what it was told to expect. The exact behavior there probably varies from browser to browser though, and you would have to run some tests to see if that is actually helpful.

However, that also requires you to know the length of the file before hand, which means building it all in memory, measuring the size, and then dumping to the browser. That has some disadvantages itself, including the fact that you are limited by the size of your memory (although you can mitigate that with php://temp). Also, the download won't start until the file is fully built, and depending on how long that takes it might result in more cancelations if the user thinks the download is frozen.

A completely different solution would be to write the file contents to disk (in the background) and then email the user with a download link when the file is built. This way they aren't waiting on your download script, and the actual download will simply be a simple file download managed directly by your web server, which will be much faster and less prone to errors.

Conor Mancone
  • 1,940
  • 16
  • 22