0

I am in pursuit to create a CSV file, with size constraints defined by Bytes.

For e.g. i need to keep writing to a file, that would contain CSV data, so it doesn't exceed file limit of 5MB.

Without any loss of data in the row.

Obvious solution would be to keep writing the file, and checking its size on every row insert, which could be tiring and lead to lot of disk IO.

Looking for something better to implement with less overhead and that could provide better performance.

s_s_
  • 463
  • 5
  • 20
  • 1
    If you have any existing code you can share, that will help us to provide an answer in the context of your code. – defines Aug 23 '20 at 00:20
  • Well as mentioned in the ticket, the one i have uses a lot of file reads & writes, and i am looking for alternatives to this approach which is more of a brute-force kind of approach. – s_s_ Aug 23 '20 at 00:25
  • Share your code anyways, so we can tell you where you can do better. – SirPilan Aug 23 '20 at 00:35
  • 2
    Count the bytes while writing? fputcsv() returns the length of the written string, just add it up and check continously... – Honk der Hase Aug 23 '20 at 00:51
  • You could also check the expected bytes on disk before writing. Given the target encoding, you can calculate the size of each character to be written beforehand. This way you can close the current file and open the next before the write, instead of handling this as an error case after write. – defines Aug 23 '20 at 00:55
  • @LarsStegelitz appreciate the help, and this is sort of what i am writing but this gives me the bytes size of the string only after the line is written. However i am looking for something that would help me prior to writing the line, so that i don't have to go back and remove the last row. – s_s_ Aug 23 '20 at 00:56
  • Then define the treshold a little lower than 5 MB, or implode + strlen before writing. Or make an assumtion (the next line will be approx. same length as the last line)... – Honk der Hase Aug 23 '20 at 09:43
  • Understood, but was reluctant if strlen would provide me corrdect byte value for the string. – s_s_ Aug 23 '20 at 22:37
  • Are there any other manner to manage data other than writing to a file? – s_s_ Aug 23 '20 at 22:38

0 Answers0