0

I have a scenario. In DB, I have a table with a huge amount of records (2 million) and I need to export them to xlsx or csv.

So the basic approach that I used, is running a query against DB and put the data into an appropriate file to download.

Problems:

  1. There is a DB timeout that I have set to 150 sec which sometimes isn't enough and I am not sure if expanding timeout would be a good idea!
  2. There is also some certain timeout with express request, So it basically timed out my HTTP req and hits for second time (for unknown reason)

So as a solution, I am thinking of using stream DB connection and with that if in any way I can provide an output stream with the file, It should work.

So basically I need help with the 2nd part, In stream, I would receive records one by one and at the same time, I am thinking of allowing user download the file progressively. (this would avoid request timeout)

I don't think it's unique problem but didn't find any appropriate pieces to put together. Thanks in advance!

Community
  • 1
  • 1
Gautam Kumar Samal
  • 758
  • 1
  • 7
  • 23

1 Answers1

0

If you see it in your log, do you run the query more than once? Does your UI timeout before the server even reach the res.end()?

Fahmi R
  • 1
  • 3