2

I am bypassing the ORM and using the Model.query function to query and return a "large" result set from PostgreSQL. The query returns around 2 million rows. When running the query directly from postgres it returns in around 20s. The query fails silently when executed from sails. Is there a limit on the number of rows that can be returned?

Travis Webb
  • 14,688
  • 7
  • 55
  • 109
Matt E
  • 457
  • 1
  • 11
  • 24

1 Answers1

0

Is there a limit on the number of rows that can be returned?

No there is no limit.

The query fails silently when executed from sails

What does "fails silently" mean? How do you know that it's failed? It may still be processing; or the adapter might have a connection timeout that you're breaching.

Two million rows serialized out of the database, translated to JSON, and piped down to the client is much different than just running SQL directly on the database. It could take 20x longer, depending on your system resource situation. I strongly recommend using sails.js's paging features to pull the data out in chunks. Pulling two million rows in one operation from a web server doesn't make a lot of sense.

Travis Webb
  • 14,688
  • 7
  • 55
  • 109
  • Fails silently as in I do not see a time out or any other error in the console. – Matt E Apr 16 '15 at 12:27
  • Thanks I know what fails silently *means*. How long did you wait? 2 million rows is a lot of rows. You're missing the point, and probably need to re-read my answer. – Travis Webb Apr 16 '15 at 14:59