0

I have a MySQL query that returns ~11,000 results that then need further queries run on each result. While the code is running it doesn't allow other users to log in to the website.

Potential problems I've seen are the use of callback functions or foreach loops but that doesn't seen to help.

code looks like this:

query(sql, params, function(err, result) {
   result.foreach(
       query(generate_sql(result), params, function(err, result) {
               //do stuff
       });
   )
});
Gabe Levasseur
  • 71
  • 1
  • 5
  • Why you use the same NodeJS App to manage authentification and doing mysql big requests? You can scale your application to 2 minimum nodes. the first one will deal with authentifications and the second one: DB stuffs – Mohamed Amjad LASRI Dec 10 '15 at 16:25
  • If your first query returns 11k results then the foreach loop fires off 11k queries simultaneously, which is definitely not what you'd want. I would suggest looking into something like the [async](https://github.com/caolan/async) function [eachSeries](https://github.com/caolan/async#each) or possibly eachLimit. Alternatively you could use promises, but I'm not very familiar with those so I don't know if they have the same possibilities as eachSeries/eachLimit. – Wouter Dec 10 '15 at 16:27

2 Answers2

0

I recommend you using Promises

Then your code would be something like that:

var Promise = require('bluebird');

_query = Promise.promisify(query);

_query(sql, params)
    .map(function(singleRow) {
        // you get here every single row of your query
    })
    .then(function() {
        // you are finished
    });
Safari
  • 3,302
  • 9
  • 45
  • 64
0

Your problem is having that much queries. 11,001 query is a very large number, and I don't think that MySQL can handle that number of queries in parallel. So another way is to use subqueries to handle your problem.

MIE
  • 444
  • 2
  • 9