0

I am working on node.js express.js app which is used for user data collection (in get request) and sends back only 200 ok response.

After getting data in request I am checking for existing entry in database dynamodb and redshift and for newer request I am storing data in dynamodb.

In this process I required to to do some synchronous operations for that I am using async's waterfall method. Also used callback in few places.

I have used pm2 module on server for clustering daemonizing and logging. I have used console.log() for logging steps.

This application perfectly working on my local system and also on server with no traffic.

All logs are working correctly also proper entires going in dynamodb

But this same app is not working if I increase traffic to large numbers. Logs also not working after increasing traffic.

I have no clue how to debug and find exact road block.

I want help for debugging issue and for and tips while working on large traffic.

P.S. :- I have tried increasing `http.globalAgent.maxSockets = large_number;

CPU and Memory utilisation is not high (~40% CPU and 60% Memory)

Local Environment 

node.js = 0.12.0
express.js = 4.x
ubuntu 14.04.x

Server Environment (AWS ec2)

node.js = 0.12.0
express.js = 4.x
Red Hat 6.6.x
Load balancer = Amazon Elastic Load Balancer
Sandip Pingle
  • 647
  • 2
  • 14
  • 37
  • 1
    You have to be more specific ("app is not working" is too broad). How is the app not working as your traffic increases? (are clients getting errors? threads not responding? logging not working? etc.) Describe the symptoms that you're experiencing. – unkhan Feb 12 '15 at 22:38
  • ^^ you would need to let others view this code and be specific about what "isn't working". – sctskw Feb 13 '15 at 02:52
  • logging is not working, clients are getting 200 ok response but data is not getting stored. – Sandip Pingle Feb 13 '15 at 04:54
  • If thats the case then it sounds like you are sending a response to the clients before the operations finish - you may be activating callbacks before the processes are done. – Kelz Feb 13 '15 at 07:56
  • @Kelz how can find out which callback is failing, I intentionally sending response back to user immateriality. why exactly same code is working with low traffic, is there any care needs for large traffic? – Sandip Pingle Feb 13 '15 at 09:15
  • Check any request handling function that is sending 200 OK and not storing the data - make certain that callback is not called until all the processes of that function are done (including data being stored) - make sure you are handling any errors from your dynamodb calls. – Kelz Feb 16 '15 at 04:53
  • After trial and error, I found out `redshift` is culprit, as soon as start querying in `redshift` program gets slowed down and stop working. Do you have any suggestion for querying to `redshift` for large traffic, or should I post new question? – Sandip Pingle Feb 16 '15 at 05:00

0 Answers0