Get request is working in testing mode but fails in production - Google App Engine

Question

I ve created my own API using cloud shell with editor of Google app engine. Here is the snippet of the code which is working fine in testing:

app.get('/fetch_ticker', (req, res) => {
    (async() => {
        let pair = req.param('pair', "BTC/ETH");
        let ex = req.param('exchange', "coinmarketcap");
        let myArr = [];
        let exchange = await new ccxt[ex]();
        let tickers = await exchange.fetchTicker(pair);

        myArr.push(tickers);

        //Send req
        res.status(200).send(myArr);
    })()
});

Now, when I try it after 'Gcloud app deploy' and run it in production, other get requests are working fine but when 'ex' is equal to 'coinmarketcap', it just constantly loading and in the end gives 500 error.

Update:

Here is the log:

2018-11-03 11:43:50 default[20181103t163752]  ==== JS stack trace =========================================
2018-11-03 11:43:50 default[20181103t163752]
2018-11-03 11:43:50 default[20181103t163752]  Security context: 0x3e6826325879 <JSObject>
2018-11-03 11:43:50 default[20181103t163752]      1: indexBy(aka indexBy) [/srv/node_modules/ccxt/js/base/functions/generic.js:~82] [pc=0x30a50d2a5374](this=0x32f
9351022d1 <undefined>,/* anonymous */=0x1e7ccf12dc59 <JSArray[37746]>,/* anonymous */=0x32f935144e51 <String[2]: id>,/* anonymous */=0x32f9351022d1 <undefined>)
2018-11-03 11:43:50 default[20181103t163752]      2: arguments adaptor frame: 2->3
2018-11-03 11:43:50 default[20181103t163752]      3: set_markets(aka setMarkets) [/srv/node_modules/ccxt/js/base/Exchange.js:613] ...
2018-11-03 11:43:50 default[20181103t163752]
2018-11-03 11:43:50 default[20181103t163752]  FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
2018-11-03 11:43:50 default[20181103t163752]   1: node::Abort() [node]

I've tried to increase the memory using node --max_old_space_size=4096 app.js but the error still remains.

In your Developer Console, go to Logging and filter by your App Engine service and version. Can you find the request that returns a 500? What is the error? — LundinCast, Oct 28 '18 at 11:13
It gives me some info in the logging: This request caused a new process to be started for your application, and thus caused your application code to be loaded for the first time. This request may thus take longer and use more CPU than a typical request for your application. — Abrar A., Oct 28 '18 at 11:31
You should have some more information in the payload of the failing request log. You may paste in your question the entire request log, after removing any private/sensible information that may appear. — LundinCast, Oct 28 '18 at 11:37

Dan Cornilescu · Accepted Answer · 2018-11-03T13:21:06.380

The log you included is just the generic request log. I'm unsure if the nodejs environment provides app logs/details as well and, if so, how do you get to them. At least in the python standard environment such logs (accessible as illustrated in Reading Application Logs on Google App Engine from Developer Console) would typically contain an error message or a traceback very useful to identify the actual problem. Without such details it's rather guesswork, which is what I'm gonna do below :)

It could be that the coinmarketcap exchange doesn't actually support fetchTicker, while your code assumes all exchanges do support it. An example from the ccxt Manual actually has a check for that:

// JavaScript
if (exchange.has['fetchTicker']) {
    console.log (await (exchange.fetchTicker ('BTC/USD'))) // ticker for BTC/USD

Or the coinmarketcap exchange server has some (temporary) outage of some sort. Or no longer serves the data in the same conditions or with the same format as expected by the ccxt library, in which case I'd replace it as the default exchange to use for one which behaves OK.

Update:

OK, the error log indicates a memory allocation problem. The typical reason is insufficient app instance memory. You can address this via your app.yaml configuration, depending on the GAE environment you use:

in standard environment via instance_class in the Runtime and app elements section
in the flexible environment via memory_gb in the Resource settings section

I've added the actual log. The thing is the get request is working fine in the test env which means that the end point is working correctly (I've also checked the doc). Still, the error occurs. — Abrar A., Nov 03 '18 at 11:55
I just added instance_class: F2 and it started to fetch. Thanks so much. — Abrar A., Nov 04 '18 at 17:26

Get request is working in testing mode but fails in production - Google App Engine

1 Answers1