we are running a production Database cluster consisting of the following:
Three AWS linux servers, each of which contains one Coordinator, one Agent, and one Primary Database. All three servers are located in a private network on AWS. The databases are ran as docker images (ArangoDB v3.1.24).
The cluster is consistent and is running smoothly. However, every couple of days, one or more coordinators generate the following warning, for a few hours, and then everything comes back to normal:
2017-09-20T14:07:33Z [1] WARNING cannot contact coordinator <COORDINATOR_ID> on endpoint <endpoint>
2017-09-20T14:01:40Z [1] ERROR Service "/_admin/aardvark" encountered error 500 while handling GET http://<host>:<port>/_db/_system/_admin/aardvark/statistics/coordshort
2017-09-20T14:01:40Z [1] ERROR TypeError: Cannot read property '0' of undefined
2017-09-20T14:01:40Z [1] ERROR at /usr/share/arangodb3/js/apps/system/_admin/aardvark/APP/statistics.js:480:64
2017-09-20T14:01:40Z [1] ERROR at arrayEach (/usr/share/arangodb3/js/node/node_modules/lodash/lodash.js:530:11)
2017-09-20T14:01:40Z [1] ERROR at Function.forEach (/usr/share/arangodb3/js/node/node_modules/lodash/lodash.js:9319:14)
2017-09-20T14:01:40Z [1] ERROR at /usr/share/arangodb3/js/apps/system/_admin/aardvark/APP/statistics.js:479:15
2017-09-20T14:01:40Z [1] ERROR at arrayEach (/usr/share/arangodb3/js/node/node_modules/lodash/lodash.js:530:11)
2017-09-20T14:01:40Z [1] ERROR at Function.forEach (/usr/share/arangodb3/js/node/node_modules/lodash/lodash.js:9319:14)
2017-09-20T14:01:40Z [1] ERROR at /usr/share/arangodb3/js/apps/system/_admin/aardvark/APP/statistics.js:477:13
2017-09-20T14:01:40Z [1] ERROR at arrayEach (/usr/share/arangodb3/js/node/node_modules/lodash/lodash.js:530:11)
2017-09-20T14:01:40Z [1] ERROR at Function.forEach (/usr/share/arangodb3/js/node/node_modules/lodash/lodash.js:9319:14)
2017-09-20T14:01:40Z [1] ERROR at mergeHistory (/usr/share/arangodb3/js/apps/system/_admin/aardvark/APP/statistics.js:440:7)
I can't really figure out why this happens, and why it stops happening on its own.
PS: When it happens, I can't access the Arango Dashboard, however the database, remains working fine.
