2

We use mongoDB as DBMS for our production database. We have enabled 2 replica sets for our DB.

Out of blue, we noticed that both the databases in our production have been dropped. We are absolutely sure that no one with access did it. When we went through the logs, we noticed that dropDatabase on both the databases had indeed been called. However, we couldn't find any log statements for the connection establishment. If someone has used mongo shell or bash to delete the DB, there should have been logs for new connections as well.

Log statement for Primary:

2017-08-17T14:38:47.019+0000 I -        [conn527] AssertionException handling request, closing client connection: 17132 SSL handshake received but server is started without SSL support
2017-08-22T00:18:22.694+0000 I COMMAND  [conn541] dropDatabase db1 starting
2017-08-22T00:18:22.708+0000 I COMMAND  [conn541] dropDatabase db1 finished
2017-08-22T00:18:23.832+0000 I COMMAND  [conn542] dropDatabase db2 starting
2017-08-22T00:18:23.843+0000 I COMMAND  [conn542] dropDatabase db2 finished

As it is quite clear from the timestamp of the first line, there was no new connection established with the mongo server in last 5 days

Can someone please help us understand as to what exactly happened to our mongodb server. Also, any pointers will be greatly appreciated.

thrust
  • 225
  • 1
  • 8
  • To find out "who" was connection 541, make search thru mongodb logs (if you have enough log files) `grep -R "\#541" *.log` – JJussi Aug 23 '17 at 07:20
  • We have logs for last 3 months and couldn't find any connection 541 or 542. Can you please give us any other pointer? – thrust Aug 23 '17 at 07:44
  • That's very strange because all "connection accepted from x.x.x.x" events are logged normally. And your older files what you grep were not 'zip' (just checking). Then 'some one' has removed those line. – JJussi Aug 23 '17 at 13:03
  • It indeed is strange. I unzipped all the files before running grep. What baffles me is that the connection id for the two drop queries is different. If someone got unauthorized access to the instance, it is very much possible that he/she deleted the log statements. However, I don't think someone will keep the connection open for a week or more just to run dropDatabase on the db later. So the connection logs should have been there in the same file. – thrust Aug 23 '17 at 13:36

1 Answers1

0

If your instance is still running you can issue a

db.adminCommand( { getLog: "global" } )

in order to find out what has happend. The getLog command reads the MongoDB log from the RAM and not from the log file - just in case someone has tampered with the log file even...

dnickless
  • 10,733
  • 1
  • 19
  • 34
  • Thanks a lot for your suggestion. However, we had restarted the server earlier. So, I am afraid this won't help. Do you have any other pointer in mind? – thrust Aug 23 '17 at 07:47