Multiple databases and collections mongodump

Question

I have something like this:

dbs=$(mongo --quiet --eval "db.getMongo().getDBNames()" --host exemple.com | \ 
            grep '"' | tr -d '"' | tr -d ',')

for db in $dbs; do
    cols=$(mongo --quiet --eval "print(db.getCollectionNames())" $db \
                  --host exemple.com | tr ',' ' ')
    for col in $cols; do
        mongodump --host example.com -q "{_id:{\$gt:$oid}}" \
                   -d $dbs -c $col --out /data/
   done
done

I'm getting:

positional arguments not allowed

How can I use mongodump for all collections in all databases ?

Double quote all your shell variables, `$dbs`, `$db`, `$cols`, `$col`. It may not solve the problem, but useful in bye-passing trivial issues — Inian, Apr 21 '17 at 13:11
`grep '"' | tr -d '"' | tr -d ','` can be simplified to: `tr -d '",'`. — agc, Apr 21 '17 at 16:28
`grep` and `tr` can be replace by `jq -r '.[]'` if you're into that sort of thing. — Stephan Henningsen, Feb 08 '21 at 22:21

score 8 · Accepted Answer · answered Apr 21 '17 at 16:38

8

Here is a working script:

  dbs=`mongo --eval "db.getMongo().getDBNames()" | grep '"' | tr -d '",' `


    for db in $dbs; do
        col=`mongo  $db --host example.com --quiet --eval "db.getCollectionNames()" | tr -d ',"[]' `
        for collection in $col; do
          mongodump --host example.com -q '{_id: {$gt: 10}}' -d $db -c $collection --out dump

       done
    done

from mongodump documentation :

--query , -q

Provides a JSON document as a query that optionally limits the documents included in the output of mongodump. You must enclose the query in single quotes (e.g. ') to ensure that it does not interact with your shell environment.

answered Apr 21 '17 at 16:38

felix

9,007
7
41
62

for local mongodb is working, but for remote I get the local db's and collections also dbs=`mongo --eval "db.getMongo().getDBNames()" | grep '"' | tr -d '",' ` echo $dbs for db in $dbs; do col=`mongo $db --host example.com --quiet --eval "db.getCollectionNames()" | tr -d ',"[]' ` for collection in $col; do mongodump --host example.com -q '{_id: {$gt: 10}}' -d $db -c $collection --out dump done done – basante Apr 25 '17 at 12:31
@basante make sure to update the host name in the 2 `mongo` calls and in the mongodump, and to upadte the query according to your needs – felix Apr 25 '17 at 12:35
yes of course I change the hostname, but still I'm getting the local dbs. Can you please show me an example, maybe I'm not setting up the --host correctly – basante Apr 25 '17 at 14:34

score 2 · Answer 2 · answered Oct 15 '19 at 07:48

2

For multiple database.

dbs=( z2p stackoverflow poststodos)  
for c in ${dbs[@]}
do
mongodump -d  $c  --out ~/backups/dump
done

answered Oct 15 '19 at 07:48

Ankit Kumar Rajpoot

5,188
2
38
32

Very clever! Just note to add "--authenticationDatabase admin " for some cases. – Mehmet Burak Sayıcı Oct 24 '22 at 00:15

score 0 · Answer 3 · answered Nov 05 '20 at 05:41

you may give this script a try, which provides a function called dump_by_filter

#!/bin/bash

# sudo apt install mongo-tools
# sudo apt install mongo-client

# dump(uri:string,col:string,out:string)
function dump {
    local uri=$1
    local col=$2
    local out=$3

    mongodump --forceTableScan  --uri=$uri  -c $col --out $out
}

# load(uri:string,dir:string)
function load {
    local uri=$1
    local dir=$2
    mongorestore --drop --uri=$uri $dir
}

# list_col(uri:string,filter:string)
function list_col {
   local res=`echo show collections |mongo $1 --quiet |grep $2` 
   echo $res
}

# dump(uri:string,filter:string,out:string)
function dump_by_filter {
    local uri=$1
    local filter=$2
    local out=$3

    local cols=$(list_col $uri $filter)
    for col in $cols; do
        dump $uri $col $out
        echo dump $col to $out
    done
}

FROM_URL="mongodb://127.0.0.1:27017/a"
TO_URL="mongodb://127.0.0.1:27017/b"

DIR="./out"
dump_by_filter $FROM_URL xx $DIR
load $TO_URL $DIR

i place it here

Multiple databases and collections mongodump

3 Answers3