How to recursively list subcollections in Firestore

Question

We back up all our firestore collections daily, using an empty array [] in:

const client = new firestore.v1.FirestoreAdminClient();

return client
    .exportDocuments({
      name: databaseName,
      outputUriPrefix: storageName,
      collectionIds:[], // empty array backs up all collections and subcollections
    })

However, according to the docs, were we ever to want to import data, we would need to import ALL collections and subcollections, which is substantial. In order to import more granularly (import only necessary collections), we need to provide exportDocuments.collectionIds an array of all our collections' and subcollections' names. ie: ['Coll1', 'Coll2', 'SubCollX', etc.]. Since our collections are subject to change, we need a way to get an array of the names of all collections and subcollections programatically.

This code gets the names of only the root collections, not any subcollections:

const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();

exports.getCollections = functions.https.onRequest(async (req, res) => {
    var rootCollectionIds = [];
    var subCollectionIds = [];

    const rootCollections = await admin.firestore().listCollections();
    
    rootCollections.map(async rootColl => {
        rootCollectionIds.push(rootColl.id);
        
        // without named doc, this returns empty:
        const subCollections = await rootColl.doc().listCollections();
        subCollectionIds = subCollections.map(subColl => subColl.id);
    });

    res.status(200).send({"rootCollectionIds": rootCollectionIds, "subCollectionIds": subCollectionIds});
});

This must be doable. In the console (https://console.cloud.google.com/firestore/import-export?project={myproject}), Google lists all collections and subcollections when using Export one or more collection groups

Best I can tell, listCollections() onlys works at the root, or on a specific document. Since we do not know which specific documents contain subcollections, we need a way to find the subcollections globally. Is there some kind of allDescendents param that could be used?

Running node.js in a Cloud Function.

Any help greatly appreciated.

As unsolicited advice, I will say that Firestore data modeling generally does not work well with dynamic collection names. In the best case, the only things that should be dynamic are the IDs of documents. Collections and subcollections should generally be predictably named and finite in number. — Doug Stevenson, Aug 27 '22 at 22:27

score 1 · Accepted Answer · answered Aug 27 '22 at 22:27

1

No, no such option exists. The only way you can find out about subcollections is to build a DocumentReference and call listCollections on it to find which subcollections are available under that one document.

answered Aug 27 '22 at 22:27

Doug Stevenson

297,357
32
422
441

Bummer. I was afraid that was the case after hours of trials. Is there nothing at the Account level to determine the names of subcollections? How does the backup feature in the console find them? For now, I will have to copy them from the console. – GAEfan Aug 28 '22 at 00:55
An export just queries all documents and their subcollection's documents using nothing more complicated than the SDK's own API. The SDKs are all open source. – Doug Stevenson Aug 28 '22 at 02:10

How to recursively list subcollections in Firestore

1 Answers1