Cloud function to export Firestore backup data. Using firebase-admin or @google-cloud/firestore?

Question

I'm currently trying to build a cloud function to export my Firestore data to my Storage Bucket.

The only example I've found on the Firebase DOCs on how to do this:

https://googleapis.dev/nodejs/firestore/latest/v1.FirestoreAdminClient.html#exportDocuments

EXAMPLE

const firestore = require('@google-cloud/firestore');

const client = new firestore.v1.FirestoreAdminClient({
  // optional auth parameters.
});

const formattedName = client.databasePath('[PROJECT]', '[DATABASE]');
client.exportDocuments({name: formattedName})
  .then(responses => {
    const response = responses[0];
    // doThingsWith(response)
  })
  .catch(err => {
    console.error(err);
  });

From that example, it seems that I need to install @google-cloud/firestore as a dependency to my cloud function.

But I was wondering if I can access these methods using only the firebase-admin package.

I've thought of that because the firebase-admin has the @google-cloud/firestore as a dependency already.

> firebase-admin > package.json

"dependencies": {
    "@firebase/database": "^0.4.7",
    "@google-cloud/firestore": "^2.0.0",    // <---------------------
    "@google-cloud/storage": "^3.0.2",
    "@types/node": "^8.0.53",
    "dicer": "^0.3.0",
    "jsonwebtoken": "8.1.0",
    "node-forge": "0.7.4"
  },

QUESTION:

Is it possible to get an instance of the FirestoreAdminClient and use the exportDocuments method using just the firebase-admin ?

Or do I really need to install the @google-cloud/firestore as a direct dependency and work with it directly?

Answer 1

The way you're accessing the admin client is correct as far as I can tell.

const client = new admin.firestore.v1.FirestoreAdminClient({});

However, you probably won't get any TypeScript/intellisense help beyond this point since the Firestore library does not actually define detailed typings for v1 RPCs. Notice how they are declared with any types: https://github.com/googleapis/nodejs-firestore/blob/425bf3d3f5ecab66fcecf5373e8dd03b73bb46ad/types/firestore.d.ts#L1354-L1364

Answer 2

Here is an implementation I'm using that allows you to do whatever operations you need to do, based on the template provided by firebase here https://firebase.google.com/docs/firestore/solutions/schedule-export

In my case I'm filtering out collections from firestore I don't want the scheduler to automatically backup

const { Firestore } = require('@google-cloud/firestore')

const firestore = new Firestore()
const client = new Firestore.v1.FirestoreAdminClient()
const bucket = 'gs://backups-user-data'

exports.scheduledFirestoreBackupUserData = async (event, context) => {
  const databaseName = client.databasePath(
    process.env.GCLOUD_PROJECT,
    '(default)'
  )

  const collectionsToExclude = ['_welcome', 'eventIds', 'analyticsData']

  const collectionsToBackup = await firestore.listCollections()
    .then(collectionRefs => {
      return collectionRefs
        .map(ref => ref.id)
        .filter(id => !collectionsToExclude.includes(id))
    })


  return client
    .exportDocuments({
      name: databaseName,
      outputUriPrefix: bucket,
      // Leave collectionIds empty to export all collections
      // or define a list of collection IDs:
      // collectionIds: ['users', 'posts']
      collectionIds: [...collectionsToBackup]
    })
    .then(responses => {
      const response = responses[0]
      console.log(`Operation Name: ${response['name']}`)
      return response
    })
    .catch(err => {
      console.error(err)
    })
}

Answer 3

Here is the full explanation with code (I use it and it works very well) on how to do automated Firestore backups by mixing Cloud Scheduler, PubSub and Firebase Function https://firebase.google.com/docs/firestore/solutions/schedule-export

Answer 4

firebase-admin just wraps the Cloud SDK and re-exports its symbols. You can use the wrapper, or use the Cloud SDK directly, or even a combination of the two if you want. If you want to use both, you have to declare an explicit dependency on @google-cloud/firestore in order to be able to import it directly into your code.