0

I tried installing the Export Collections to BigQuery extension to send firestore collection data to BigQuery. The installation succeeded but could never get the queries to work. I kept getting the error:

Not found: Dataset myproject:firestore_events was not found in location US

I tried uninstalling and reinstalling. I now have two instances of the extension and attempts to uninstall fail with ther error:

failed to remove IAM roles from Service Account for projects/myproject/instances/firestore-bigquery-export

Is there a way to uninstall manually? Is there a way to install manually?

Doug Stevenson
  • 297,357
  • 32
  • 422
  • 441
Duane
  • 273
  • 1
  • 11

2 Answers2

0

The following error message means that your FROM statement may not contain the proper query structure - project, database, and table:

Not found: Dataset myproject:firestore_events was not found in location US

For example:

SELECT *
FROM `my-project.my-database.my-table`

Please note the backticks.

Refer to Error: Not found: Dataset my-project-name:domain_public was not found in location US for details about a similar issue.

Now, in regards to the second error message:

failed to remove IAM roles from Service Account for projects/myproject/instances/firestore-bigquery-export

I suspect that after reinstalling and uninstalling one of the extension instances, its service account might already be deleted and thus failing to remove its IAM roles.

The installation of this extension generates a new service account that has the BigQuery Data Editor role to access your project and resources. This role allows the extension to configure and export data into BigQuery.

From documentation:

When you uninstall an extension from your project, Firebase deletes the service account created for that instance of the extension. After this deletion of the service account, the extension cannot run in your project because it no longer has any access rights to your project or data.

I was able to uninstall and reinstall the Export Collections to BigQuery extension to avoid duplicates in my project without any issues.

To answer your last question, I am not aware of any other methods to install/uninstall Firebase extensions other than what is shown in the documentation. You can install the extension using the Firebase Console or the Firebase CLI:

1 - Install the Firebase CLI or update it to the latest version:

npm install -g firebase-tools

2 - Install the extension:

firebase ext:install firestore-bigquery-export --project=projectId_or_alias

Edited

We can take the following steps to troubleshoot error:

failed to remove IAM roles from Service Account for projects

  • Please make sure that you're the owner of the project that you are installing extensions in.
  • Refer to this GitHub issue to uninstall the extension using the CLI.
  • If that didn't help, try disabling and re-enabling the Firebase Extensions API inside the GCP Console.
  • Please make sure that Firebase Management API is enabled in your project.
  • If that didn't help, navigate to the IAM Permissions page on the Cloud Console, and find the service account.
    • Add the Cloud Functions Developer and Firebase Extensions API Service Agent roles.
      • If those roles not available for some reason, just adding the Project Editor role also works as a catch-all. However, using the catch-all also provides additional permissions to the service accounts.
sllopis
  • 2,292
  • 1
  • 8
  • 13
  • Thank you for your response. I still haven't been able to remove the two instances (now three!) instances of the extension. Everytime I try to add the extension, I get an error. When I retry, it succeeds but doesn't work. I've tried remove the service account manually, but still no luck. The error I get on initial install is: failed to update IAM roles for Service Account for projects/myproject/instances/firestore-bigquery-export-a5hr-2. Not sure what to do. – Duane Oct 13 '20 at 21:33
  • I'm working on reproducing the error messages that you're seeing but I am still unable to. I edited and deleted my extension's service account before completely uninstalling it, but no errors are shown. Did you modify, add to, or delete any of these service account assigned roles (otherwise your installed extension won't work as expected)? Please note that each installed instance of an extension has its own service account. – sllopis Oct 14 '20 at 07:00
  • What role does the Google account that you are logged in as have? To install or manage extensions, you must be assigned one of these roles: Owner or Editor or Firebase Admin, as per the [documentation](https://firebase.google.com/docs/extensions/overview-use-extensions#console-vs-cli). Please bear in mind that service accounts created for extensions are in the format: _ext-extension-instance-id@project-id.iam.gserviceaccount.com_. Let me know if this SA gets created during installation and removed upon uninstallation. – sllopis Oct 14 '20 at 07:01
  • My user role is Owner. For the third instance of the extension that I have not attempted to remove, there is a service account in that format. The description is "BigQuery Integration." Of course, it still has the installation error of "failed to update IAM roles for Service Account for project" message. – Duane Oct 14 '20 at 13:27
  • Are there other configuration settings I can check? – Duane Oct 14 '20 at 13:28
  • I see, like I said earlier I suspect that service account doesn't have the permissions needed to communicate with the Deployment Manager service, which Firebase Extensions uses to deploy functions for the user. I will edit my answer, since it can get lengthy, to show the next steps that we can take. – sllopis Oct 15 '20 at 06:36
0

I had a similar issue when installing a Firebase Extension.

My error was adding an IAM role that did not exist, this not only failed to install the extension, but also then stopped me from uninstalling with the error failed to remove IAM roles from Service Account for.

To fix this issue I edited what I knew was the broken permission to be the correct role and then ran the following...

firebase ext:update ${ext-name} . --project={project-name}.

Alternativley, removing all roles completetly from the configuration and updating will have the same result.

One completed, the uninstallation process will automatically continue and uninstall your extension.

Darren Ackers
  • 148
  • 1
  • 6