IAP access token with Gmail API

Question

I have the following Error

google.auth.exceptions.RefreshError: The credentials do not contain the necessary fields need to refresh the access token. You must specify refresh_token, token_uri, client_id, and client_secret.

My project is built with Angular front side and flask. My web app is protected with IAP with Gmail.readonly scope ( I need to list all labels for the user).

So I need to get the access_token from IAP oauth2 and pass it to the back for call Gmail API Angular side I try to get my access_token with this

Init gapi

 async initGoogleAuth(): Promise<void> {
    const payload = new Promise((resolve) => {
      gapi.load('auth2', resolve);
    });
    return payload.then(async () => {
      await gapi.auth2
        .init({
          client_id: this.clientId,
          fetch_basic_profile: true,
          prompt: 'select_account'
        })
        .then(auth => {
          this.gapiSetup = true;
          this.authInstance = auth;
        });
    }).catch(e => {
      console.log('Error initiating Google auth');
      console.log(e);
      return;
    });
  }

authenticate if gapi is init

async authenticate(): Promise<any> {
    if (!this.gapiSetup) {
      await this.initGoogleAuth();
    }

    const signInInstance = this.authInstance.signIn();
    await signInInstance.then(result => {
      const googleUser = this.authInstance.currentUser.get();
      this.isSignedIn = googleUser.isSignedIn();
      const profile = this.authInstance.currentUser.get().getBasicProfile()
      localStorage.setItem('email', profile.getEmail());
      localStorage.setItem('userName', profile.getName());
      localStorage.setItem('imageUrl', profile.getImageUrl());
      localStorage.setItem('token', this.authInstance.currentUser.get().getAuthResponse().id_token);
      localStorage.setItem('accessToken', this.authInstance.currentUser.get().getAuthResponse().access_token);
      console.log(this.authInstance.currentUser.get().getAuthResponse(true))
    }).catch(e => {
      console.log(e);
    });
  }

From my component, I pass through header access_toke

 sendRequest() {
    const headers = {
      headers: {
        "x-dvtm-access-token": localStorage.getItem('accessToken'),
        "x-Goog-Authenticated-User-Email": localStorage.getItem('email')
      }
    };
    // add "/check-preserve" for prod
    this.httpClient.get(environment.apiUrl + "/check-preserve", headers).subscribe(res => {
      if (typeof res === "string") {
        this.processResult(JSON.parse(res))
      }
      console.log(typeof (res));
      this.loading = false
    }, err => {
      this.loading = false
      console.error(err);
    });
  }

Flask side I get the token here and build credentials :

access_token = request.headers.get('x-dvtm-access-token')
user = request.headers.get('X-Goog-Authenticated-User-Email').replace("accounts.google.com:", "")

if not access_token or not user:
    return "Not authorized", 401



creds = google.oauth2.credentials.Credentials(access_token)
logging.info(creds.__dict__)
response = check_preserve_label(creds, user=user)

What I am missing? thank you

            EDIT 

SO I change my flask to send HTTP request via (requests module) and add header Authorization : . But I have the feeling that IAP oauth2 do not provide scope access ( even if is set on configuration)

'error': {'code': 403, 'message': 'Request had insufficient authentication scopes.', 'errors': [{'message': 'Insufficient Permission', 'domain': 'global', 'reason': 'insufficientPermissions'}], 'status': 'PERMISSION_DENIED'}

SO I might need a double authentication Once for IAP ( domain access) and other for scope access ( Gmail for me ) source Issue traker