PHP Google_Service_Dataflow not finding file in bucket

Question

I've come across an issue using the googleapis/google-api-php-client library, specifically the Dataflow Service that I cannot solve.

When I try to use the library I set up the request like so:

$this->client = new \Google_Client();
$this->client->setAuthConfig(config_path('google-service-account.json'));
$this->client->setIncludeGrantedScopes(true);
$this->client->addScope(\Google_Service_Dataflow::CLOUD_PLATFORM);

$body = [
    "gcsPath" => "gs://{$this->bucket}/{$this->template}",
    "location" => "us-central1",
];

$parameters = new \Google_Service_Dataflow_LaunchTemplateParameters;
$parameters->setJobName($this->jobname);
$parameters->setParameters($body);

$service = new \Google_Service_Dataflow($this->client);
$request = $service->projects_templates->launch($this->project, $parameters);

And I get the following error:

{
  "error": {
    "code": 400,
    "message": "(11f8b78933fc59c3): Bad file name: , expected 
    'gs://\u003cbucket\u003e/\u003cpath\u003e'",
    "errors": [
      {
        "message": "(11f8b78933fc59c3): Bad file name: , expected 
        'gs://\u003cbucket\u003e/\u003cpath\u003e'",
        "domain": "global",
        "reason": "badRequest"
      }
    ],
    "status": "INVALID_ARGUMENT"
  }
}

It seems that the path is getting corrupted along the way, I've checked and it gets fine until the Guzzle object is instantiated to send the request inside the library.

I'm pretty lost at this point so any suggestion or clue is welcome.

Thank you in advance.

What is the string contained in `$this->bucket`. `\u003c` is `<` and `\u003e` is `>`. Do you have a url encoding issue or a parameter passing issue from other code? — John Hanley, Jan 14 '19 at 19:27

score 1 · Accepted Answer · answered Jan 14 '19 at 14:30

No gcsPath is given in the query params for the request constructed by the SDK.

This is because gcsPath is set to be an option for Google_Service_Dataflow_LaunchTemplateParameters.

It is documented that request query parameters be given as optional params (See https://github.com/googleapis/google-api-php-client-services/blob/v0.81/src/Google/Service/Dataflow/Resource/ProjectsTemplates.php#L73.)

$opt_params = [
    "gcsPath" => "gs://{$this->bucket}/{$this->template}",
    "location" => "us-central1",
];

$template_params = [
   // Keep template params here.
];

$launch_params = new \Google_Service_Dataflow_LaunchTemplateParameters;
$launch_params->setJobName($this->jobname);
$parameters->setParameters($template_params);

$service = new \Google_Service_Dataflow($this->client);
$request = $service->projects_templates->launch($this->project, $parameters, $opt_params);

Thank you very much, that actually solved the issue with the path, now the SDK finds the file but still cannot read it, I've already checked in case it was smth permissions related but no luck, do you have any idea? — Asur, Jan 14 '19 at 14:35
I have edited the question and I will repost the second part separately, I think is the best thing to do, thank you very much for your help. — Asur, Jan 14 '19 at 14:53

PHP Google_Service_Dataflow not finding file in bucket

1 Answers1