1

I am trying to connect Streamsets to Google Analytics. However I am having trouble setting it up. With a regular CURL request I would do the following:

Step 1) Go to the following link to get the authorization code:

https://accounts.google.com/o/oauth2/auth?client_id={CLIENTID}.apps.googleusercontent.com&redirect_uri=urn:ietf:wg:oauth:2.0:oob&scope=https://www.googleapis.com/auth/analytics&response_type=code

Step 2) Do a post request to fetch the access & refreshment tokens 

curl -i -v --request POST -H "Content-Type: application/x-www-form-urlencoded" -d "code={AUTHORIZATION CODE FROM STEP 1}&client_id={CLIENT ID}.apps.googleusercontent.com&client_secret={CLIENT SECREP}&scope=https://www.googleapis.com/auth/analytics&grant_type=authorization_code&redirect_uri=urn:ietf:wg:oauth:2.0:oob" https://accounts.google.com/o/oauth2/token

Step 3)

Do the actual curl request:

curl -X POST -H "Authorization: Bearer ya29.{AUTH RESPONSE}" -H "Content-Type: application/json" --data "{\"reportRequests\": [{\"viewId\": \"ga:{NUMBER}\",\"dateRanges\": [{\"start_date\": \"2016-01-01\", \"endDate\": \"yesterday\"}],\"metrics\": [{\"expression\":\"ga:sessions\"}], \"dimensions\": [{\"name\":\"ga:dimensions2\"}, {\"name\":\"ga:dimensions3\"}],\"samplingLevel\": \"LARGE\",\"pageToken\": \"\", \"pageSize\": 10000}]}" https://analyticsreporting.googleapis.com/v4/reports:batchGet

And in Streamsets I tried to set it up like this: (I am using images because it is easier to follow.)

1) I put the resource URL the same as step 3, where I fetch the data

2) The Request body is also the same as in step 3

3) And this is the OAuth2 screen. I blurred the credentials.

3b) What I tried: I tried to use the Credentials Grant Type with a Client ID & Client Secret as well.

However I still don't know how to fetch my data. I am not receiving any output and to fetch the data with a CURL request seems to me like a 3 step process, not a 2 step one (as it seems to be in Streamsets). Is Streamsets able to fetch this or should i write a origin of my own? Because to me it seems at first sight that it is not able to do this. I have read both the Streamsets and Google Analytics manuals but they haven't really helped me a lot.

Thank you in advance.

Adriaan
  • 13
  • 1
  • 4
  • Hi Adriaan - I'm trying to reproduce this. What kind of account did you create to get the OAuth client id and secret? Also, you might want to engage with the StreamSets team via one of our community channels - see https://streamsets.com/community/ – metadaddy May 04 '18 at 17:23
  • 1
    Hey, first of all thank you for the answer. I actually solved this by writing a own custom connector to GA. It seems that out of the box it was not so easy to use the already existing client. But the combination of this tutorial: https://github.com/streamsets/tutorials/blob/master/tutorial-origin/readme.md together with https://developers.google.com/analytics/devguides/reporting/core/v4/quickstart/service-java made it pretty easy to figure it out. I think it's because of the Google API. I connected to Purecloud with the client HTTP origin and everything seems to work instantaneously. – Adriaan May 15 '18 at 12:05
  • nice - if you'd like to contribute the custom origin, I think it would be a great addition to Data Collector. Feel free to ping me at pat@streamsets.com. Also - you should post your solution as the answer to this question :-) – metadaddy May 16 '18 at 17:28

0 Answers0