-2

I use Pentaho data integration to create the job for load data to Google BigQuery from Google Cloud Storage with "Google BigQuery Loader" step. While the step successfully load data into BigQuery dataset table (checked by BigQuery job log and table data), it throws NPE:

2019/10/24 10:21:31 - Job 1 - Starting entry [Google BigQuery Loader]
2019/10/24 10:21:31 - Job 1 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : java.lang.NullPointerException
2019/10/24 10:21:31 - Job 1 -   at com.pentaho.di.job.entries.google.bigquery.JobEntryBigQueryLoader.execute(JobEntryBigQueryLoader.java:383)
2019/10/24 10:21:31 - Job 1 -   at org.pentaho.di.job.Job.execute(Job.java:680)
2019/10/24 10:21:31 - Job 1 -   at org.pentaho.di.job.Job.execute(Job.java:821)
2019/10/24 10:21:31 - Job 1 -   at org.pentaho.di.job.Job.execute(Job.java:497)
2019/10/24 10:21:31 - Job 1 -   at org.pentaho.di.job.Job.run(Job.java:384)
2019/10/24 10:21:31 - Job 1 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : A serious error occurred during job execution: 
2019/10/24 10:21:31 - Job 1 - Unexpected error occurred while launching entry [Google BigQuery Loader.0]
2019/10/24 10:21:31 - Job 1 -  at org.pentaho.di.job.Job.run (Job.java:384)
2019/10/24 10:21:31 - Job 1 -  at org.pentaho.di.job.Job.execute (Job.java:497)
2019/10/24 10:21:31 - Job 1 -  at org.pentaho.di.job.Job.execute (Job.java:821)
2019/10/24 10:21:31 - Job 1 -  at org.pentaho.di.job.Job.execute (Job.java:680)
2019/10/24 10:21:31 - Job 1 -  at com.pentaho.di.job.entries.google.bigquery.JobEntryBigQueryLoader.execute (JobEntryBigQueryLoader.java:383)
2019/10/24 10:21:31 - Job 1 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : org.pentaho.di.core.exception.KettleException: 
2019/10/24 10:21:31 - Job 1 - Unexpected error occurred while launching entry [Google BigQuery Loader.0]
2019/10/24 10:21:31 - Job 1 -  at org.pentaho.di.job.Job.run (Job.java:384)
2019/10/24 10:21:31 - Job 1 -  at org.pentaho.di.job.Job.execute (Job.java:497)
2019/10/24 10:21:31 - Job 1 -  at org.pentaho.di.job.Job.execute (Job.java:821)
2019/10/24 10:21:31 - Job 1 -  at org.pentaho.di.job.Job.execute (Job.java:680)
2019/10/24 10:21:31 - Job 1 -  at com.pentaho.di.job.entries.google.bigquery.JobEntryBigQueryLoader.execute (JobEntryBigQueryLoader.java:383)
2019/10/24 10:21:31 - Job 1 - 
2019/10/24 10:21:31 - Job 1 -   at org.pentaho.di.job.Job.execute(Job.java:824)
2019/10/24 10:21:31 - Job 1 -   at org.pentaho.di.job.Job.execute(Job.java:497)
2019/10/24 10:21:31 - Job 1 -   at org.pentaho.di.job.Job.run(Job.java:384)
2019/10/24 10:21:31 - Job 1 - Caused by: java.lang.NullPointerException
2019/10/24 10:21:31 - Job 1 -   at com.pentaho.di.job.entries.google.bigquery.JobEntryBigQueryLoader.execute(JobEntryBigQueryLoader.java:383)
2019/10/24 10:21:31 - Job 1 -   at org.pentaho.di.job.Job.execute(Job.java:680)
2019/10/24 10:21:31 - Job 1 -   at org.pentaho.di.job.Job.execute(Job.java:821)
2019/10/24 10:21:31 - Job 1 -   ... 2 more
2019/10/24 10:21:31 - Spoon - Job has ended.

PDI 8.1 and 8.2 show the same result. PDI 8.3 do not have Google BigQuery Loader step.

Thanks for any help or a workaround.

reddto
  • 27
  • 1
  • 1
  • 7

1 Answers1

0

I got workaround with PDI Shell Script step and gcloud bq cli. Issue in Pentaho bugtracker.

reddto
  • 27
  • 1
  • 1
  • 7
  • The name "[Google BigQuery Loader](https://help.pentaho.com/Documentation/8.1/Products/Data_Integration/Job_Entry_Reference/Google_BigQuery_Loader)" could be confusing. This extention/plugin belongs to Pentaho, not BigQuery Since you are using Cloud Storage and BigQuery (both are services in GCP), you are able to [load data directly into BigQuery from Cloud Storage](https://cloud.google.com/bigquery/docs/loading-data-cloud-storage) and viceversa. – rsantiago Nov 12 '19 at 01:15