3

Vertex AI endpoints provide Batch Prediction functionality that is very useful to perform predictions on a large amount of data.

However, every time I make a new batch prediction the endpoint creates a new Dataset inside BigQuery. This is very uncomfortable because if I make 100 batch predictions per day, I will obtain 100 new Datasets inside BigQuery.

Is there a way to make all predictions converge inside the same Dataset? I mean, each prediction is a new Table inside a specific Dataset of BigQuery.

user15064662
  • 31
  • 1
  • Hello, welcome to Stack Overflow! If this or any answer has solved your question please consider [accepting it](https://meta.stackexchange.com/q/5234/179419) by clicking the check-mark. This indicates to the wider community that you've found a solution and gives some reputation to both the answerer and yourself. Another option is to [upvote the answer](https://stackoverflow.com/help/someone-answers) if you feel it is useful for you. There is no obligation to do this. – Hector Martinez Rodriguez Sep 12 '22 at 14:30

1 Answers1

0

Yes there is a way to make Vertex predictions in the same Dataset you need, to specify the Dataset ID in the OUTPUT_URI in your Batch Prediction.

bq://bqprojectId.bqDatasetId

If you just specify the Project ID, Vertex AI creates a new output dataset for you. Specifying the dataset ID is an optional resource.

You can also specify the table ID but this is not always the best.

bq://bqprojectId.bqDatasetId.bqTableId

You can see in this document the destination prefix while doing a batch prediction.

Jose Gutierrez Paliza
  • 1,373
  • 1
  • 5
  • 12