Vertex AI size limit when predicting on custom model

Question

I made a picture classification model, trained it and deployed it using vertex AI and linked it to an endpoint. I manage to make prediction using this code :

from typing import Dict, List, Union

from google.cloud import aiplatform
from google.protobuf import json_format
from google.protobuf.struct_pb2 import Value


def predict_custom_trained_model_sample(
    project: str,
    endpoint_id: str,
    serving_input: str,
    b64str: [],
    location: str = "europe-west1",
    api_endpoint: str = "europe-west1-aiplatform.googleapis.com",
):


# The AI Platform services require regional API endpoints.
client_options = {"api_endpoint": api_endpoint}

# Initialize client that will be used to create and send requests.
# This client only needs to be created once, and can be reused for multiple requests.

client = aiplatform.gapic.PredictionServiceClient(client_options=client_options)

parameters_dict = {}
parameters = json_format.ParseDict(parameters_dict, Value())
endpoint = client.endpoint_path(
    project=project, location=location, endpoint=endpoint_id
)

instances = [{serving_input: {"b64": i}} for i in b64str]

response = client.predict(
    endpoint=endpoint, instances=instances, parameters=parameters
)

print(" deployed_model_id:", response.deployed_model_id)

# The predictions are a google.protobuf.Value representation of the model's predictions.

predictions = response.predictions

for prediction in predictions:
    print("prediction:", prediction)


import base64
paths = [path_1, path_2, path3 etc...]
b64str = []
for path in paths:
    with open(path, "rb") as f:
         file_content = f.read()
         b64str.append(base64.b64encode(file_content).decode("utf-8"))

predict_custom_trained_model_sample(
    project=ProjectID,
    endpoint_id=EndpointID,
    serving_input=serving_input,
    b64str=b64str,
    location="europe-west1")

The problem is on some picture I got this error :

_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
    status = StatusCode.FAILED_PRECONDITION
    details = "The request size (3240080 bytes) exceeds 1.500MB limit."
    debug_error_string = "{"created":"@1665500473.023741583","description":"Error received from peer ipv4:74.125.140.95:443","file":"/home/conda/feedstock_root/build_artifacts/grpc-split_1656146941531/work/src/core/lib/surface/call.cc","file_line":966,"grpc_message":"The request size (3240080 bytes) exceeds 1.500MB limit.","grpc_status":9}"

but those pictures are totally normal and I can use my model on them without problems when loading it locally.

I've tried reshaping every picture to (512, 512) because my model trained on this specific shape but it's even worst as no picture can be predicted when doing this.

Does someone know how to do it in another way or how to force vertex to accept larger file ?

score 1 · Answer 1 · answered Nov 11 '22 at 07:41

This is a limitation from Vertex AI model serving, there is a public feature request link which is the second most voted feature. Please vote and a comment to show such feature is important for us. I could be that Cloud Run don't have such limitation but you will loose some nice feature of Vertex AI model serving.

Vertex AI size limit when predicting on custom model

1 Answers1