0
Run at startup - downloads the model
@app.on_event("startup")
def startup():
path = os.environ["AIP_STORAGE_URI"]
path = os.path.join(path, model_file)
logger.info(f"Loading model state from: {path}")

if path.startswith("gs://"):
        destination_file_name = f"/tmp/{model_file}"
        storage_client = storage.Client()
        with open(destination_file_name, "wb") as file:
            # Download file from GCP
            storage_client.download_blob_to_file(path, file)

    else:
        logger.info("Local path...")
        destination_file_name = path
    
    model_files = joblib.load(destination_file_name)
    global_items["model"] = model_files


    logger.info("State successfully retrieved from GCP")



@app.get("/health")
async def health_check():
    """Health check endpoint."""
    return Response("healthy", status_code=status.HTTP_200_OK)


# Define prediction request logic
@app.get("/predict")
async def prediction(data: Data):
    """Make predictions"""
    try:

    except Exception as error:

    finally:


if __name__ == "__name__":
    uvicorn.run(app, host="0.0.0.0", port=8080, reload=False, log_level="info")

Using a model from the model registry and Image that has the following code. Endpoint is logs up until logger.info("State successfully retrieved from GCP") but gets stuck afterwards on uvicorn.access:send:480 - 10.20.3.1:33074 - "GET /health HTTP/1.1" 404..

Please help on how to get the health check right, if that's the problem. Thanks

  • I'm guessing here, but does loading the model take a lot of time? Could it be that GCP is running health checks while it's still loading? – M.O. Nov 02 '22 at 20:16
  • Thanks for replying. The model is being loaded successfully, I think, as this log shows: logger.info("State successfully retrieved from GCP") – Moustafa Ashraf Nov 02 '22 at 21:52
  • 1
    Yes, but I'm thinking that GCP might be trying to access the healthcheck endpoint while the model is still loading, and that's why the server is not responding. – M.O. Nov 02 '22 at 22:24
  • Oh, that's a good point. Let me investigate this. – Moustafa Ashraf Nov 03 '22 at 20:44
  • I realised that I had a mistake when specifying the health route in the serving container that is attached to the model. I am confused whether to be happy or feel sad on the time wasted.. :'( – Moustafa Ashraf Nov 08 '22 at 15:38

0 Answers0