FastAPI with Ultralytics YOLOv8: "NameError: predictions is not defined"

Question

I am using FastAPI to serve a Yolov8 trained model from the Ultralytics library for object detection. However, I'm encountering an issue when trying to predict using the loaded model. The specific error I get is:

NameError: predictions is not defined.

Here is the relevant code I have:

import io
import logging
import numpy as np
from typing import Dict, List
from pathlib import Path
from PIL import Image
from aiseal.utils.config import get_settings
from aiseal.utils.general import (
    _wrapped,
)
from hashlib import sha1
from ultralytics import YOLO

settings = get_settings()
model_path = Path(settings.model_path)
logger = logging.getLogger("fastapi")
# Load the model (this should ideally be done once during startup)
model = YOLO(model_path)

def preprocess_image(img_bytes: bytes) -> np.ndarray:
    """
    Convert byte image to the format your model expects.
    Resize and normalize.
    """
    # Convert bytes to PIL Image
    image = Image.open(io.BytesIO(img_bytes))
    
    # Resize to the input size your model expects (e.g., 224x224 for common architectures)
    image = image.resize((640, 640))
    
    # Convert to numpy array and normalize
    image_array = np.asarray(image) / 255.0
    
    # Add batch dimension (because deep learning models usually expect a batch of images, not a single image)
    image_array = np.expand_dims(image_array, axis=0)
    
    return image_array


async def inference(img_bytes: bytes) -> List:
    # Process the image
    processed_image = preprocess_image(img_bytes)

    predictions = model.predict(source = processed_image)
    class_idx = np.argmax(predictions[0])
        
    # For now, return the class index. In a real-world scenario, you might map this to a class name or take further steps.
    result = [class_idx]

    return result

I've ensured:

The model loads successfully without errors.
The image preprocessing works as expected
The error occurs specifically at the following line

predictions = model.predict(source = processed_image)

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\uvicorn\protocols\http\httptools_impl.py", line 426, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\fastapi\applications.py", line 289, in __call__
    await super().__call__(scope, receive, send)
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\starlette\applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\starlette\middleware\errors.py", line 184, in __call__
    raise exc
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\starlette\middleware\errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\starlette\middleware\exceptions.py", line 79, in __call__
    raise exc
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\starlette\middleware\exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 20, in __call__
    raise e
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\starlette\routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\starlette\routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\starlette\routing.py", line 66, in app
    response = await func(request)
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\fastapi\routing.py", line 273, in app
    raw_response = await run_endpoint_function(
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\fastapi\routing.py", line 190, in run_endpoint_function
    return await dependant.call(**values)
  File "C:\Users\gyjch\Desktop\aiseal\src\aiseal\routers\psdetect.py", line 34, in inference
    data = await sealdetect.inference(image_bytes)
  File "C:\Users\gyjch\Desktop\aiseal\src\aiseal\inference\sealdetect.py", line 44, in inference
    predictions = model.predict(source = processed_image)
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\ultralytics\engine\model.py", line 246, in predict
    return self.predictor.predict_cli(source=source) if is_cli else self.predictor(source=source, stream=stream)
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\ultralytics\engine\predictor.py", line 197, in __call__
    return list(self.stream_inference(source, model, *args, **kwargs))  # merge list of Result into one
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\utils\_contextlib.py", line 35, in generator_context
    response = gen.send(None)
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\ultralytics\engine\predictor.py", line 249, in stream_inference     
    im = self.preprocess(im0s)
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\ultralytics\engine\predictor.py", line 124, in preprocess
    im = np.stack(self.pre_transform(im))
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\ultralytics\engine\predictor.py", line 150, in pre_transform        
    return [LetterBox(self.imgsz, auto=auto, stride=self.model.stride)(image=x) for x in im]
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\ultralytics\engine\predictor.py", line 150, in <listcomp>
    return [LetterBox(self.imgsz, auto=auto, stride=self.model.stride)(image=x) for x in im]
  File "C:\Users\gyjch\AppData\Local\Programs\Python\Python39\lib\site-packages\ultralytics\data\augment.py", line 587, in __call__
    img = cv2.copyMakeBorder(img, top, bottom, left, right, cv2.BORDER_CONSTANT,
cv2.error: OpenCV(4.8.0) D:\a\opencv-python\opencv-python\opencv\modules\core\src\copy.cpp:1026: error: (-215:Assertion failed) top >= 0 && bottom >= 0 && left >= 0 && right >= 0 && _src.dims() <= 2 in function 'cv::copyMakeBorder'

Can anyone provide insights into what might be causing this?

can you share the entire traceback? because in your error it shows `Predictions` whereas in your code it shows `predictions` — Jeril, Aug 16 '23 at 01:55
The error on that traceback is not the same error in the post title. Which is it? — John Gordon, Aug 16 '23 at 02:10
@JohnGordon Sorry, I am new to coding. I add a break point and found the predictions is not defined — J.c, Aug 16 '23 at 02:13
can you run your code without breakpoints and check the error — Jeril, Aug 16 '23 at 02:15
@Jeril I run my code, and it runs well, and I can also go to localhost:58000/docs to post my image, but after I upload the image, it show 500 internal server error and have that traceback — J.c, Aug 16 '23 at 02:21

FastAPI with Ultralytics YOLOv8: "NameError: predictions is not defined"

0 Answers0