I have replace the MaskRcnn modle of DSP-SLAM with the yolov8 module and have rewritten the codes in the original repository. However, I keep getting a error "Aborted (core dumped)" when I run the yolo model together with DSP-SLAM.
This is the error I get:
DSP-SLAM: Object Oriented SLAM with Deep Shape Priors.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.
Input sensor was set to: Monocular
Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!
/home/linux/GitClone/DSP-SLAM
Ultralytics YOLOv8.0.154 Python-3.8.17 torch-2.0.1 CUDA:0 (NVIDIA GeForce RTX 4070 Ti, 12001MiB)
Setup complete ✅ (24 CPUs, 31.2 GB RAM, 243.4/915.3 GB disk)
device used is cuda:0
Camera Parameters:
- fx: 930.166
- fy: 930.166
- cx: 480
- cy: 270
- k1: -0.147571
- k2: -0.0943432
- p1: 0
- p2: 0
- fps: 15
- color order: RGB (ignored if grayscale)
ORB Extractor Parameters:
- Number of Features: 2000
- Scale Levels: 8
- Scale Factor: 1.2
- Initial Fast Threshold: 20
- Minimum Fast Threshold: 7
-------
Start processing sequence ...
Images in the sequence: 1943
New Map created with 158 points
New Keyframe
terminate called after throwing an instance of 'pybind11::error_already_set'
what(): RuntimeError: std::bad_alloc
At:
/home/linux/anaconda3/envs/reconstruct_M/lib/python3.8/site-packages/ultralytics/utils/torch_utils.py(134): fuse_conv_and_bn
/home/linux/anaconda3/envs/reconstruct_M/lib/python3.8/site-packages/ultralytics/nn/tasks.py(132): fuse
/home/linux/anaconda3/envs/reconstruct_M/lib/python3.8/site-packages/ultralytics/nn/autobackend.py(103): __init__
/home/linux/anaconda3/envs/reconstruct_M/lib/python3.8/site-packages/ultralytics/engine/predictor.py(305): setup_model
/home/linux/anaconda3/envs/reconstruct_M/lib/python3.8/site-packages/ultralytics/engine/model.py(238): predict
/home/linux/anaconda3/envs/reconstruct_M/lib/python3.8/site-packages/torch/utils/_contextlib.py(115): decorate_context
/home/linux/anaconda3/envs/reconstruct_M/lib/python3.8/site-packages/ultralytics/engine/model.py(98): __call__
/home/linux/GitClone/DSP-SLAM/reconstruct/detector2d.py(120): make_prediction
/home/linux/GitClone/DSP-SLAM/reconstruct/mono_sequence.py(79): get_detections
/home/linux/GitClone/DSP-SLAM/reconstruct/mono_sequence.py(151): get_frame_by_id
Aborted (core dumped)
GPU=RTX4070 Ti, Cuda=11.8, torch = 2.0.1
yolov8 runs independently as illustrated on the official documentation
I have tried to change the size of the model from X to L to M to n but no difference.
Have restarted my computer multiple times. But doesn't work.
Have update Ultralytics package.
Have changed the frame rate per second.
This is my code:
class Detector2DY8(object):
def __init__(self, configs):
ultralytics.checks()
config_weights = configs.Detector2DY8.weight_path
self.yolo_model = YOLO(config_weights)
self.yolo_model.to('cuda')
print("device used is ",self.yolo_model.device)
self.min_bb_area = configs.min_bb_area
self.predictions = None
def make_prediction(self, image, object_class="cars"):
assert object_class == "chairs" or object_class == "cars"
self.predictions = self.yolo_model(image)
indices = [i for i, x in enumerate(self.predictions[0].boxes.cls.tolist()) if x in object_class_table[object_class]]
masks = []
n_det = len(indices)
boxes = [[self.predictions[0].boxes.xyxy.tolist()[i]] for i in indices]
masks = [self.predictions[0].masks.data.tolist()[i] for i in indices]
scores = [self.predictions[0].boxes.conf.tolist()[i] for i in indices]
boxes = np.concatenate(boxes, axis=0)
scores = np.stack(scores, axis=0)
if n_det == 0:
masks = np.zeros((0, 0, 0))
else:
masks = np.stack(masks,axis=0)
assert boxes.shape[0] == masks.shape[0]
return self.get_valid_detections(boxes, masks, scores)
def visualize_result(self, image, filename):
self.model.show_result(image, self.predictions, out_file=filename)
def get_valid_detections(self, boxes, masks, scores):
# Remove those on the margin
cond1 = (boxes[:, 0] >= 30) & (boxes[:, 1] > 10) & (boxes[:, 2] < 1211) & (boxes[:, 3] < 366)
boxes_area = (boxes[:, 2] - boxes[:, 0]) * (boxes[:, 3] - boxes[:, 1])
# Remove those with too small bounding boxes
cond2 = (boxes_area > self.min_bb_area)
cond3 = (scores >= 0.70)
valid_mask = (cond2 & cond3)
valid_instances = {"pred_boxes": boxes[valid_mask, :4],
"pred_masks": masks[valid_mask, ...]}
return valid_instances
@staticmethod
def save_masks(masks):
mask_imgs = torch.tensor(masks).cpu().numpy()
n = mask_imgs.shape[0]
for i in range(n):
cv2.imwrite("mask_%d.png" % i, mask_imgs[i, ...].astype(np.float32) * 255.)
``
