The result is empty when prediction of Faster RCNN model (Pytorch)

Question

I'm trying to train Faster RCNN model. After training, I try to predict the result of image but the result is empty.
My data is w: 1600, h: 800, c: 3, classes: 7, bounding boxes:(x1, y1, x2, y2)
My model is below.

My model

import torchvision
from torchvision.models.detection.faster_rcnn import FastRCNNPredictor
from torchvision.models.detection.mask_rcnn import MaskRCNNPredictor
from torchvision.models.detection import FasterRCNN
from torchvision.models.detection.rpn import AnchorGenerator
      
def get_instance_segmentation_model(num_classes):
    backbone = torchvision.models.vgg16(pretrained=True).features
    backbone.out_channels = 512

    anchor_generator = AnchorGenerator(sizes=((32, 64, 128, 256, 512),),
                                    aspect_ratios=((0.5, 1.0, 2.0),))

    roi_pooler = torchvision.ops.MultiScaleRoIAlign(featmap_names=['0'],
                                                    output_size=7,
                                                    sampling_ratio=2)
    model = FasterRCNN(backbone,
                    num_classes=2,
                    rpn_anchor_generator=anchor_generator,
                    box_roi_pool=roi_pooler)
    in_features = model.roi_heads.box_predictor.cls_score.in_features
    model.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)


    return model

device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')

num_classes = 2

model = get_instance_segmentation_model(num_classes)

model.to(device)


params = [p for p in model.parameters() if p.requires_grad]
optimizer = torch.optim.SGD(params, lr=0.005,
                            momentum=0.9, weight_decay=0.0005)

lr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer,
                                               step_size=3,
                                               gamma=0.1)

training

# let's train it for 10 epochs
num_epochs = 10

for epoch in range(num_epochs):
    # train for one epoch, printing every 10 iterations
    train_one_epoch(model, optimizer, train_data_loader, device, epoch, print_freq=10)
    # update the learning rate
    lr_scheduler.step()
    # evaluate on the test dataset
    evaluate(model, valid_data_loader, device=device)

prediction:

prediction
[{'boxes': tensor([], device='cuda:0', size=(0, 4)),
  'labels': tensor([], device='cuda:0', dtype=torch.int64),
  'scores': tensor([], device='cuda:0')}]

I also came across same problem, did you find any solution? – tkarahan Mar 01 '21 at 19:08 — tkarahan, Mar 01 '21 at 19:08

score 3 · Answer 1 · answered Mar 14 '21 at 00:49

You should change the number of classes to

    model = FasterRCNN(backbone,
                    num_classes=YOUR_CLASSES+1,  # +1 is for the background
                    rpn_anchor_generator=anchor_generator,
                    box_roi_pool=roi_pooler)

Remember that the class 0 is reserved for the background, so your classes should start from 1.

Also, please make sure your network has converged during the training.

The result is empty when prediction of Faster RCNN model (Pytorch)

1 Answers1