How to get values return by Tuple Object in Maskcrnn libtorch

Question

I’m new in C++ and libtorch, I try load model by torchscript and execute inference, the codes like below:

   torch::jit::script::Module module;
   try {
       module = torch::jit::load("../../weights/card_extraction/pytorch/2104131340/best_model_27_mAP=0.9981_torchscript.pt");
   }
   catch (const c10::Error& e) {
      std::cerr << "Error to load model\n";
      return -1;
   }
   std::cout << "Load model successful!\n";
   torch::DeviceType device_type;
   device_type = torch::kCPU;
   torch::Device device(device_type, 0);
   module.to(device);
   
   torch::Tensor sample = torch::zeros({3, 800, 800});
   std::vector<torch::jit::IValue> inputs;
   std::vector<torch::Tensor> images;
   images.push_back(sample);
   /* images.push_back(torch::ones({3, 224, 224})); */

   inputs.push_back(images);

   auto t1 = std::chrono::high_resolution_clock::now();
   auto output = module.forward(inputs);
   auto t2 = std::chrono::high_resolution_clock::now();
   int duration = std::chrono::duration_cast<std::chrono::milliseconds> (t2 - t1).count();
   std::cout << "Inference time: " << duration << " ms" << std::endl;
   std::cout << output << std::endl;

And the result like this:

Load model successful!
[W mask_rcnn.py:86] Warning: RCNN always returns a (Losses, Detections) tuple in scripting (function )
Inference time: 2321 ms
({}, [{boxes: [ CPUFloatType{0,4} ], labels: [ CPULongType{0} ], scores: [ CPUFloatType{0} ], masks: [ CPUFloatType{0,1,800,800} ]}])

How do I get value boxes, labels, scores and masks from return output object using c++ ? I tried many ways but compile always error with “c10::IValue” error thrown.

And more question, why is the time inference when I convert the model to torchscript, executed by C++ is slower than python? Many thanks

Answer 1

You can get access to the element like here: I can parse Tuple arguments and get access to it like in Tensor format. It can help u.

auto output1_t = output.toTuple()->elements()[0].toTensor();
auto output2_t = output.toTuple()->elements()[1].toTensor();

https://discuss.pytorch.org/t/how-can-i-get-access-to-first-and-second-tensor-from-tuple-returned-from-forward-method-in-libtorch-c-c/139741

Answer 2

To draw bounding boxes you can try the following code

auto bbox = output.at("pred_boxes").toTensor();
int num_instances = bbox.sizes()[0];
std::cout << mask.size() << std::endl;
//cv::resize(img, img, cv::Size(244, 244));
for(int num = 0; num < bbox.sizes()[0]; ++num)
{
    float x1 = bbox[num][0].item().toFloat();
    float y1 = bbox[num][1].item().toFloat();
    float x2 = bbox[num][2].item().toFloat();
    float y2 = bbox[num][3].item().toFloat();
    cv::rectangle(img, cv::Point2f(x1, y1), cv::Point2f(x2, y2), cv::Scalar(255));
}