Running detectron2 in C++. Encountered TorchVision Error while building tools/deploy/torchscript_mask_rcnn.cpp

Question

I want to run the detectron2 model in C++. I am using the steps mentioned in the readme. I took the following steps:

1. I setup libtorch by unzipping libtorch-cxx11-abi-shared-with-deps-2.0.0+cpu.zip in /home/hiqbal/Downloads/libtorch
2. Installed torch vision by following steps here. This installs the torchVision /usr/local/include/torchvision/.
3. Used /tools/deploy/export_model.py to convert the model to .tr. The command used to export the model is:

./export_model.py --config-file ../../configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml \
    --output ./output --export-method tracing --format torchscript \
    MODEL.WEIGHTS /home/pathto/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl \
    MODEL.DEVICE CPU

The model was successfully exported and model.tr is created in /deploy/output/

4. Then to build tools/deploy/torchscript_mask_rcnn.cpp , I used the following command:

cmake -DCMAKE_PREFIX_PATH=/home/hiqbal/Downloads/libtorch

The above command generated the following error:

~/detectron2/tools/deploy$ cmake -DCMAKE_PREFIX_PATH=/home/hiqbal/Downloads/libtorch
CMake Warning:
  No source or binary directory provided.  Both will be assumed to be the
  same as the current working directory, but note that this warning will
  become a fatal error in future CMake releases.


-- Caffe2: Found gflags with new-style gflags target.
-- Caffe2: Found glog with new-style glog target.
-- Caffe2: Found protobuf with old-style protobuf targets.
-- Caffe2: Protobuf version 3.12.4
-- Caffe2: Found gflags with new-style gflags target.
-- Caffe2: Found glog with new-style glog target.
-- Caffe2: Found protobuf with old-style protobuf targets.
-- Caffe2: Protobuf version 3.12.4
-- Configuring done (0.0s)
CMake Error in CMakeLists.txt:
  Imported target "TorchVision::TorchVision" includes non-existent path

    "/usr/lib/include"

  in its INTERFACE_INCLUDE_DIRECTORIES.  Possible reasons include:

  * The path was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and references files it does not
  provide.

-- Generating done (0.0s)
CMake Generate step failed.  Build files cannot be regenerated correctly.

Just for the sake of it, I tried cmake -DCMAKE_PREFIX_PATH=/home/hiqbal/Downloads/libtorch:/usr/local/include/torchvision/ but no luck.

---

To test torchVision, I created a test directory and created main.cpp

main.cpp

#include <torch/torch.h>
// #include <torchvision/vision.h>
#include </usr/local/include/torchvision/vision.h>

int main() {
  // Load an image from a file
  auto img = torch::vision::image::ReadImage("/home/hiqbal/Downloads/sample_abc.jpg");

  // Convert image to tensor
  auto tensor = torch::vision::image::ToTensor(img);

  // Normalize image tensor
  auto mean = torch::tensor({0.485, 0.456, 0.406});
  auto std = torch::tensor({0.229, 0.224, 0.225});
  auto normalized = torch::vision::transforms::Normalize(mean, std)(tensor);

  // Apply a transformation
  auto transformed = torch::vision::transforms::Resize(256)(normalized);

  // Save the transformed image
  torch::vision::image::WriteImage("transformed.jpg", transformed);

  return 0;
}

and CMakeLists.txt

cmake_minimum_required(VERSION 3.0)
# project name
project(debugfunc)

# define path to the libtorch extracted folder
set(CMAKE_PREFIX_PATH /home/hiqbal/Downloads/libtorch)

find_package(Torch REQUIRED)
find_package(TorchVision REQUIRED)

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${TORCH_CXX_FLAGS}")

add_executable(${PROJECT_NAME} main.cpp)
target_compile_features(${PROJECT_NAME} PUBLIC cxx_range_for)
target_link_libraries(${PROJECT_NAME} TorchVision::TorchVision)
set_property(TARGET ${PROJECT_NAME} PROPERTY CXX_STANDARD 14)

Building this code generates the following error:

/usr/local/include/torchvision/vision.h:10:40: warning: ‘_register_ops’ initialized and declared ‘extern’
   10 | extern "C" VISION_INLINE_VARIABLE auto _register_ops = &cuda_version;
      |                                        ^~~~~~~~~~~~~
/home/hiqbal/testcode/main.cpp: In function ‘int main()’:
/home/hiqbal/testcode/main.cpp:19:21: error: ‘torch::vision’ has not been declared
   19 |   auto img = torch::vision::image::ReadImage("/home/hiqbal/Downloads/sample_abc.jpg");
      |                     ^~~~~~
/home/hiqbal/testcode/main.cpp:22:24: error: ‘torch::vision’ has not been declared
   22 |   auto tensor = torch::vision::image::ToTensor(img);
      |                        ^~~~~~
/home/hiqbal/testcode/main.cpp:27:28: error: ‘torch::vision’ has not been declared
   27 |   auto normalized = torch::vision::transforms::Normalize(mean, std)(tensor);
      |                            ^~~~~~
/home/hiqbal/testcode/main.cpp:30:29: error: ‘torch::vision’ has not been declared
   30 |   auto transformed = torch::vision::transforms::Resize(256)(normalized);
      |                             ^~~~~~
/home/hiqbal/testcode/main.cpp:33:10: error: ‘torch::vision’ has not been declared
   33 |   torch::vision::image::WriteImage("transformed.jpg", transformed);

I would really appreciate help in addressing this blocker. The purpose is to run detectron2 model in C++. If someone has a better approach along with all the sample code and steps, that would be great help. Thanks.

Answer 1

The solution that worked for me is:

1. Removing the torchVision installation
2. Pulling the GitHub repo for torchVision and using cmake to build it.