1

I want to reduce the object detection model size. For the same, I tried optimising Faster R-CNN model for object detection using pytorch-mobile optimiser, but the .pt zip file generated is of the same size as that of the original model size.

I used the code mention below

import torch
import torchvision
from torch.utils.mobile_optimizer import optimize_for_mobile

model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)

model.eval()
script_model = torch.jit.script(model)
from torch.utils.mobile_optimizer import optimize_for_mobile
script_model_vulkan = optimize_for_mobile(script_model, backend='Vulkan')
torch.jit.save(script_model_vulkan, "frcnn.pth")
Ivan
  • 34,531
  • 8
  • 55
  • 100
Shalini k
  • 27
  • 6

1 Answers1

0

You have to quantize your model first
follow these steps here
& then use these methods

from torch.utils.mobile_optimizer import optimize_for_mobile
script_model_vulkan = optimize_for_mobile(script_model, backend='Vulkan')
torch.jit.save(script_model_vulkan, "frcnn.pth")

EDIT:

Quantization process for resnet50 model

import torchvision
model = torchvision.models.resnet50(pretrained=True)
import os
import torch

def print_model_size(mdl):
    torch.save(mdl.state_dict(), "tmp.pt")
    print("%.2f MB" %(os.path.getsize("tmp.pt")/1e6))
    os.remove('tmp.pt')
print_model_size(model) # will print original model size
backend = "qnnpack"
model.qconfig = torch.quantization.get_default_qconfig(backend)
torch.backends.quantized.engine = backend
model_static_quantized = torch.quantization.prepare(model, inplace=False)
model_static_quantized = torch.quantization.convert(model_static_quantized, inplace=False)



print_model_size(model_static_quantized) ## will print quantized model size
Prajot Kuvalekar
  • 5,128
  • 3
  • 21
  • 32
  • I tried the quantization followed by torch scripting, it worked. Thank you. But when i tried to quantize frcnn model, got the following error, "RuntimeError: Quantized backend not supported" `model_quantized= torchvision.models.quantization.resnet50(pretrained=True, quantize=True)`, let me know what changes needs to be done here.. – Shalini k Feb 02 '21 at 07:30
  • can you check what this returns `torch.backends.quantized.supported_engines`. And which os are you using? – Prajot Kuvalekar Feb 02 '21 at 08:55
  • I tried `torch.backends.quantized.supported_engines`, i got ['qnnpack' , 'none']. OS is ubuntu. – Shalini k Feb 02 '21 at 09:10
  • 'qnnpack' , should be OK....need to look @ ur code once....if its private try to debug on ur own.... – Prajot Kuvalekar Feb 02 '21 at 09:54
  • `import torchvision` `model_quantized=torchvision.models.quantization.resnet18(pretrained=True`,`quantize=True)` `model = torchvision.models.resnet18(pretrained=True)' `import os` `import torch` `def print_model_size(mdl):` `torch.save(mdl.state_dict(), "tmp1.pt")` `print("%.2f MB" %(os.path.getsize("tmp1.pt")/1e6))` `os.remove('tmp.pt')` `print_model_size(model)` `print_model_size(model_quantized)` I'm trying to quantize resnet and frcnn model – Shalini k Feb 02 '21 at 10:13
  • I'm trying to reduce the resnet and frcnn model size and use them for object detection on edge device. Im using the same quantization method you mentioned in the above comment. – Shalini k Feb 02 '21 at 10:23
  • so it worked for frcnn & not resnet50..? is my understanding correct? – Prajot Kuvalekar Feb 02 '21 at 10:52
  • It only worked for mobilenetV2. I'm getting runtime error for both frcnn and resnet while quantizing it. – Shalini k Feb 02 '21 at 10:58
  • hi i guess u have taken quantized model which are alraedy available in torchvision module and tested its file size.(which ll always execute. i ll update my answer for resnet50.plz check it. – Prajot Kuvalekar Feb 02 '21 at 11:53
  • It worked, even tried for frcnn. Thank you. – Shalini k Feb 02 '21 at 12:07
  • can u plz mark it as correct answer and vote it ;-) – Prajot Kuvalekar Feb 02 '21 at 12:07
  • and also the vote(this answer is useful)? – Prajot Kuvalekar Feb 02 '21 at 12:11