Optimizing Sentence Transformer models using HuggingFace Optimum

Question

I am looking to optimize some of the sentence transformer models from huggingface using optimum library. I am following the below documentation:

https://huggingface.co/blog/optimum-inference

I understand the process but I am not able to use model_id because our network restricts accessing huggingface using its APIs. I have downloaded these models locally and I am trying to following the same procedure but I cannot find any examples on how to pass the onnx checkpoint model to ORTOptimizer class.

This is how I converted the vanilla model to onnx checkpoint:

from pathlib import Path
import transformers
from transformers.onnx import FeaturesManager
from transformers import AutoConfig, AutoTokenizer, AutoModel
# load model and tokenizer
feature = "question-answering"
model_path=r'local\path\to\all-MiniLM-L12-v2'
model = AutoModel.from_pretrained(model_path)
tokenizer = AutoTokenizer.from_pretrained(model_path)

model_kind, model_onnx_config = FeaturesManager.check_supported_model_or_raise(model, feature=feature)
onnx_config = model_onnx_config(model.config)

onnx_path = Path("onnx")

# export
onnx_inputs, onnx_outputs = transformers.onnx.export(
        preprocessor=tokenizer,
        model=model,
        config=onnx_config,
        opset=13,
        output=onnx_path
)

This saves the model as onnx checkpoint. Then I try to load the onnx model along with its original model configuration:

from optimum.onnxruntime import ORTModelForQuestionAnswering

task = "question-answering"

ORTModelForQuestionAnswering(model= onnx_path, config = r'local\path\to\all-MiniLM-L12-v2\config.json')

I get the following error from the above code:

C:\Anaconda3\lib\site-packages\optimum\onnxruntime\modeling_ort.py in __init__(self, model, config, use_io_binding, **kwargs)
    641 
    642     def __init__(self, model=None, config=None, use_io_binding=True, **kwargs):
--> 643         super().__init__(model, config, use_io_binding, **kwargs)
    644         # create {name:idx} dict for model outputs
    645         self.model_outputs = {output_key.name: idx for idx, output_key in enumerate(self.model.get_outputs())}

C:\Anaconda3\lib\site-packages\optimum\onnxruntime\modeling_ort.py in __init__(self, model, config, use_io_binding, **kwargs)
    123         self.model_save_dir = kwargs.get("model_save_dir", None)
    124         self.latest_model_name = kwargs.get("latest_model_name", "model.onnx")
--> 125         self.providers = model.get_providers()
    126         self._device = get_device_for_provider(self.providers[0])
    127 

AttributeError: 'WindowsPath' object has no attribute 'get_providers'

How do I resolve this issue and is there any examples for the process that I am trying to do i.e. load sentence transformer models locally rather than using model_id.

Appreciate all the help!

Optimizing Sentence Transformer models using HuggingFace Optimum

0 Answers0