I am using a pretrained bert model by azure automl
import joblib
from azureml.core.model import Model
from azureml.core import Workspace
import os
import pickle
ws = Workspace.from_config()
path=Model.get_model_path(model_name, version=1, _workspace=ws)
model = joblib.load(os.path.join(path, model_file))
The model is loaded properly, but when I try to predict using a dataframe as input:
model.predict(df)
The prediction is done by a batch size of 8, is there any way to change the batch size as it is so slow?
I have only access to the pkl file of the trained model.