How can one profile an ONNX model with random inputs, without having to specify the input shape?

Question

I was given several ONNX models. I'd like to profile them. How can one profile them with random inputs, without having to specify the input shape?

I'd prefer not to have to manually find out the input shape for each model and format my random inputs accordingly.

You can see appropriate code in profiling script of onnxruntime https://github.com/microsoft/onnxruntime/blob/8a86b346a5fef042ab8a91a50eb05feab482e122/onnxruntime/python/tools/transformers/profiler.py#L464-L503 — SolomidHero, Aug 08 '22 at 21:29

score 0 · Answer 1 · answered Aug 08 '22 at 21:04

If your model is in ONNX format, it has info about shapes stored in it. Thus, you don't need to specify them manually. So you need to read model by onnx.load function, then capture all info from .graph.input (list of input infos) attribute for each input and then create randomized inputs.

This snippet will help. It assumes that sometimes inputs has dynamic shape dims (like 'length' or 'batch' dims that can be variable on inference):

import numpy as np
import onnx
from onnx import mapping
model = onnx.load('/path/to/model.onnx')


input_types = [
  mapping.TENSOR_TYPE_TO_NP_TYPE[_input.type.tensor_type.elem_type]
  for _input in model.graph.input
]

# random size for dynamic dims
input_dynamic_sizes = {
  d.dim_param: np.random.randint(10, 20)
  for _input in model.graph.input
  for d in _input.type.tensor_type.shape.dim
  if d.dim_param != ''
}

# parse shapes of inputs
# assign randomed value to dynamic dims
input_shapes = [
  tuple(
    d.dim_value if d.dim_value > 0 else input_dynamic_sizes[d.dim_param]
    for d in _input.type.tensor_type.shape.dim
  )
  for _input in model.graph.input
]

# randomize inputs with given shape and types
# change this to match your required test inputs
inputs = [
  (np.random.randn(*_shape) * 10).astype(_type)
  for _shape, _type in zip(input_shapes, input_types)
]

Example of model.graph.input:

[name: "input"
type {
  tensor_type {
    elem_type: 1
    shape {
      dim {
        dim_value: 1
      }
      dim {
        dim_param: "sequence"
      }
    }
  }
}
, name: "h0"
type {
  tensor_type {
    elem_type: 1
    shape {
      dim {
        dim_value: 2
      }
      dim {
        dim_value: 1
      }
      dim {
        dim_value: 64
      }
    }
  }
}
, name: "c0"
type {
  tensor_type {
    elem_type: 1
    shape {
      dim {
        dim_value: 2
      }
      dim {
        dim_value: 1
      }
      dim {
        dim_value: 64
      }
    }
  }
}
]

How can one profile an ONNX model with random inputs, without having to specify the input shape?

1 Answers1