0

As described in Python API Doc, there are some params in onnxruntime session options coressponding to memory configurations such as:

  • enable_cpu_mem_arena
  • enable_mem_usage
  • enable_mem_pattern

There are some descriptions for them but I can not understaned their usage and the technical concepts behind them precisely.

Can anyone give me some clarity about these parameters?

Mohsen Mahmoodzadeh
  • 111
  • 1
  • 10

1 Answers1

1

enable_mem_pattern

Enable the memory pattern optimization.

The idea is if the input shapes are the same, we could trace the internal memory allocation and generate a memory pattern for future request. So next time we could just do one allocation with a big chunk for all the internal memory allocation.

Note Memory pattern optimization is only available when Sequential Execution mode is enabled (see OrtApi::SetSessionExecutionMode)

enable_cpu_mem_arena

Enable the memory arena on CPU.

Arena may pre-allocate memory for future usage.

Info based on the api doc: https://onnxruntime.ai/docs/api/c/struct_ort_api.html

Y00
  • 666
  • 1
  • 7
  • 23