0

How can I find the list of all the environment variables supported in a HuggingFace model?

E.g. on https://huggingface.co/tiiuae/falcon-40b-instruct model, based on the comment provided in this post: https://discuss.huggingface.co/t/cpu-memory-utilization-too-high-when-running-inference-on-falcon-40b-instruct/44200/2

I see that the model supports environment variables for defining

MAX_CONCURRENT_REQUESTS(default 128)
MAX_INPUT_LENGTH(default 1000)
MAX_TOTAL_TOKENS (default 1512)
MAX_BATCH_SIZE (default none)
SplitXor
  • 165
  • 1
  • 10

0 Answers0