5

I am using Tensorflow's Anaconda distribution with MKL support.

from tensorflow.python.framework import test_util
test_util.IsMklEnabled()

This code prints True. However, when I compile my Keras model I still get

Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2

This is not the behavior I was expecting.

ozgur
  • 2,549
  • 4
  • 25
  • 40
  • 1
    almost certainly just complaining that you're not using the optimal version, and if you changed to a version supporting AVX2, you could get better performance. – hanshenrik Dec 30 '18 at 17:50
  • Do you get any speedup in anything from building Tensorflow yourself with MKL + AVX2+FMA enabled? If there's any work that isn't handed off to MKL functions, it would be done with 128-bit SSE/SSE2 instructions, not 256-bit AVX. – Peter Cordes Jan 05 '19 at 00:28
  • @PeterCordes I'm going to answer that detail asap. – ozgur Jan 06 '19 at 22:23

2 Answers2

4

The MKL-DNN portions of the tensorflow execution (which is the main acceleration provided by MKL engineers) are JIT'ed at runtime. So the instruction set targeted at binary creation is not relevant to the MKL-DNN code. It will only effect the other math, mainly the Eigen library and whatever functions haven't been replaced with MKL-DNN functions.

Nathan Greeneltch
  • 84
  • 2
  • Can you point me to a document or something of that sort that I can be sure about it? If what you say is true, it seems like a bug. Because AFAIK, tensorflow and numpy uses MKL (if available) for all vector operations; no need for AVX2. – ozgur Jan 08 '19 at 10:59
  • The JITing stategy is explained in this old github answer by vpirogov (he is lead ENG on the mkl-dnn team). https://github.com/intel/mkl-dnn/issues/3 . otherwise, I don't know of any official docs that mention this. – Nathan Greeneltch Jan 09 '19 at 18:01
  • also, the warning is addressed in my own doc on Intel's site in the "Warning on ISA above AVX2:" section: https://software.intel.com/en-us/articles/intel-optimization-for-tensorflow-installation-guide – Nathan Greeneltch Jan 09 '19 at 18:25
  • @NathanGreeneltch, slightly offtopic, but related question. If the host system is rather old, and TensorFlow is running in a docker container with MKL-DNN, will it benefit from the CPU instruction sets or it will still lag in performance compared to a compile with relevant flags provided? – Atif Feb 14 '20 at 13:15
2

This warning can be ignored. The reason this is appearing is because of the instruction set flags set while building TF. However, MKL-DNN(The math lib responsible for performing core computations in tensorflow-mkl) at the run time will use the latest vector Instruction sets supported by your machine. If you look at the intel optimized tensorflow install guide, you can get more details. So bottomline is, although these warning messages appear, AVX2 is being used by MKL at the run time

Preethi Venkatesh
  • 61
  • 2