Does tensorflow's tfprof output theoretical FLOPS?

Question

I used tfprof to profile a machine learning algorithm. This is sample output: ==================Model Analysis Report====================== node name | # float_ops _TFProfRoot (--/3163.86b flops) InceptionResnetV2/InceptionResnetV2/Mixed_6a/Branch_1/Conv2d_0b_3x3/convolution (173.41b/173.41b flops) InceptionResnetV2/InceptionResnetV2/Conv2d_4a_3x3/convolution (167.25b/167.25b flops)

Here, in '167.25b/167.25b flops', what does the second 167.25b denote? Is it theoretical flops?

score 1 · Answer 1 · answered Jan 05 '18 at 00:10

Yes, it is the theoretical flops. Ops can register statistics using the RegisterStatistics annotation.

Here is an example of one such registration:

@ops.RegisterStatistics("MatMul", "flops")
def _calc_mat_mul_flops(graph, node):
  """Calculates the compute resources needed for MatMul."""
  transpose_a = node.attr["transpose_a"].b
  a_shape = graph_util.tensor_shape_from_node_def_name(graph, node.input[0])
  a_shape.assert_is_fully_defined()
  if transpose_a:
    k = int(a_shape[0])
  else:
    k = int(a_shape[1])
  output_shape = graph_util.tensor_shape_from_node_def_name(graph, node.name)
  output_shape.assert_is_fully_defined()
  output_count = np.prod(output_shape.as_list())
  return ops.OpStats("flops", (k * output_count * 2))

Does tensorflow's tfprof output theoretical FLOPS?

1 Answers1