how tensorflow inference in fp16 with model trained in fp32

Question

Is there any seamless way available with best fp16 performance being achieved in NV V100/P100? E.g. I've a model and implementation being trained in fp32. The App works perfectly. Now, I'd like to explore the experience of fp16. Is there any simple way to enable this.

i have very similar issue . I want to take my trained fp32 model and run inference with fp16 . Did you figure our or any idea how to do it ? — user179156, Nov 02 '18 at 05:57

7oud · Answer 1 · 2019-02-01T02:04:40.253

0

try this method, and I found inference with fp16 is faster on Pascal architecture GPU, can someone give an explanation?

edited Feb 01 '19 at 02:04

answered Jan 31 '19 at 09:21

7oud

75
1
8

how tensorflow inference in fp16 with model trained in fp32

1 Answers1