Questions tagged [half-precision-float]

half-precision 16-bit floating point

Most uses of 16-bit floating point are the binary16 aka half-precision floating point format, but other formats with different choices of exponent vs. significand bits are possible.

(However, related formats like Posit that have similar uses but a different binary format are not covered by this tag)

The tag wiki has links to more info, and lists other tags. (This tag was temporarily a synonym of , but should stay separate because half-precision is less widely implemented than float / binary32 and double / binary64.)


16-bit floating point has less precision (mantissa aka significand bits) and less range (exponent bits) than the widely used 32-bit single-precision IEEE754 binary32 float or 64-bit binary64 double. But it takes less space, reducing memory bandwidth requirements, and on some GPUs has better throughput.

It's fairly widely supported on GPUs, but on x86 CPUs at least, support is limited to conversion to/from float. (And only on CPUs that support AVX and the F16C extension, e.g. Intel starting with IvyBridge.)

If any CPU SIMD extension supported math on half-precision directly, it would have twice the elements per SIMD vector and thus twice the throughput of float for vectorizable tasks. But such support is not widespread in 2020 if it exists at all.

70 questions
3
votes
1 answer

Gathering half-float values using AVX

Using AVX/AVX2 intrinsics, I can gather sets of 8 values, either 1,2 or 4 byte integers, or 4 byte floats using: _mm256_i32gather_epi32() _mm256_i32gather_ps() But currently, I have a case where I am loading data that was generated on an nvidia GPU…
Bram
  • 7,440
  • 3
  • 52
  • 94
3
votes
0 answers

C++ support of _Float16

I want to do half-precision floating-point computations on Jetson AGX Xavier(with armv8.2 architecture) which supports half-precision floating-point natively. As _Float16 data type is an arithmetic data type for half-precision floating-point, I…
Sooora
  • 171
  • 9
3
votes
0 answers

math library for half-precision numbers

I want to use half-precision arithmetic on Cortex-A76. I installed clang compiler to use _Float16 data type (which is for 16-bit arithmetic purpose). I was wondering is there any library for half-precision mathematic functions? (like for…
Sooora
  • 171
  • 9
3
votes
0 answers

half-precision floating point numbers

I want to use half-precision floating-point numbers in c++ on an ARM processor. I want to use half-precision numbers for arithmetic purposes. I don't know how can I define half numbers in C++? Is there any data type for half numbers in C++? Thanks…
Sooora
  • 171
  • 9
3
votes
1 answer

tf.keras.layers.Conv2D() is not working with Keras backend set to float16

I am trying to train a CNN in Google Colab (Also tried with Tesla v100), with keras backend set to float16. tf.keras.backend.set_floatx('float16') but it throws an error while compiling the model with Conv2D. model =…
2
votes
0 answers

Reciprocal of fp16 in OpenCL

In my OpenCL kernel I use 16bit floating point values of type half from the cl_khr_fp16 extension. Although this gives me code that works well, I noticed with AMD's radeon developer tools that the reciprocal is computed in 32 bits (gpu target is…
Bram
  • 7,440
  • 3
  • 52
  • 94
2
votes
2 answers

Double vs Float vs _Float16 (Running Time)

I have a simple question in C language. I am implementing a half-precision software using _Float16 in C (My mac is based on ARM), but running time is not quite faster than single or double-precision software. I tested half, single, double with a…
2
votes
0 answers

TensorFlow mixed precision training: Conv2DBackpropFilter not using TensorCore

I am using the keras mixed precision API in order to fit my networks in GPU. Typically in my code this will look like this. A MWE would be: from tensorflow.keras.mixed_precision import experimental as mixed_precision use_mixed_precision = True if…
2
votes
1 answer

CUDA half float operations without explicit intrinsics

I am using CUDA 11.2 and I use the __half type to do operations on 16 bit floating point values. I am surprised that the nvcc compiler will not properly invoke fused multiply add instructions when I do: __half a,b,c; ... __half x = a * b +…
Bram
  • 7,440
  • 3
  • 52
  • 94
2
votes
1 answer

Casting __fp16 to float fails to link on Clang 9

I need to read a file containing floating point numbers stored in binary16 format and convert them to float. Based on https://releases.llvm.org/9.0.0/tools/clang/docs/LanguageExtensions.html#half-precision-floating-point, I read the data into…
Alexey Romanov
  • 167,066
  • 35
  • 309
  • 487
2
votes
0 answers

How to keep the accuacy when convert a pytorch model to TensorRT in FP16 mode

I want to use TensorRT FP16 mode to accelerate my pytorch model, but there is a loss of precision. My question is how can I use the mixed precision training of pytorch, to avoid the loss of accuracy when converting to a TensorRT FP16 model. I have…
Yao
  • 43
  • 3
2
votes
0 answers

Is there a way to test whether a target has native half-precision support in Clang?

Clang documentation says: _Float16 arithmetic will be performed using native half-precision support when available on the target (e.g. on ARMv8.2a); otherwise it will be performed at a higher precision (currently always float) and then truncated…
Alexey Romanov
  • 167,066
  • 35
  • 309
  • 487
2
votes
2 answers

How to get the value of kCVPixelFormatType_DepthFloat16 (half-point float)?

I am working with a project on iOS front depth camera with swift. The media type is kCVPixelFormatType_DepthFloat16, half-point float at 640*360 dimension with 30fps according to apple documentation. I was stuck in how to further retrieve and…
debiluz
  • 51
  • 8
2
votes
0 answers

fp16 support in the Object Detection API(tensorflow)

fp16 support in the Object Detection API [Feature request] · Issue #3706 · tensorflow/models https://github.com/tensorflow/models/issues/3706 I asked on the github issue: fp16 support in the Object Detection API (tensorflow), but none of them solved…
gzchen
  • 62
  • 5
2
votes
2 answers

how to do convolution with fp16(Eigen::half) on tensorflow

How can I use tensorflow to do convolution using fp16 on GPU? (the python api using __half or Eigen::half). I want to test a model with fp16 on tensorflow, but I got stucked. Actually, I found that fp16 convolution in tensorflow seems like casting…
Di Huang
  • 63
  • 8