SSE intrinsics: Convert 32-bit floats to UNSIGNED 8-bit integers
There is no direct conversion from float to byte, _mm_cvtps_pi8
is a composite. _mm_cvtps_pi16
is also a composite, and in this case it's just doing some pointless stuff that you undo with the shuffle. They also return annoying __m64
's.
Anyway, we can convert to dwords (signed, but that doesn't matter), and then pack (unsigned) or shuffle them into bytes. _mm_shuffle_(e)pi8
generates a pshufb
, Core2 45nm and AMD processors aren't too fond of it and you have to get a mask from somewhere.
Either way you don't have to round to the nearest integer first, the convert will do that. At least, if you haven't messed with the rounding mode.
Using packs 1: (not tested) -- probably not useful, packusdw
already outputs unsigned words but then packuswb
wants signed words again. Kept around because it is referred to elsewhere.
cvtps2dq xmm0, xmm0
packusdw xmm0, xmm0 ; unsafe: saturates to a different range than packuswb accepts
packuswb xmm0, xmm0
movd somewhere, xmm0
Using different shuffles:
cvtps2dq xmm0, xmm0
packssdw xmm0, xmm0 ; correct: signed saturation on first step to feed packuswb
packuswb xmm0, xmm0
movd somewhere, xmm0
Using shuffle: (not tested)
cvtps2dq xmm0, xmm0
pshufb xmm0, [shufmask]
movd somewhere, xmm0
shufmask: db 0, 4, 8, 12, 80h, 80h, 80h, 80h, 80h, 80h, 80h, 80h, 80h, 80h, 80h, 80h
We can solve the unsigned clamping issue by doing the first stage of packing with signed saturation. [0-255]
fits in a signed 16-bit int, so values in that range will remain unclamped. Values outside that range will stay on the same side of it. Thus, the signed16 -> unsigned8 step will clamp them correctly.
;; SSE2: good for arrays of inputs
cvtps2dq xmm0, [rsi] ; 4 floats
cvtps2dq xmm1, [rsi+16] ; 4 more floats
packssdw xmm0, xmm1 ; 8 int16_t
cvtps2dq xmm1, [rsi+32]
cvtps2dq xmm2, [rsi+48]
packssdw xmm1, xmm2 ; 8 more int16_t
; signed because that's how packuswb treats its input
packuswb xmm0, xmm1 ; 16 uint8_t
movdqa [rdi], xmm0
This only requires SSE2, not SSE4.1 for packusdw
.
I assume this is the reason SSE2 only included signed pack from dword to word, but both signed and unsigned pack from word to byte. packuswd
is only useful if your final goal is uint16_t
, rather than further packing. (Since then you'd need to mask off the sign bit before feeding it to a further pack).
If you did use packusdw -> packuswb
, you'd get bogus results when the first step saturated to a uint16_t
> 0x7fff. packuswb
would interpret that as a negative int16_t
and saturate it to 0. packssdw
would saturate such inputs to 0x7fff
, the max int16_t
.
(If your 32-bit inputs are always <= 0x7fff, you can use either, but SSE4.1 packusdw
takes more instruction bytes than SSE2 packsswd
, and never runs faster.)
If your source values can't be negative, and you only have one vector of 4 floats, not many, you can use harold's pshufb
idea. If not, you need to clamp negative values to zero rather than truncate the by shuffling the low bytes into place.
Using
;; SSE4.1, good for a single vector. Use the PACK version above for arrays
cvtps2dq xmm0, xmm0
pmaxsd xmm0, zeroed-register
pshufb xmm0, [mask]
movd [somewhere], xmm0
may be slightly more efficient than using two pack
instructions, because pmax
can run on port 1 or 5 (Intel Haswell). cvtps2dq
is port 1 only, pshufb
and pack*
are port 5 only.