Fast way to split an int into bytes
If you need something very fast, you should look into writing a C extension (see this). You can do this with or without cython. If you write a lot of these, I definitely recommend looking into cython.
This type of things is exactly what Python is awesome for: you can write your speed critical components in C directly, and it will interact (almost) seamlessly with your python code.
If you're doing it a lot, the fastest approach is to create a specialized Struct
instance and pre-bind the pack
method:
# Done once
int_to_four_bytes = struct.Struct('<I').pack
# Done many times (you need to mask here, because your number is >32 bits)
y1, y2, y3, y4 = int_to_four_bytes(x & 0xFFFFFFFF)
Using struct.pack
directly would use a cached Struct
object after the first use, but you'd pay cache lookup costs to go from format string to cached Struct
every time, which is suboptimal. By creating and prebinding the pack
of a Struct
object (which is implemented in C in CPython), you bypass all Python byte code execution beyond the actual function call, and spend no time on cache lookups. On my machine, this runs in about 205 ns, vs. 267 ns for shift and mask (without reassigning x
).
An alternate approach (for more general, not struct
compatible sizes) is using int.to_bytes
; for example, in this case:
y1, y2, y3, y4 = (x & 0xFFFFFFFF).to_bytes(4, 'big')
which takes about the same amount of time as the manually shifting and masking approach (it took 268 ns per loop), but scales to larger numbers of bytes better.