Editing binary streams containing '\x00' bytes

sed is a text utility. It works with text lines (sequences of non-NUL characters (not bytes) of limited length delimited by a newline character).

If you want to change the 2nd and 5th byte of a sequence of bytes, it won't work for several reasons:

  • sed works on text. If the input contains NUL characters, doesn't end in a newline character, has more than LINE_MAX bytes in between two newline characters, contains sequences of bytes that don't form valid characters, depending on the sed implementation, it won't work at all. (note that GNU sed doesn't have many of those limitations).
  • even if that binary input happens to form valid text, . matches characters, not bytes, so may match more than one byte.
  • because the sed code is run for every line of the input, that would change the second and fifth character of each line, not of the whole input.

To treat input as arbitrary arrays of bytes (without the NUL byte limitation, or length limitations), you may want to use perl instead:

 dd.... | perl -0777 -pe 'for $o (1, 4) {substr($_, $o, 1) = "|"}'

Example:

$ printf 'a\0b\0cd' |
>   perl -0777 -pe 'for $o (1, 4) {substr($_, $o, 1) = "|"}' |
>   od -Ax -tx1 -tc
000000  61  7c  62  00  7c  64
         a   |   b  \0   |   d
000006

Or you could use an intermediate text representation, like using vim's xxd helper:

dd... | xxd -p | sed '1s/../7c/2;1s/../7c/5' | xxd -p -r

xxd -p gives a hex dump with 60 characters per line by default. Above we're replacing the second and fifth 2-digit hex of the first line with 7c, the number for ASCII |.

Tags:

Binary

Sed