How to escape the null character in here-document?(bash and/or dash)
Here-documents in Bash and dash don't support this. You can't store a null in a variable, they are removed from command substitutions, you can't write one in literally, and you can't use ANSI-C quoting inside the here-document. Neither shell is null-friendly and they are generally treated as (C-style) string terminators if one does get in.
You have a few options: use a real file, use zsh, use process substitution, or use standard input.
You can do exactly what you want in zsh, which is much more null-friendly.
zsh% null=$(printf '\x00')
zsh% hexdump -C <<EOT
heredoc> a${null}b${null}
heredoc> EOT
00000000 61 00 62 00 0a |a.b..|
00000005
Note though that heredocs have an implicit terminating newline, which may not be desirable (it'll be an extra field for shuf after the final null).
For Bash, you can use process substitution almost equivalently to your heredoc in combination with printf or echo -e to create nulls inline:
bash$ hexdump -C < <(
printf 'item 1\x00item\n2\x00'
)
00000000 69 74 65 6d 20 31 00 69 74 65 6d 0a 32 00 |item 1.item.2.|
0000000e
This is not necessarily entirely equivalent to a here-document, because those are often secretly put into real files by the shell (which matters for seekability, among other things).
Since you probably want to suppress terminating newlines, you can't even use a heredoc internally within the commands there - it has to be printf/echo -ne if safe to get fine-grained control over the output.
You can't do process substitution in dash, but in any shell you could pipe in standard input from a subshell:
dash$ (
printf 'item 1\x00'
printf 'item\n2\x00'
) | hexdump -C
00000000 69 74 65 6d 20 31 00 69 74 65 6d 0a 32 00 |item 1.item.2.|
0000000e
shuf is happy to read from standard input by default, so that should work for your concrete use case as I understand it. If you have a more complex command, being on the right-hand side of a pipeline can introduce some confounding elements with scoping.
Finally, you could write your data into a real file using printf and use that instead of a here-document. That option has been covered in the other answer. You'll need to make sure you clean up the file afterwards, and may want to use mktemp or similar if available to create a safe filename if there are any live security concerns.
Thank you all. Let I post one answer base on you all and maybe best for me.
This script works nicely in bash and dash, no require real file or process substitution in bash, no require an extra slow external program call, even you don't need to worry about any escape problem in entities as %s in C printf, but you should still take care to string escape in your shell itself.
#!/bin/sh
printf '%s\0' "[tag1]
key1=value1
key2=value2
[/tag1]
" "[tag2]
key3=value3
key4=value4
[/tag2]
" | shuf --zero-terminated
#also see man printf(1)
For shuf only(not intent to general here-document alternative):
shuf --echo "[tag1]
key1=value1
key2=value2
[/tag1]" "[tag2]
key3=value3
key4=value4
[/tag2]"
I do not think you can do what you want in a heredoc. However it is trivial to do using echo as the following example shows:
$ cat demo
#!/bin/bash
echo -ne "one\0" > outfile
echo -ne "two\0" >> outfile
echo -ne "three\0" >> outfile
$ ./demo
$ od -a outfile
0000000 o n e nul t w o nul t h r e e nul
0000016
$