How to pop (read&remove) a line of a file?

Whatever the prevailing wisdom is, having to read a whole file each time in order to cut off its last line is dumb, no matter if it's done in a single process/command or two or three.

Unlike sed -i, tail is smart enough not to read a file from the beginning in order to determine what its last line is; also, linux has a truncate(1) utility, which combined with tail allows you to "pop" the last line(s) of a huge file in O(1) time:

# usage popline file [line_count, 1 by default]
popline()(LC_CTYPE=C; l=`tail -n "${2:-1}" "$1"; echo t`; l=${l%t}; truncate -s "-${#l}" "$1"; printf %s "$l")

$ wc -l /tmp/foo
3579500 /tmp/foo
$ cp /tmp/foo /tmp/foo1 && time sed -i '$d' /tmp/foo1

real    0m1.077s
user    0m0.457s
sys     0m0.156s
$ cp /tmp/foo /tmp/foo1 && time popline /tmp/foo1
 */

real    0m0.052s
user    0m0.002s
sys     0m0.003s

The goal is to have a single command that outputs the last line of a file, and at the same time deletes that line from the original file.

sed -i -e '${w /dev/stdout' -e 'd;}' file

This would run the following sed script:

${
    w /dev/stdout
    d
}

This writes the last line to /dev/stdout and then deletes it. All other lines are written back into original file through the -i option.

The script on the command line has to be split in two as there is no way to otherwise delimit the output filename of the w command (other than inserting a literal newline).

With ed:

ed -s file <<END_ED
p
d
w
END_ED

ed opens the file file and places the cursor on the last line of the file. The first command prints that line to standard output, the second deletes it, and the last command writes the buffer back to the file. Using ed in this way may not be advisable on huge files.