Set pipe capacity in Linux

writes over the default pipe capacity will still require waking up the downstream command

If your goal is not to wake up the downstream command too often, did you try using the -p option to buffer? It should cause buffer to hold writes until the buffer is filled to certain percentage. You might need the -s option as well to write large chunks.

Update: D'oh, the pipes between the commands still limit things. Maybe try using the following adapter program:

#define _GNU_SOURCE
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
int main(int argc, char** argv)
{
  fcntl(atoi(argv[1]), F_SETPIPE_SZ, atoi(argv[2]));
  execvp(argv[3],argv+3);
  while (1);
}

as in:

adapter 1 (BIGSIZE) cmd1 | cmd2

or even:

adapter 1 (BIGSIZE) cmd1 | adapter 1 (BIGSIZE) buffer [args] | cmd2

if cmd1 still makes small writes.


Based on DepressedDaniel and Stéphane Chazelas suggestions I settled on the closest thing to a oneliner I could find:

function hugepipe {
  perl -MFcntl -e 'fcntl(STDOUT, 1031, 1048576) or die $!; exec { $ARGV[0] } @ARGV or die $!' "$@"
}

This allows to do:

hugepipe <command> | <command>

and the pipe between the two commands is going to have the capacity specified via the fcntl in the perl script.