How to merge efficiently gigantic files with C#

So "merging" is really just writing the files one after the other? That's pretty straightforward - just open one output stream, and then repeatedly open an input stream, copy the data, close. For example:

static void ConcatenateFiles(string outputFile, params string[] inputFiles)
{
    using (Stream output = File.OpenWrite(outputFile))
    {
        foreach (string inputFile in inputFiles)
        {
            using (Stream input = File.OpenRead(inputFile))
            {
                input.CopyTo(output);
            }
        }
    }
}

That's using the Stream.CopyTo method which is new in .NET 4. If you're not using .NET 4, another helper method would come in handy:

private static void CopyStream(Stream input, Stream output)
{
    byte[] buffer = new byte[8192];
    int bytesRead;
    while ((bytesRead = input.Read(buffer, 0, buffer.Length)) > 0)
    {
        output.Write(buffer, 0, bytesRead);
    }
}

There's nothing that I'm aware of that is more efficient than this... but importantly, this won't take up much memory on your system at all. It's not like it's repeatedly reading the whole file into memory then writing it all out again.

EDIT: As pointed out in the comments, there are ways you can fiddle with file options to potentially make it slightly more efficient in terms of what the file system does with the data. But fundamentally you're going to be reading the data and writing it, a buffer at a time, either way.

Do it from the command line:

copy 1.txt+2.txt+3.txt combined.txt

or

copy *.txt combined.txt

Do you mean with merge that you want to decide with some custom logic what lines go where? Or do you mean that you mainly want to concatenate the files into one big one?

In the case of the latter, it is possible that you don't need to do this programmatically at all, just generate one batch file with this (/b is for binary, remove if not needed):

copy /b "file 1.tsv" + "file 2.tsv" "destination file.tsv"

Using C#, I'd take the following approach. Write a simple function that copies two streams:

void CopyStreamToStream(Stream dest, Stream src)
{
    int bytesRead;

    // experiment with the best buffer size, often 65536 is very performant
    byte[] buffer = new byte[GOOD_BUFFER_SIZE];

    // copy everything
    while((bytesRead = src.Read(buffer, 0, buffer.Length)) > 0)
    {
        dest.Write(buffer, 0, bytesRead);
    }
}

// then use as follows (do in a loop, don't forget to use using-blocks)
CopStreamtoStream(yourOutputStream, yourInputStream);