File.ReadAllLines or Stream Reader

The StreamReader read the file line by line, it will consume less memory. Whereas, File.ReadAllLines read all lines at once and store it into string[], it will consume more memory. And if that string[] is larger than int.maxvalue then that will produce memory overflow(limit of 32bit OS).

So, for bigger files StreamReader will be more efficient.

If you want to process each line of a text file without loading the entire file into memory, the best approach is like this:

foreach (var line in File.ReadLines("Filename"))
{
    // ...process line.
}

This avoids loading the entire file, and uses an existing .Net function to do so.

However, if for some reason you need to store all the strings in an array, you're best off just using File.ReadAllLines() - but if you are only using foreach to access the data in the array, then use File.ReadLines().

Microsoft uses a StreamReader in File.ReadAllLines:

    private static String[] InternalReadAllLines(String path, Encoding encoding)
    {
        Contract.Requires(path != null);
        Contract.Requires(encoding != null);
        Contract.Requires(path.Length != 0);

        String line;
        List<String> lines = new List<String>();

        using (StreamReader sr = new StreamReader(path, encoding))
            while ((line = sr.ReadLine()) != null)
                lines.Add(line);

        return lines.ToArray();
    }