c# Best Method to create a log file

I would recommend log4net.

You would need multiple log files. So multiple file appenders. Plus you can create the file appenders dynamically.

Sample Code:

using log4net;
using log4net.Appender;
using log4net.Layout;
using log4net.Repository.Hierarchy;

// Set the level for a named logger
public static void SetLevel(string loggerName, string levelName)
{
    ILog log = LogManager.GetLogger(loggerName);
    Logger l = (Logger)log.Logger;

    l.Level = l.Hierarchy.LevelMap[levelName];
    }

// Add an appender to a logger
public static void AddAppender(string loggerName, IAppender appender)
{
    ILog log = LogManager.GetLogger(loggerName);
    Logger l = (Logger)log.Logger;

    l.AddAppender(appender);
}

// Create a new file appender
public static IAppender CreateFileAppender(string name, string fileName)
{
    FileAppender appender = new
        FileAppender();
    appender.Name = name;
    appender.File = fileName;
    appender.AppendToFile = true;

    PatternLayout layout = new PatternLayout();
    layout.ConversionPattern = "%d [%t] %-5p %c [%x] - %m%n";
    layout.ActivateOptions();

    appender.Layout = layout;
    appender.ActivateOptions();

    return appender;
}

// In order to set the level for a logger and add an appender reference you
// can then use the following calls:
SetLevel("Log4net.MainForm", "ALL");
AddAppender("Log4net.MainForm", CreateFileAppender("appenderName", "fileName.log"));

// repeat as desired

Sources/Good links:

Log4Net: Programmatically specify multiple loggers (with multiple file appenders)

Adding appenders programmatically

How to configure log4net programmatically from scratch (no config)

Plus the log4net also allows to write into event log as well. Everything is configuration based, and the configuration can be loaded dynamically from xml at runtime as well.

Edit 2:

One way to switch log files on the fly: Log4Net configuration file supports environment variables:

Environment.SetEnvironmentVariable("log4netFileName", "MyApp.log");

and in the log4net config:

<param name="File" value="${log4netFileName}".log/>

I would not use third party libraries, I would log to an xml file.

This is a code sample that do logging to a xml file from different threads:

private static readonly object Locker = new object();
private static XmlDocument _doc = new XmlDocument();

static void Main(string[] args)
{
    if (File.Exists("logs.txt"))
        _doc.Load("logs.txt");
    else
    {
        var root = _doc.CreateElement("hosts");
        _doc.AppendChild(root);
    }

    for (int i = 0; i < 100; i++)
    {
        new Thread(new ThreadStart(DoSomeWork)).Start();
    }
}

static void DoSomeWork()
{
    /*
     * Here you will build log messages
     */
    Log("192.168.1.15", "alive");
}

static void Log(string hostname, string state)
{
    lock (Locker)
    {
        var el = (XmlElement)_doc.DocumentElement.AppendChild(_doc.CreateElement("host"));
        el.SetAttribute("Hostname", hostname);
        el.AppendChild(_doc.CreateElement("State")).InnerText = state;
        _doc.Save("logs.txt");
    }
}

You might want to use the Event Log ! Here's how to access it from C# http://support.microsoft.com/kb/307024/en

But whatever is the method that you will use, I'd recommend to output to a file every time something is appended to the log rather than when your process exits, so you won't lose data in the case of crash or if your process is killed.