How to determine if a previous instance of my application is running?

Jeroen already answered this, but the best way by far is to use a Mutex... not by Process. Here's a fuller answer with code.

I've updated this answer after seeing some comments about a race condition to address that by instead using the Mutex Constructor

Boolean createdNew;
Mutex mutex;

try
{      
   mutex = new Mutex(false, "SINGLEINSTANCE" out createdNew);
   if (createdNew == false)
   {
      Console.WriteLine("Error : Only 1 instance of this application can run at a time");
      Application.Exit();
   }

   // Run your application
}
catch (Exception e)
{
    // Unable to open the mutex for various reasons
}
finally 
{
    // If this instance created the mutex, ensure that
    // it's cleaned up, otherwise we can't restart the
    // application
    if (mutex && createdNew) 
    {
        mutex.ReleaseMutex();
        mutex.Dispose();
    }
}

Notice the try{} finally{} block. If you're application crashes or exits cleanly but you don't release the Mutex then you may not be able to restart it again later.

The proper way to use a mutex for this purpose:

private static Mutex mutex;

static void Main()
{
    // STEP 1: Create and/or check mutex existence in a race-free way
    bool created;
    mutex = new Mutex(false, "YourAppName-{add-your-random-chars}", out created);
    if (!created)
    {
        MessageBox.Show("Another instance of this application is already running");
        return;
    }

    // STEP 2: Run whatever the app needs to do
    Application.Run(new Form1());

    // No need to release the mutex because it was never acquired
}

The above won't work for detecting if several users on the same machine are running the app under different user accounts. A similar case is where a process can run both under the service host and standalone. To make these work, create the mutex as follows:

        var sid = new SecurityIdentifier(WellKnownSidType.WorldSid, null);
        var mutexsecurity = new MutexSecurity();
        mutexsecurity.AddAccessRule(new MutexAccessRule(sid, MutexRights.FullControl, AccessControlType.Allow));
        mutexsecurity.AddAccessRule(new MutexAccessRule(sid, MutexRights.ChangePermissions, AccessControlType.Deny));
        mutexsecurity.AddAccessRule(new MutexAccessRule(sid, MutexRights.Delete, AccessControlType.Deny));
        _mutex = new Mutex(false, "Global\\YourAppName-{add-your-random-chars}", out created, mutexsecurity);

Two differences here - firstly, the mutex needs to be created with security rights that allow other user accounts to open/acquire it. Second, the name must be prefixed with "Global" in the case of services running under the service host (not sure about other users running locally on the same machine).