Close running version of program before installing update (Inno Setup)

In version 5.5.0 (Released on May 2012) Inno Setup added support for the Restart Manager API on Windows Vista and newer.

Quote from MSDN linked documentation (emphasis mine):

The primary reason software installation and updates require a system restart is that some of the files that are being updated are currently being used by a running application or service. Restart Manager enables all but the critical applications and services to be shut down and restarted. This frees the files that are in use and allows installation operations to complete. It can also eliminate or reduce the number of system restarts that are required to complete an installation or update.

The good thing is: you don't need to write custom code in the installer or your application to ask the user to close it, or close it automatically.

If you want your application to restart after the update is complete, you have to call the RegisterApplicationRestart function from your application first.

The default values for the new directives closes all the .exe, .dll and .chm files contained within the [Files] section of the installer.

The changes related to it are (from release notes):

• Added new [Setup] section directive: CloseApplications, which defaults to yes. If set to yes and Setup is not running silently, Setup will now pause on the Preparing to Install wizard page if it detects applications using files that need to be updated by the [Files] or [InstallDelete] section, showing the applications and asking the user if Setup should automatically close the applications and restart them after the installation has completed. If set to yes and Setup is running silently, Setup will always close and restart such applications, unless told not to via the command line (see below).
• Added new [Setup] section directive: CloseApplicationsFilter, which defaults to *.exe,*.dll,*.chm. Controls which files Setup will check for being in use. Setting this to *.* can provide better checking at the expense of speed.
• Added new [Setup] section directive: RestartApplications, which defaults to yes. Note: for Setup to be able to restart an application after the installation has completed, the application needs to be using the Windows RegisterApplicationRestart API function.
• Added new command line parameters supported by Setup: /NOCLOSEAPPLICATIONS and /NORESTARTAPPLICATIONS. These can be used to override the new CloseApplications and RestartApplications directives.
• Added new [Code] support function: RmSessionStarted.
• TWizardForm: Added new PreparingMemo property.

If the application has a Mutex, you can add an AppMutex value in your Inno Setup installer and it will display a message telling the user to stop the program. You might be able to find the Mutex (if it's got one) by using SysInternals Process Explorer and selecting the program / process and looking at the Handles (CTRL-H) in the Lower Pane.

Here's a link to the a KB article that mentions several methods:
http://www.vincenzo.net/isxkb/index.php?title=Detect_if_an_application_is_running

Alternatively, you might try this (UNTESTED) code in the InitializeSetup:

[Setup]
;If the application has  Mutex, uncomment the line below, comment the InitializeSetup function out, and use the AppMutex.
;AppMutex=MyApplicationMutex

[Code]
const
  WM_CLOSE = 16;

function InitializeSetup : Boolean;
var winHwnd: Longint;
    retVal : Boolean;
    strProg: string;
begin
  Result := True;
  try
    //Either use FindWindowByClassName. ClassName can be found with Spy++ included with Visual C++. 
    strProg := 'Notepad';
    winHwnd := FindWindowByClassName(strProg);
    //Or FindWindowByWindowName.  If using by Name, the name must be exact and is case sensitive.
    strProg := 'Untitled - Notepad';
    winHwnd := FindWindowByWindowName(strProg);
    Log('winHwnd: ' + IntToStr(winHwnd));
    if winHwnd <> 0 then
      Result := PostMessage(winHwnd,WM_CLOSE,0,0);
  except
  end;
end;