Is it possible to run 64-bit applications on a 32-bit operating system, if the hardware is 64-bit?

32 bit Mac OS X can run 64 bit applications on 64 bit hardware.

The reason this works is because the OS X (Darwin) kernel actually runs in the compatibility sub mode of long mode.

Windows and Linux do not do this and enforce a cleaner cut between x86 and x64. With Windows or Linux you can either run the 64 bit kernel and 64 bit drivers (and run both 64 bit and 32 bit programs) or the 32 bit kernel and 32 bit drivers (and run only 32 bit and 16 bit programs).

Note that real mode 16 bit code does not work in long mode and hence Mac OS X would probably not be able to run 16 bit programs if they existed for Mac OS X and if Mac OS X runs in long mode (including compatibility mode). Windows also cannot run 16 bit real mode code in 64 bit mode.

VMware Fusion or Parallels running on 32 bit Mac OS X on 64 bit hardware do run 64 bit guests.

Each 32 bit process running on 32 bit Mac OS X on 64 bit hardware gets, I think, 4 GB of user virtual address space and 4 GB of kernel virtual address space. 64 bit processes get, I assume, 8 TB of user virtual address space and 4 GB of kernel virtual address space (because the kernel runs in 32 bit mode).

The Mac OS X kernel is (like most Mac OS X programs) a universal binary containing binary code for PowerPC, x86 (32 bit), and x64 (64 bit). Switching from one platform to another "simply" loads the appropriate part of the binary. The x86 code checks if long mode is available and switches to long mode if possible and runs in compatibility mode to achieve the desired result.

N.B.: While Mac OS X Snow Leopard does not run on PowerPC the kernel still contains PowerPC code, presumably because PowerPC binaries might make use of it. PowerPC binaries still run on x86 and x64 Macs.

If you have a processor that supports hardware virtualization (the newest Intels do) and the VM software to support it, it is possible to install a 64 bit OS in a VM when the host OS is 32 bit. You could then run the 64 bit app inside the VM. But that's about as close as I can see you getting.