Break free of a fullscreen app on Mac OS X
First, try hitting esc. That'll make well behaved programs such as DVD Player or VLC exit Full Screen Mode.
No luck there, try command+tab and see if you can cycle to another open program or the Finder. ( OP tried this, but just adding for completion's sake )
Then try option+command+esc next, to try and Force Quit the application.
If none of that works...you could try to ssh into the machine and kill the offending process via Terminal..
That'd be all I try before rebooting the Mac...
Shift+Option+Command+Escape, when held down for three seconds, should force-kill the active application in Mac OS X, similar to ending an unresponsive task in windows.
To exit Mac OS X Mountain Lion application full screen mode:
- Move cursor to top of screen to display the main menu
- Click the blue icon now on the far right of the menu:
In OS X 8.2, an app in full screen mode will display the now hidden application menu by moving the cursor to the top of the screen. At the far right of the menu appears a blue background arrows version of the "full screen" icon (with the diagonal arrows) that you used to initiate the app to full screen mode. Select that to toggle out of full screen mode.