How to protect an OS X Server from an unauthorized physical connection?

Have you considered using Remote Desktop instead of VNC?

Have you considered locking it up? Put it in a box and lock it. Leave only enough holes for airflow(many small and/or thin ones) and the only big holes are there to permit power and network cables.