Caps Lock delay
Learn to use Shift instead, that is the common way to enter a capital letter.
If you want to hit Caps Lock anyway to produce a capital letter, do note that there are reports that the delay is a known issue ((Launchpad Bug #1376903). The Arch Linux wiki describes a method to remove this delay which seems to work.
For your convenience, this script performs the manual steps documented at that wiki page:
#!/bin/sh
xkbcomp -xkb "$DISPLAY" - | sed 's#key <CAPS>.*#key <CAPS> {\
repeat=no,\
type[group1]="ALPHABETIC",\
symbols[group1]=[ Caps_Lock, Caps_Lock],\
actions[group1]=[ LockMods(modifiers=Lock),\
Private(type=3,data[0]=1,data[1]=3,data[2]=3)]\
};\
#' | xkbcomp -w 0 - "$DISPLAY"
This needs to be started as part of your desktop session (do not use Upstart for example). See this post for a method, do not forget to enable the execute bit of the script.