Why there is a delay between when a key is pressed and when it is accepted?

Because probably you put on the Slow Keys option from System Settings → Universal Access → Typing. Turn it off:

I had a similar problem. I stumbled on the answer while trying the solution here. Somehow the screen reader was turned on and was causing erratic behaviour. Check it under System Settings → Universal Access → Seeing. Make sure Screen Reader is set to OFF. Screen Reader OFF

Ubuntu has apparently mirrored the "slow keys" functionality in Windows, and by default set this up so it's enabled by holding down Shift key for too long automatically turns on this setting!

The appearance of this setting in recent years has been driving many people nuts, and the habit of holding down Shift while thinking of what to type means that it's easy to turn on this setting by accident.

The accepted answer does not work anymore in latest version of Ubuntu, there is no "Slow Keys" option anymore in gnome-control-center.

The only way I've found to reliably disable this is via command line:

# Get current value of keyboard accessibility (enabled = true , disabled = false)
gsettings get org.gnome.desktop.a11y.keyboard enable
# Get current value of slowkeys-enable
gsettings get org.gnome.desktop.a11y.keyboard slowkeys-enable
# Turn it off!
gsettings set org.gnome.desktop.a11y.keyboard slowkeys-enable false

# Get current value of stickykey-enable
gsettings get org.gnome.desktop.a11y.keyboard stickykeys-enable
# Turn it off!
gsettings set org.gnome.desktop.a11y.keyboard stickykeys-enable false
# Finally, turn the entire keyboard accessibility options off (this should disable the SHIFT key shortcuts)
gsettings get org.gnome.desktop.a11y.keyboard enable false