Why does Shift+Insert paste from CLIPBOARD in some applications and PRIMARY in others?
All the apps you've mentioned are gtk+ apps so it's quite easy to answer Why... Because in all gtk+ apps (except one), Shift+Insert pastes from CLIPBOARD - i.e. it's equivalent to Ctrl+V. The shortcut is hardcoded in gtkentry.c (line 2022) and gtktextview.c (line 1506):
gtk_binding_entry_add_signal (binding_set, GDK_KEY_Insert, GDK_SHIFT_MASK,
"paste-clipboard", 0);
It is also documented in the GTK+ 3 Reference Manual under GtkEntry:
The “paste-clipboard” signal
void
user_function (GtkEntry *entry,
gpointer user_data)
The ::paste-clipboard signal is a keybinding signal which gets emitted
to paste the contents of the clipboard into the text view.
The default bindings for this signal are Ctrl-v and Shift-Insert.
As far as I know this was done for consistency with other DE's (see KDE's Qt key bindings in QTextEdit Class) and Windows OS1.
The only exception is gnome-terminal. After long debates, the devs have decided (for consistency with other terminals) that, in gnome-terminal, Shift+Insert should paste from PRIMARY and Ctrl+Shift+V should paste from CLIPBOARD (although you have the options to customize some shortcuts).
As to How do you paste selection with a keyboard shortcut... there's no straightforward way.
The easiest way is to assign a shortcut to a script that runs xdotool click 2 (simulates clicking the middle-mouse button). While this works (and it should work with all or most DE's and toolkits), it only works if the mouse cursor is actually over the text entry box, otherwise it fails.
Another relatively easy way is via Gnome Accessibility, if it's available on your system. It also requires the presence of a numpad. Go to Universal Access >> Pointing & Clicking and enable Mouse Keys. Make sure NumLock is off. You can then use the numpad keys to move the cursor and click. To simulate a middle-mouse button click, press (and release) * (asterisk) then press 5 (here's a short guide). This solution seems to always work in a gtk+ environment. The downside is that it requires Gnome Accessibility and a numpad. Also, you cannot customize the shortcut.
An interesting solution was proposed on gnome-bugzilla (bug 643391). (Update 2018: issue has now been moved here.) It requires patching some source files and setting configuration options in ~/.config/gtk-3.0/gtk.css (or ~/.gtkrc-2.0 for gtk+ 2 apps). I haven't tried it personally but the feedback is positive.
Ideally, you would patch the source files and define a "paste-selection" signal then bind Shift+Insert to "paste-selection" instead of "paste-clipboard". Andy's code (attached in the bug report linked above) could serve as a guide on how to do that. Even then, it would only affect gtk+ apps (I'm not a KDE/Qt guy so I have no idea how to alter Qt apps behavior).
1: (not to mention IBM's CUA)