Can I click on a filename and line number in iterm2 to open that file in Visual Studio Code at the correct line number in OS X

It looks like you can just select "Open with editor..." in advanced settings > semantic history now

UPDATE - see the newer accepted answer by @csilk for more recent solution

It turns out you can, though it is not entirely obvious.

Go to Iterm2 Preferences > Profiles > (chosse the profile) > Advanced, then in the Semantic History section choose: Run Command, and enter the following:

/usr/local/bin/code -g \1:\2

If that appears to open in a new window then also try changing your Visual Studio Code settings by adding the following to your settings.json, press ⌘, and add in the right hand pane add:

"window.openFilesInNewWindow": false

EDIT - as mentioned in the comments by @BenjaminPasero adding -r to the command line will also force the file to be opened in the last active Code window, e.g.:

/usr/local/bin/code -r -g \1:\2