Drupal - What is the purpose of Views "Attachment", "Content pane" & "Context"?

The Attachment option is to create yet another view that you can attach before and/or after any of the displays on your current view.

An example of its use is if you enable the taxonomy views the taxonomy pages are overwritten by the view and if you were to add a couple of extra fields like images, description, etc to your taxonomy term you can the use an attachment to display it as the header of each taxonomy page.

The EVA module extend this functionality by allowing you to attach views to content types and other entities.

The other two I have never use, but:

Here is an explanation about "Content pane":

http://drupalize.me/videos/basics-views-content-panes

Content pane is usually used inside a panel (Panels module). You can think of it as a block, but usable inside Panels, than can get arguments, etc from the panel.