Is there a way to delete something that has been shelved?

TortoiseSVN's Unshelve does not delete (drop) the shelf for me, regardless of whether the unshelving yields conflicts or not. (I'm using TortoiseSVN 1.11.1, Build 28492)

Workaround: delete/drop the shelf using SVN directly, rather than via TortoiseSVN. In a console:

$ svn help x-shelf-drop
x-shelf-drop: Delete a shelf.
usage: x-shelf-drop SHELF [PATH ...]

  Delete the shelves named SHELF from the working copies containing PATH
  (default PATH is '.')

…

$ svn x-shelf-drop "Name of shelf"
deleted 'Name of shelf'

Docs: https://cwiki.apache.org/confluence/display/SVN/Shelving+in+Svn-trunk

This might not be an answer, but I think you will have to wait for TortoiseSVN to reach SVN version 1.11, and until then to continue to use your workaround.

New in version 1.11 is the command x-shelf-drop (shelf-drop) to delete a shelf, as listed in the Apache article of Shelving in Svn-trunk.

Current TortoiseSVN version is described as:

The current version 1.10.1 is linked against the Subversion library 1.10.2.