How to delete things when I'm using the storyboard?

You just select the label you want to delete, and hit the delete button on your keyboard. That will remove the object. It works for me.

I use the delete back button!

You can also edit the Document outline, which is just on the left of the storyboard editor. It may be hidden, but you can show it by clicking the small circle with a triangle inside it. Here is a screenshot to help you. If you select items there and hit delete. It will remove them whenever it is possible.