How can I remove a horizontal line I inserted into my notebook?

You can select it just like any other cell, and then delete it.

On my system, it is a cell of the form:

Cell[" ", "Text",
 Editable -> False, Selectable -> False, 
 CellFrame -> {{0, 0}, {0, 3}}, ShowCellBracket -> False, 
 CellMargins -> {{0, 0}, {1, 1}}, 
 CellElementSpacings -> {"CellMinHeight" -> 1}, CellFrameMargins -> 0,
 CellFrameColor -> RGBColor[0, 0, 1], CellSize -> {Inherited, 5}
]

This can be verified by wrapping the Cell above with NotebookPut@*Notebook@*List.

In Mathematica 12, to remove a horizontal line from a cell, you select the cell (bracket on the RHS) and right click with the mouse and select Properties. Go to Cell Options/Cell Frame Options and remove CellFrame. It will be bolded.

v12.2 update.

If you have just inserted the line, then Undo will remove it.

If the line is at the same height as an edge of the right bracket of a cell, then select this cell, go to menu Insert > Horizontal Line > None or uncheck the box. This is the inverse procedure of creating the line.

I have a notebook with a line at the end, apparently not attached to any cell.

To remove it, I had to to Edit > Select All, then as before Insert > ...

Such a detached line can be created with Insert > Horizontal Line > Paste ... and it cannot be selected so it is bit hard to delete it.

You can see the two sorts of lines on the screenshot.