What is the correct way to caption a `tikzpicture`?

You need to put the tikzpicture in a figure; the following shows the caption fine:

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{arrows}
\usepackage{caption}

\begin{document}

\begin{figure}
\centering
\begin{tikzpicture}[object/.style={thin,double,<->}]
  \draw[object] (7cm,-1cm) -- (6cm,-2cm) node[midway,above] {a label};
\end{tikzpicture}
\caption{A caption.}
\end{figure}

\end{document}

I know this question is old and solved, but here an additional hint: You could try to use the package "capt-of" for generating captions for figures of any type.

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{arrows}
\usepackage{capt-of}


\begin{document}

\begin{tikzpicture}[object/.style={thin,double,<->}]
      \draw[object] (7cm,-1cm) -- (6cm,-2cm) node[midway,above] {a label};
\end{tikzpicture}
\captionof{figure}{A caption.}

\end{document}

If you find yourself using the [h!] option a lot then you need to consider whether you really want to be using a figure. A figure is meant to be able to float so that it can go into a position that looks nice instead of at the place where you reference it. If you know that you definitely want the picture to be where you reference it then consider simply including the picture inline.

Look at how it's done in some books. I use Knuth's The Art of Computer Programming for ideas on placement etc., the TeX book is probably better but I don't have it.