How to set width of caption to width of figure?

You need to store the image in a savebox and then wrap the caption into a minipage with the width of the box.

\documentclass{article}
\usepackage{graphicx}
\newsavebox\mysavebox

\usepackage{lipsum} % for example text only
\begin{document}

\lipsum

\begin{figure}
  \centering
  \sbox\mysavebox{\includegraphics[height=5cm]{example-image}}%
  \usebox\mysavebox
  \par
  \begin{minipage}{\wd\mysavebox}
  \caption{My very long long long long long long long long long long long long long long long long long long long caption }
  \end{minipage}
\end{figure}

\lipsum

\end{document}

---

The adjustbox package simplifies this approach and also avoids "bad box" warnings if the image is larger then the usual text width.

\documentclass{article}
\usepackage{graphicx}
\usepackage{adjustbox}
\newlength\mylength

\usepackage{lipsum}
\begin{document}

\lipsum

\begin{figure}
\adjustimage{height=5cm, gstore width=\mylength, center}{example-image}
%alternative: \adjustbox{gstore width=\mylength,center}{\includegraphics[height=10cm]{example-image}}
\par% or empty line, needed to get caption below the image, not to the rigth
\adjustbox{minipage=\mylength,center}{\caption{My very long long long long long long long long long long long long long long long long long long long caption }}
\end{figure}

\lipsum

\end{document}

There are two other solutions: with the measuredfigure environment, from threeparttable, and the \ffigbox command from floatrow; use asoptional argument \ffigbox[\FBwidth]{image}{caption}. One advantage is that if you want the caption to be only a little wider than the figure width, you just have to change[\FBwidth] to, say, [1.25\FBwidth]:

\documentclass{article}
\usepackage{graphicx}
\usepackage{threeparttable, floatrow}
\newsavebox\mysavebox

\usepackage{lipsum} %
\begin{document}
%
\lipsum[2]
\begin{center}
\begin{measuredfigure} \centering
\includegraphics[height=5cm]{example-image}
 \caption{My very long long long long long long long long long long long long long long long long long long long caption }
\end{measuredfigure}
\end{center}

\lipsum[3]
\begin{figure} \centering
\ffigbox[1.25\FBwidth]%
{\includegraphics[height=5cm]{example-image}}
 { \caption{My very long long long long long long long long long long long long long long long long long long long caption }}
\end{figure}

\end{document}