Non Floating Figures

Obviously, place a figure where I placed the table, and note that in the \captionof argument.

\documentclass{article}
\usepackage{caption}
\begin{document}
\begin{minipage}{\textwidth}
\centering
\begin{tabular}{|l|l|}
\hline
     hello & world\\
     hello & world\\
     hello & world\\
     hello & world\\
\hline
\end{tabular}\hspace{1em}
\parbox[c]{2in}{\captionof{table}{my table has this somewhat long caption}}
\end{minipage}
\end{document}

For figures, a \raisebox may be necessary for vertical positioning:

\documentclass{article}
\usepackage{caption}
\usepackage[demo]{graphicx}
\begin{document}
\begin{minipage}{\textwidth}
\centering
\raisebox{-.8in}{\includegraphics[width=2in,height=2in]{figure}}
\hspace{1em}
\parbox[c]{2in}{\captionof{figure}{my table has this somewhat long caption}}
\end{minipage}
\end{document}

You can use the floatand floatrow package: the first defines a H option for the placement that enforces the float placement "here" ([!h] meaning "here, as much as possible). With the second, you can define caption settings.

So, if you want all your figures to have a sideways caption (and not other floats), you could write in your preamble, if you want your caption to be, say, on the left side, at the top of the figure, something like:

\usepackage{float}
\usepackage{floatrow}
\floatsetup[figure]{capposition=beside, capbesideposition={left,top}}

In the document body, just write:

\begin{figure}[H]
  \centering
  \includegraphics[width=80mm]{image.png}
  \caption{This text is added at the side of the image as a figure description.}
\end{figure}

If you want sideways captions only for some floats, don't write \floatsetup… in the preamble, and write:

\thisfloatsetup{capposition=beside, capbesideposition={left,top}}
\begin{figure}[H]
   .............

Of course you have access to may other settings (fonts, width of cation, &c.)

To stop figures from floating outside the section, I recommend using the placeins package.

To put the caption to the side of the picture, use minipage inside \begin{figure} and \end{figure}. For example:

\usepackage[section,subsection,subsubsection]{placeins}
%...
%...
\FloatBarrier
    \begin{figure}[htbp!]
        \begin{center}
            \begin{minipage}[c]{.45\linewidth}
            \centering
            \caption{The caption of the picture on the right side}
            \label{fig:pictureonright}
            \end{minipage}%
            \begin{minipage}[c]{.45\linewidth}
            \centering
            \includegraphics[width=6cm]{figs/youpicture}
            \end{minipage}
        \end{center}
    \end{figure}

Hope it works for you!

PS: If you want to read more on working with graphics in LaTeX and PdfLaTeX, here is a recommended source: http://tug.ctan.org/tex-archive/info/epslatex/english/epslatex.pdf