Printing a handout of a Beamer presentation, the resulting pdf is shrunk!

This has nothing to do with TeX, just with your printing properties. The document generated by your above MWE has a size of 128 mm \times 192 mm, so it is much smaller then Din A4. If you print it without scaling you will end up with a small image in the centre of the paper:

If you let your printer do its job and either scale it (e.g. 140% seems to be a good start) or fit to paper, you will get the whole page filled:

---

Or, instead of fiddling around with the printer options, compile the pdf directly to a Din A4 version, i.e. not interfering with the pgf resizing by omitting on second screen:

\documentclass[handout]{beamer}
\usepackage{pgfpages}
\pgfpagesuselayout{2 on 1}[landscape,a4paper,border shrink=5mm]
\setbeameroption{show notes}


\setbeamercolor{background canvas}{bg=pink}

\mode<presentation> {
    \usetheme{Singapore}
}
\begin{document}


\begin{frame}[plain]
    \titlepage  
\note{
    Notes for the title page
}
\end{frame}


\section*{Outline}
\begin{frame}[plain]
\frametitle{Outline}
    \tableofcontents

\note{
    Notes for the table of contents
}
\end{frame}


\section{Frame 1}
\begin{frame}
\frametitle{Frame with notes 1}
    FIRST FRAME CONTENTS

\note{
    Notes for the first frame
}
\end{frame}


\section{Frame 2}
\begin{frame}
\frametitle{Frame with notes 2}
    SECOND FRAME CONTENTS
\note{
    Notes for the second frame
}
\end{frame}


\section{Frame 3}
\begin{frame}
    \frametitle{Frame with notes 3}
    SECOND FRAME CONTENTS
    \note{
        Notes for the third frame
    }
\end{frame}


\end{document}

The following solution is really ugly and I hope someone finds a nicer one. I set the papersize explicitly to A4 (the height is half of it, because it consists of two slides):

\documentclass[handout, xcolor={dvipsnames,table}]{beamer}
\usepackage{pgfpages}

% A4 Paper Size and some small additional margin
\paperheight=143.5mm %297mm / 2 - 5mm
\paperwidth=205mm % 210mm -5mm

% border shrink and a4paper doesn't seem to work like I wanted it to..
\pgfpagesuselayout{resize to}[a4paper,border shrink=5mm] % could also use letterpaper
\setbeameroption{show notes on second screen=bottom} % Beamer manual, section 19.3
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\setbeamercolor{background canvas}{bg=pink}

\setbeamertemplate{note page}[plain] % Beamer manual, section 19.1
\newlength{\parskipbackup}
\setlength{\parskipbackup}{\parskip}
\newlength{\parindentbackup}
\setlength{\parindentbackup}{\parindent}
\newcommand{\baselinestretchbackup}{\baselinestretch}

\usetemplatenote{

  \insertslideintonotes{0.3}%

  \rmfamily \scriptsize%
  \setlength{\parindent}{1em} \setlength{\parskip}{1ex}%
  \renewcommand{\baselinestretch}{1}%

  \insertnote%

  \setlength{\parskip}{\parskipbackup}%
  \setlength{\parindent}{\parindentbackup}%
  \renewcommand{\baselinestretch}{\baselinestretchbackup}%
}
\begin{document}

\begin{frame}
Title slide
\note{this is a note}
\end{frame}

\begin{frame}
This is the first slide
\note{this is another note}
\end{frame}

\end{document}

To change the borders just shrink the papersizes.