How to automatically put a [Go To Summary] | [Go Back] on each section?

The following example allows you to adapt the text to be displayed with each sectional unit and provides a one-way link back to the summary/top/ToC:

\documentclass{article}

\usepackage{xcolor}
\usepackage[colorlinks]{hyperref}

\newcommand{\goToSummaryText}{\hyperlink{summary}{\textcolor{green}{\small\mdseries [Go To Top]}}}

\makeatletter
\newcommand{\addGoToSummary}{\renewcommand{\Sectionformat}[2]{##1 \goToSummaryText}}
\newcommand{\removeGoToSummary}{\renewcommand{\Sectionformat}[2]{##1}}
\makeatother

\let\oldtableofcontents\tableofcontents
\renewcommand{\tableofcontents}{%
  \hypertarget{summary}% Insert internal document link
  \oldtableofcontents}

\begin{document}

\tableofcontents

\addGoToSummary% Add "Go To Summary" to each section

\section{Fermat's Principle}\label{sec:fermat}
Fermat's principle states that the path light takes from one point
to another is not necessarily the one with the smallest distance,
but rather the path which can be traversed in the shortest time.

\removeGoToSummary% Remove "Go To Summary" from each section

\section{Geometrical Optics}
Geometrical optics is an approximation for light propagation in
cases where the wavelength is very small compared to the
structures with which the light interacts. Snell's Law, describing
refraction, can be derived from Fermat's principle
(see section~\ref{sec:fermat}).

\end{document}

Combining @Werner answer, @touhami comment and with the answers:

1. Going "back" when using hyperref
2. How to change color for a block of texts?
3. What point (pt) font size are \Large etc.?
4. How to insert pipe symbol in (La)TeX?
5. Why abntex2 class is inserting a new line after the chapter title?
6. How can the go to summary be fixed so the \section[Some]{Some more} does not throw all these errors?
7. What is the equivalent to `\Sectionformat` on memoir class for `\Chapterformat`?
8. Why xapptocmd is reducing the vertical space between \partname and \parttile?

I added this:

\definecolor{ultramarine}{RGB}{0,32,96}

\newcommand{\goToSummaryText}{{%
    \small\mdseries
    \hyperlink{summary}{\textcolor{ultramarine}{$\leftleftarrows$}}
    {$|$}
    \Acrobatmenu{GoBack}{\textcolor{ultramarine}{$\leftarrow$}}
}}

This is a full example:

\documentclass{article}
\usepackage{xcolor}

\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}

\definecolor{link_color}{RGB}{26,13,178}
\usepackage[backref,colorlinks,linkcolor=link_color]{hyperref}

\definecolor{ultramarine}{RGB}{0,32,96}
\RequirePackage{xpatch}
\RequirePackage{amssymb}
\newcommand{\goToSummaryText}{{%
    \small\mdseries
    \hyperlink{summary}{\textcolor{ultramarine}{$\leftleftarrows$}}
    {$|$}
    \Acrobatmenu{GoBack}{\textcolor{ultramarine}{$\leftarrow$}}
}}
\makeatletter
    \newif\ifismemoirloaded\ismemoirloadedfalse
    \newif\ifisabntexloaded\isabntexloadedfalse
    \@ifclassloaded{memoir}{%
        \ismemoirloadedtrue%
    }{}
    \@ifclassloaded{abntex2}{%
        \isabntexloadedtrue%
    }{}
    \newcommand{\addGoToSummary}
    {%
        \@ifundefined{printparttitle}{\message{printparttitle patch for addGoToSummary could NOT
                    be applied because there is no printparttitle command available!^^J}}{%
            \let\oldAddGoToprintparttitle\printparttitle
            \xapptocmd{\printparttitle}{~\protect\goToSummaryText}{}{}
        }
        \@ifundefined{Sectionformat}{\message{Sectionformat patch for addGoToSummary could NOT
                    be applied because there is no Sectionformat command available!^^J}}{%
            \let\oldAddGoToSectionformat\Sectionformat
            \xapptocmd{\Sectionformat}{~\protect\goToSummaryText}{}{}
        }
        \ifismemoirloaded
            \ifisabntexloaded
                \let\oldAddGoToABNTEXchapterupperifneeded\ABNTEXchapterupperifneeded
                \xapptocmd{\ABNTEXchapterupperifneeded}{~\protect\goToSummaryText}{}{}
            \else
                \let\oldAddGoToprintchaptertitle\printchaptertitle
                \xapptocmd{\printchaptertitle}{~\protect\goToSummaryText}{}{}
            \fi
        \else
            \@ifundefined{Chapterformat}{\message{Chapterformat patch for addGoToSummary could NOT
                        be applied because there is no Chapterformat command available!^^J}}{%
                \let\oldAddGoToChapterformat\Chapterformat
                \xapptocmd{\Chapterformat}{~\protect\goToSummaryText}{}{}
            }
        \fi
    }
    \newcommand{\removeGoToSummary}
    {%
        \@ifundefined{oldAddGoToprintparttitle}{}{\let\printparttitle\oldAddGoToprintparttitle}
        \@ifundefined{oldAddGoToSectionformat}{}{\let\Sectionformat\oldAddGoToSectionformat}
        \ifismemoirloaded
            \ifisabntexloaded
                \@ifundefined{oldAddGoToABNTEXchapterupperifneeded}{}{\let\ABNTEXchapterupperifneeded\oldAddGoToABNTEXchapterupperifneeded}
            \else
                \@ifundefined{oldAddGoToprintchaptertitle}{}{\let\printchaptertitle\oldAddGoToprintchaptertitle}
            \fi
        \else
            \@ifundefined{oldAddGoToChapterformat}{}{\let\Chapterformat\oldAddGoToChapterformat}
        \fi
    }
\makeatother
\let\oldAddGoTotableofcontents\tableofcontents
% Insert internal document link
\renewcommand{\tableofcontents}{%
    \hypertarget{summary}%
    \oldAddGoTotableofcontents%
}

\begin{document}

\tableofcontents
\addGoToSummary

\section{Fermat’s Principle}

    \label{sec:fermat}
    Fermat’s principle states that the path light takes from one point
    to another is not necessarily the one with the smallest distance,
    but rather the path which can be traversed in the shortest time.


\section{Geometrical Optics}

    Geometrical optics is an approximation for light propagation in
    cases where the wavelength is very small compared to the
    structures with which the light interacts. Snell’s Law, describing
    refraction, can be derived from Fermat’s principle
    (see section~\ref{sec:fermat}).

\end{document}

Which produces this PDF: