What is the recommended way to assign a value to a variable and retrieve it for later use?

Well, I know it's not what you want, but the standard route is either with a def or a newcommand. The problem that you've touched on is the global namespace. If you use newcommand it will tell you if the command is already set (as an error in compilation which will name the offending newcommand attempt), protecting you from these issues.

% Set your new variable. In this case it will be 
% called "\MyNewVariable" and the value will be "hello".
\newcommand{\MyNewVariable}{hello}

% Use to get the variable.
\MyNewVariable

If you could prepend them all with your name or some other mnemonic, then you're unlikely to get clashes. This mnemonic then behaves as your own (sort of) namespace.

Assuming that the first instance of your variable is set with a newcommand without raising an error, then a renewcommand can be used to alter that variable later.

Sounds like you're looking for a key-value system. May I suggest pgfkeys? Including Yiannis' idea of using properties for each variable, I'd do it like this:

\documentclass{article}
\usepackage{pgfkeys}

\newcommand{\setvalue}[1]{\pgfkeys{/variables/#1}}
\newcommand{\getvalue}[1]{\pgfkeysvalueof{/variables/#1}}
\newcommand{\declare}[1]{%
 \pgfkeys{
  /variables/#1.is family,
  /variables/#1.unknown/.style = {\pgfkeyscurrentpath/\pgfkeyscurrentname/.initial = ##1}
 }%
}

\declare{}

\begin{document}
 \setvalue{VARIABLE1 = foo foo bar}
 \getvalue{VARIABLE1}

 \declare{test/}
 \setvalue{test/property = 12}
 \getvalue{test/property}
\end{document}

Less than ten lines, even counting the ones that have only braces. The operation is very simple: pgfkeys stores variables as "files" in "directories"; I've decided that yours should be in the /variables directory, so not in the global namespace. (By the way, pgfkeys keys never conflict with normal macro names, so its "global namespace" is different from the macros namespace.)

The \setvalue macro just changes directory appropriately and then calls your assignment. The \getvalue macro retrieves the variable from the correct directory.

The only trick is that in pgfkeys, a key needs to be "known" before it is assigned, or else you have to call it as key/.initial = value. Since I don't want to force you to write that, I created a "handler" for unknown variables that just adds this piece of code behind the scenes.

You declare a variable with properties using \declare{variable/}, and then you can use variable/property as a variable name in \setvalue (you can also use variable/ as a default directory, so write

\setvalue{variable, property 1 = value 1, property 2 = value 2}

which is convenient). The \declare macro just sets up the unknown handler for the "directory" /variables/variable/ (which means that the cryptic line \declare{} at the beginning sets up the /variables/ directory itself).

I would rather define them using a sort of a Lisp way, or object orientated way.

In the minimal below, we use:

\setproperty{test}{aproperty}{12}
\getproperty{test}{aproperty}

Think of them as representing test.aproperty (we actually define them a test@paproperty), this way it is highly unlikely to conflict with any existing commands, other than possibly your own:

The minimal:

\documentclass{article}
\makeatletter
%  Properties a la Lisp.
\def\ece#1#2{\expandafter#1\csname#2\endcsname}%
% \setproperty{ATOM}{PROPNAME}{VALUE} defines the property PROPNAME on the
% ``atom'' ATOM to have VALUE.
% 
\def\setproperty#1#2#3{\ece\protected@edef{#1@p#2}{#3}}%
\def\setpropertyglobal#1#2#3{\ece\protected@xdef{#1@p#2}{#3}}%
%
% 
% \getproperty{ATOM}{PROPNAME} expands to the value of the property
% PROPNAME on ATOM, or to nothing (i.e., \empty), if the property isn't
% present.
% 
\def\getproperty#1#2{%
  \expandafter\ifx\csname#1@p#2\endcsname\relax
  % then \empty
  \else \csname#1@p#2\endcsname
  \fi
}%
%
\makeatother

\begin{document}
\setproperty{test}{aproperty}{12}
\getproperty{test}{aproperty}
\end{document}