CAS with a standard language

You should definitely use SymPy. It was invented in large part for this reason, because Ondřej Čertík didn't like how CAS systems like Mathematica invented their own language, which you had to learn if you wanted to write non-trivial stuff using the system.

SymPy is written entirely in pure Python, and the general execution of is in some kind of Python interpreter (for example, the isympy script is a convenience script that calls "from sympy import *" and defines some common variable names, and it also uses IPython if you have it installed).

Because of this, it's very easy to write scripts that work with SymPy, because they would just be Python scripts, where you import SymPy and work with it. But it's also easy to extend with your own functions. By subclassing SymPy objects and defining methods on them, you can make them work with built-in SymPy functions.

(full disclosure: I am the lead developer of SymPy)

If you're a big fan of C numerous libraries exist for doing certain tasks. MPIR/GMP are two libraries for doing arbitrary precision arithmetic. On top of these you'll find more specialised libraries like MPC which adds support for the complex field.

Flint is a library that sits on top of MPIR, for example, providing an array of number theory functions. PARI/GP I believe provides a similar foundation but via the GMP library.

I'm sure if you googled "library for linear algebra" or "library for x" you would find it. However, a CAS unites all of this functionality.

Now we come to sage. From what I understand of the sage project, it is not so much a collection of open source libraries but an interface uniting them. For example, from the tutorial:

sage: a = 5   # a is an integer
sage: type(a)
<type 'sage.rings.integer.Integer'>
sage: a = 5/3  # now a is a rational number
sage: type(a)
<type 'sage.rings.rational.Rational'>
sage: a = 'hello'  # now a is a string
sage: type(a)
<type 'str'>

Is a sage session. As you're probably aware sage.rings.integer.Integer is not a built-in python type.

I would recommend giving sage a go. I think it is the closest thing out there to what you're after. I personally don't find Mathematica too bad.

If you don't want to install all of Sage (which is quite big), you could perhaps try one of its components, SymPy, on its own to be begin with.

If you use C++ maybe GiNaC is useful (I haven't tried it).