What is ECDHE-RSA?

ECDHE suites use elliptic curve diffie-hellman key exchange, where DHE suites use normal diffie-hellman. This exchange is signed with RSA, in the same way in both cases.

The main advantage of ECDHE is that it is significantly faster than DHE. This blog article talks a bit about the performance of ECDHE vs. DHE in the context of SSL.

To add a bit of information on what @CodesInChaos says:

When you use ECDHE instead of DHE, you may obtain the following advantages:

• Better performance. ECDHE is faster, for a given security level; @CodesInChaos points to an article which gives figures; see also this answer for why elliptic curve offer better performance.
• Smaller messages. An ECDH public key, with a 224-bit curve, will be encoded over 56 bytes, whereas a classical DH public key of similar strength must use a 2048-bit modulus and will use 256 bytes. Since there are two such message in a SSL handshake, ECDHE saves you about 400 bytes. That's not a lot, but it can make a difference in some contexts.
• Biodiversity. ECDH relies on the hardness of a mathematical problem which is distinct from the one used for classical DH. To some extent, classical DH can be viewed as a very specific sub-case of ECDH (computations modulo a prime are isomorphic to curve point addition in an anomalous curve) so we can handwave an argument about how ECDH is inherently at least as strong as DH (if ECDH is broken, so is DH). In practice, there are sub-exponential (i.e. faster) algorithms for solving discrete logarithm, which is why we must use a 2048-bit modulus instead of a 224-bit modulus for plain DH; while no such "faster" algorithm is known to break the elliptic-curve variant.
• Fashionability. Elliptic curves are cool. This is an important advantage, although it is customarily expressed with a more serious-looking and boring terminology ("compliance", "Approved algorithm"...).
• Cleansing. If you enforce ECDHE usage, you will automatically reject old implementations which do not know how to do ECDHE. If you use ECDHE for your Web server, you no longer have to worry about IE 6.0 or 7.0 ! The cryptographic arguments for ECDHE are a good excuse to kill off such dinosaurs which should have fossilized away long ago.

I see two questions

"where do the elliptic curves in ECDHE-RSA are exactly used?"

I see this is as "How are elliptic curves used when it's RSA in ECDHE_RSA?". Well, check RFC 4492, Section 2, ECDHE_RSA - it's pretty good. Essentially, the server certificate is an RSA certificate (i.e. with long term RSA keys) but during the TLS handshake it instead agrees a transient/temporary/Ephemeral (the E is DHE) EC public key with DH. So the long term authenticity is confirmed via the server cert's RSA signature but the transient keys are derived via ephemeral EC keys (which then generate the symmetric key)

What upsides has ECDHE-RSA over DHE-RSA?

This is already addressed in the other answers.