What exactly is LLVM?

LLVM (used to mean "Low Level Virtual Machine" but not anymore) is a compiler infrastructure, written in C++, which is designed for compile-time, link-time, run-time, and "idle-time" optimization of programs written in arbitrary programming languages. Originally implemented for C/C++, the language-independent design (and the success) of LLVM has since spawned a wide variety of front-ends, including Objective C, Fortran, Ada, Haskell, Java bytecode, Python, Ruby, ActionScript, GLSL, and others.

Read this for more explanation
Also check out Unladen Swallow

LLVM is a library that is used to construct, optimize and produce intermediate and/or binary machine code.

LLVM can be used as a compiler framework, where you provide the "front end" (parser and lexer) and the "back end" (code that converts LLVM's representation to actual machine code).

LLVM can also act as a JIT compiler - it has support for x86/x86_64 and PPC/PPC64 assembly generation with fast code optimizations aimed for compilation speed.

Unfortunately disabled since 2013, there was the ability to play with LLVM's machine code generated from C or C++ code at the demo page.

A good summary of LLVM is this:

At the frontend you have Perl, and many other high level languages. At the backend, you have the natives code that run directly on the machine.

At the centre is your intermediate code representation. If every high level language can be represented in this LLVM IR format, then analysis tools based on this IR can be easily reused - that is the basic rationale.