JIT compiler vs offline compilers

JIT has advantages, but I don't see it taking over completely. Conventional compilers can spend more time optimizing, while a JIT needs to strike a balance between too much optimization (taking more time than is saved by optimization) and too little (taking too much time in straight execution).

The obvious answer is to use each where it's superior. JITs can take advantage of run-time profiling more easily than conventional optimizers (although there are compilers that can take run-time profiles as input to guide optimization), and can generally afford to do more CPU-specific optimizations (again, lots of conventional compilers do this, but if you expect to run the executable on different systems they can't take full advantage of it). Conventional compilers can spend more time, and do it in different ways.

Therefore, the language system of the future will have good optimizing compilers that will emit executable code designed for use by good optimizing JIT compilers. (This is also, for many people, the language system of the present.) (The language system of the future will also support everything from modern Python/VB scripting to the ugliest high-speed number crunching.)

As with many things, this was foreshadowed by Lisp. Quite some time ago, some Lisp systems (can't really say many, there haven't been all that many Common Lisp implementations) interpreted Lisp functions by compiling them on the fly. Lisp S-expressions (what code is written in) are fairly straightforward descriptions of parse trees, so compilation could go pretty fast. In the meantime, an optimizing Lisp compiler could crunch the code where performance was really important ahead of time.

Yes, there certainly are such scenarios.

• JIT compilation can use runtime profiling to optimize specific cases based on measurement of the characteristics of what the code is actually doing at the moment, and can recompile "hot" code as necessary. That's not theoretical; Java's HotSpot actually does this.
• JITters can optimize for the specific CPU and memory configuration in use on the actual hardware where the program happens to be executing. For example, many .NET applications will run in either 32-bit or 64-bit code, depending upon where they are JITted. On 64 bit hardware they will use more registers, memory, and a better instruction set.
• Virtual method calls inside of a tight loop can be replaced with static calls based on runtime knowledge of the type of the reference.

I think there will be breakthroughs in the future. In particular, I think that the combination of JIT compilation and dynamic typing will be significantly improved. We are already seeing this in the JavaScript space with Chrome's V8 and TraceMonkey. I expect to see other improvements of similar magnitude in the not-too-distant future. This is important because even so-called "statically typed" languages tend to have a number of dynamic features.

Yes, JIT compilers can produce faster Machine Code optimized for the current environment. But practically VM programs are slower than Native programs because JITing itself consumes time (more Optimization == more time), and for many methods JITing them may consume more time than executing them. And that's why GAC is introduced in .NET

A side effect for JITing is large memory consumption. However that's not related to computation speed, it may slow down the whole program execution, because large memory consumption increases the probability that your code will be paged out to the secondary storage.

Excuse me for my bad English.