CD Notes

Just-In-Time Compilation

Complete guide to JIT compilation, tiered compilation, adaptive optimization, and runtime code generation

JIT compilation is a powerful technique that compiles code at runtime to improve performance. It combines the benefits of interpreted and compiled languages.

What is JIT Compilation?

JIT (Just-In-Time) compilation is the process of compiling code to native machine code at runtime, just before execution. This allows dynamic languages to achieve near-native performance while maintaining flexibility.

Without JIT (Pure Interpretation)

Source Code → Interpreter → Executes (slower)

With JIT

Source Code → Interpreter → Bytecode/IR → JIT → Native Code → Executes (faster)

(initial runs) (after profiling)

JIT Benefits and Tradeoffs

Benefits

Dynamic Language Support: Languages can modify behavior at runtime
Optimization Opportunities: Profile-guided optimization
Adaptive Performance: Optimize based on actual execution patterns
Reduced Binary Size: No need to store all code in machine form
Startup Performance: Bytecode available immediately
Polymorphism Optimization: Speculative optimization for virtual calls

Tradeoffs

Compilation Overhead: Runtime compilation takes CPU resources
Memory Overhead: Machine code cache takes memory
Unpredictability: Performance can vary based on profiling phase
Debugging Complexity: Harder to debug optimized code

JIT Compilation Phases

	Code executed line-by-line
	Profiling: track method calls
	Profiling: branch prediction
	Profiling: type information
	Duration: milliseconds to sec
	Hot method detected
	(called many times)
	Compilation threshold reached
	Trigger JIT compilation
	Profile-guided optimization
	Convert to native machine code
	Apply type specialization
	Inlining based on profiles
	Execute native machine code
	Much faster than interpretation

Tiered Compilation Strategy

Modern JVMs use multiple compilation tiers:

Tier Structure: Tier 3 (Full Optimization) ▲ │ Tier 2 (C2 Compiler) (Optimization) ▲ │ Tier 1 (C1 Compiler) (Quick Opt) ▲ │ Interpreter (Profile Only) │ Code Execution

Execution Flow:

Example

Code Starts ↓ Interpreted (profiling) ↓ (Hot method detected) Compile with Tier 1 (C1) - Quick compilation, some optimization ↓ (Still frequently called) Compile with Tier 2 (C2) - Aggressive optimization ↓ Optimized native execution

JIT Compilation Process

Hot Method Detected │ ├──────────────┐ │ │ ▼ ▼ C1 Compiler C2 Compiler (Quick) (Thorough) │ │ ├──────────────┤ │ ▼ ┌────────────────────────────────┐ │ Profiling Analysis │ │ - Call frequency │ │ - Branch probability │ │ - Type information │ │ - Null check patterns │ └────────────────────────────────┘ │ ▼ ┌────────────────────────────────┐ │ Optimizations Applied │ │ - Inlining │ │ - Dead code elimination │ │ - Escape analysis │ │ - Branch prediction │ │ - Speculative optimization │ └────────────────────────────────┘ │ ▼ ┌────────────────────────────────┐ │ Code Generation │ │ - Register allocation │ │ - Instruction scheduling │ │ - Assembly generation │ │ - Code cache management │ └────────────────────────────────┘ │ ▼ Native Machine Code (in code cache) │ ▼ Execution (fast!)

JIT Optimization Example

Java Code (Hot Method):

Example

public int processArray(int[] arr) { int sum = 0; for (int i = 0; i < arr.length; i++) { sum += arr[i]; } return sum; }

Initial Interpretation (slow):

Example

For each iteration: 1. Load arr reference 2. Load i 3. Call arr.length 4. Compare i < length 5. Load arr[i] 6. Add to sum 7. Increment i 8. Jump back (Many operations per iteration)

After JIT Compilation (optimized):

Optimizations applied

1. Bounds check elimination

(Proven safe after one check)

2. Loop unrolling

(Process 4 elements per iteration)

3. Inlining

(Method calls inlined)

4. Speculative optimization

(Assume no exceptions)

Result

Tight assembly loop, minimal overhead

4x-10x faster!

Deoptimization

JIT compilers also need to handle deoptimization:

Assumption: "Type is always int" │ ├─ Optimization made based on this │ ├─ Code executed many times (fast!) │ ├─ Then... UNEXPECTED INPUT: type is now Long │ ├─ Assumption violated! │ ▼ Deoptimization triggered │ ├─ Return to interpreter ├─ Discard optimized code ├─ Continue with interpreter (correct behavior) │ ├─ Recompile with better type information

Code Cache Management

The code cache stores compiled native code:

Code Cache (e.g., 256 MB) ┌──────────────────────────────┐ │ Compiled Method 1 │ │ (optimized native code) │ ├──────────────────────────────┤ │ Compiled Method 2 │ ├──────────────────────────────┤ │ Compiled Method 3 │ ├──────────────────────────────┤ │ ... │ ├──────────────────────────────┤ │ Unused Space │ │ (when cache is full, oldest │ │ code may be evicted) │ └──────────────────────────────┘

JIT in Different Languages

Language	JIT	Framework
Java	Yes	HotSpot JVM (C1, C2)
C#	Yes	.NET Runtime (Tier 0, 1, 2)
JavaScript	Yes	V8 (Ignition, TurboFan)
Python	Partial	PyPy (with JIT)
Go	No	Compiled directly
Rust	No	Compiled directly

Interview Q&A

Q: What is JIT compilation? A: JIT (Just-In-Time) compilation is compiling code to native machine code at runtime, just before execution. It combines interpretation flexibility with compiled performance.

Q: Why use JIT instead of ahead-of-time compilation? A: JIT allows languages to be dynamic (code modification, reflection) and enables profile-guided optimizations that static compilation cannot achieve. It provides both flexibility and performance.

Q: What is the warm-up phase? A: The warm-up phase is initial execution where code is interpreted and profiled. The JVM collects information about execution patterns before deciding to JIT compile.

Q: How does deoptimization work? A: Deoptimization occurs when assumptions made during JIT compilation (like type information) are violated at runtime. The JVM discards optimized code and returns to interpretation.

Q: What is code cache? A: Code cache is memory that stores compiled native code from JIT compilation. When it fills up, the oldest or least-used code may be evicted.

Exam Focus

Revise definitions, diagrams, examples, and short-answer points for Just-In-Time Compilation.

Interview Use

Prepare one clear explanation, one practical example, and one common mistake for this Compiler Design topic.

Search Terms

compiler-design, compiler design, compiler, design, modern, technologies, just, time

Continue learning this concept

Modern Compiler TechnologiesBytecode and Virtual MachinesComprehensive guide to bytecode design, virtual machine architecture, and performance optimization Modern Compiler TechnologiesGCC ArchitectureIn-depth exploration of GNU Compiler Collection structure, intermediate representations, and optimization pipeline Modern Compiler TechnologiesJava Compiler and BytecodeDeep dive into Java compilation process, bytecode format, and JVM architecture Modern Compiler TechnologiesLLVM: The Low-Level Virtual MachineComprehensive introduction to LLVM architecture, IR, optimizations, and how it powers modern compilers Modern Compiler TechnologiesPython Interpreter ArchitectureDeep exploration of Python compilation, bytecode, virtual machine, and execution model Modern Compiler TechnologiesWebAssembly BasicsComplete introduction to WebAssembly, bytecode format, execution model, and practical usage

PreviousJava Compiler and Bytecode NextLLVM: The Low-Level Virtual Machine

CD Notes

Just-In-Time Compilation

Complete guide to JIT compilation, tiered compilation, adaptive optimization, and runtime code generation

JIT compilation is a powerful technique that compiles code at runtime to improve performance. It combines the benefits of interpreted and compiled languages.

What is JIT Compilation?

Without JIT (Pure Interpretation)

Source Code → Interpreter → Executes (slower)

With JIT

Source Code → Interpreter → Bytecode/IR → JIT → Native Code → Executes (faster)

(initial runs) (after profiling)

JIT Benefits and Tradeoffs

Benefits

Dynamic Language Support: Languages can modify behavior at runtime
Optimization Opportunities: Profile-guided optimization
Adaptive Performance: Optimize based on actual execution patterns
Reduced Binary Size: No need to store all code in machine form
Startup Performance: Bytecode available immediately
Polymorphism Optimization: Speculative optimization for virtual calls

Tradeoffs

Compilation Overhead: Runtime compilation takes CPU resources
Memory Overhead: Machine code cache takes memory
Unpredictability: Performance can vary based on profiling phase
Debugging Complexity: Harder to debug optimized code

JIT Compilation Phases

	Code executed line-by-line
	Profiling: track method calls
	Profiling: branch prediction
	Profiling: type information
	Duration: milliseconds to sec
	Hot method detected
	(called many times)
	Compilation threshold reached
	Trigger JIT compilation
	Profile-guided optimization
	Convert to native machine code
	Apply type specialization
	Inlining based on profiles
	Execute native machine code
	Much faster than interpretation

Tiered Compilation Strategy

Modern JVMs use multiple compilation tiers:

Tier Structure: Tier 3 (Full Optimization) ▲ │ Tier 2 (C2 Compiler) (Optimization) ▲ │ Tier 1 (C1 Compiler) (Quick Opt) ▲ │ Interpreter (Profile Only) │ Code Execution

Execution Flow:

Example

JIT Compilation Process

JIT Optimization Example

Java Code (Hot Method):

Example

public int processArray(int[] arr) { int sum = 0; for (int i = 0; i < arr.length; i++) { sum += arr[i]; } return sum; }

Initial Interpretation (slow):

Example

For each iteration: 1. Load arr reference 2. Load i 3. Call arr.length 4. Compare i < length 5. Load arr[i] 6. Add to sum 7. Increment i 8. Jump back (Many operations per iteration)

After JIT Compilation (optimized):

Optimizations applied

1. Bounds check elimination

(Proven safe after one check)

2. Loop unrolling

(Process 4 elements per iteration)

3. Inlining

(Method calls inlined)

4. Speculative optimization

(Assume no exceptions)

Result

Tight assembly loop, minimal overhead

4x-10x faster!

Deoptimization

JIT compilers also need to handle deoptimization:

Code Cache Management

The code cache stores compiled native code:

JIT in Different Languages

Language	JIT	Framework
Java	Yes	HotSpot JVM (C1, C2)
C#	Yes	.NET Runtime (Tier 0, 1, 2)
JavaScript	Yes	V8 (Ignition, TurboFan)
Python	Partial	PyPy (with JIT)
Go	No	Compiled directly
Rust	No	Compiled directly

Interview Q&A

Q: What is code cache? A: Code cache is memory that stores compiled native code from JIT compilation. When it fills up, the oldest or least-used code may be evicted.

Exam Focus

Revise definitions, diagrams, examples, and short-answer points for Just-In-Time Compilation.

Interview Use

Prepare one clear explanation, one practical example, and one common mistake for this Compiler Design topic.

Search Terms

compiler-design, compiler design, compiler, design, modern, technologies, just, time

Continue learning this concept

PreviousJava Compiler and Bytecode NextLLVM: The Low-Level Virtual Machine