How HopX Achieves 100ms Cold Starts

When developers hear "virtual machine," they think of slow boot times—30 seconds to minutes. Containers are faster but still take seconds. HopX sandboxes start in under 100 milliseconds.

This isn't marketing spin. It's the result of carefully designed infrastructure that prioritizes startup latency above all else. Here's how we do it.

The Cold Start Problem

Cold start is the time between requesting a new execution environment and having it ready to run code. It's the enemy of responsive AI systems.

Traditional cold start times:

• Virtual Machines: 30-60 seconds
• Docker containers: 2-10 seconds
• AWS Lambda: 100ms - 5s (depending on runtime)
• Kubernetes pods: 5-30 seconds

For AI agents that need to spawn sandboxes dynamically, these numbers are unacceptable. An agent waiting 10 seconds to start code execution breaks the user experience.

Our Target: Sub-100ms

We set an aggressive target: sandboxes must be ready in under 100 milliseconds. That's the threshold where latency becomes imperceptible to humans.

Achieving this required rethinking every layer of the stack.

The Technology Stack

1. Firecracker Micro-VMs

At the core of HopX is Firecracker, the virtualization technology developed by AWS for Lambda and Fargate.

Why Firecracker?

• Minimal VMM (Virtual Machine Monitor) - only essential devices
• Boots a minimal Linux kernel in ~125ms
• Memory footprint of ~5MB per VM
• Full hardware virtualization (not containers)

Traditional VM:                  Firecracker Micro-VM:

┌─────────────────────┐         ┌─────────────────────┐

│    Guest OS         │         │   Minimal Guest     │

│    (Full kernel)    │         │   (Stripped kernel) │

├─────────────────────┤         ├─────────────────────┤

│    BIOS/UEFI        │         │   Minimal Boot      │

│    Device Models    │         │   (No BIOS)         │

├─────────────────────┤         ├─────────────────────┤

│    QEMU/KVM         │         │   Firecracker       │

│    (Complex VMM)    │         │   (Minimal VMM)     │

└─────────────────────┘         └─────────────────────┘

      ~30 seconds                    ~125ms

2. Memory Snapshots

Booting even a minimal kernel in 125ms isn't fast enough. We use memory snapshots to eliminate boot time entirely.

How snapshots work:

1. Boot a sandbox to a "ready" state
2. Capture complete memory state (snapshot)
3. Store snapshot on fast storage
4. Restore snapshot instead of booting

Cold boot path:           Snapshot restore path:

BIOS → Kernel →          Restore memory pages →

Init → Services →        Resume execution

Ready (~125ms)           (~15ms)

3. Copy-on-Write Memory

When restoring snapshots, we don't copy all memory upfront. We use copy-on-write (CoW) semantics:

1. Map snapshot pages as read-only
2. Only copy pages when written to
3. Most pages are never written

This means restore time is nearly instant—we're just setting up page table mappings.

# Conceptual representation

class SnapshotRestore:

    def restore(self, snapshot):

        # Map pages read-only (microseconds)

        for page in snapshot.pages:

            self.map_readonly(page)

        # Pages only copied when written (later)

        # Most pages never copied at all

4. Pre-warmed Pool

For the fastest possible starts, we maintain a pool of pre-restored sandboxes:

Request → [Pre-warmed Pool] → Sandbox Ready

             ↓

         ~10ms (just hand off)

The pool automatically scales based on demand patterns:

• More capacity during peak hours
• Fewer standby sandboxes during low usage
• Machine learning predicts demand spikes

5. Optimized Rootfs

Our root filesystem images are optimized for fast loading:

Traditional Linux rootfs:

• Full package manager
• Documentation
• Multiple locales
• Development headers
• Size: 500MB - 2GB

HopX rootfs:

• Runtime-only binaries
• Single locale (C.UTF-8)
• No documentation
• Stripped binaries
• Size: 50MB - 200MB

Smaller images mean:

• Faster snapshot loading
• Less memory pressure
• More sandboxes per host

6. Minimal Kernel Configuration

We use a custom Linux kernel configuration optimized for our use case:

Disabled:

- USB support

- Sound

- Bluetooth

- Wireless

- Most filesystems (keep ext4)

- Unnecessary drivers

Enabled:

- virtio (fast virtual devices)

- KVM guest support

- Minimal TTY

- Network (virtio-net)

- Block devices (virtio-blk)

Result: Kernel boots faster, uses less memory, has smaller attack surface.

The Complete Boot Path

Here's what happens when you call Sandbox.create():

T+0ms:    API receives request

T+2ms:    Auth and rate limiting

T+5ms:    Select host with capacity

T+8ms:    Check pre-warmed pool

          ↓

          If available:

T+10ms:     Claim sandbox from pool

T+12ms:     Configure networking

T+15ms:     Return sandbox handle

          ↓

          If pool empty:

T+10ms:     Start snapshot restore

T+25ms:     Memory mapping complete

T+30ms:     Resume VM execution

T+35ms:     Configure networking

T+40ms:     Start user process

T+50ms:     Ready for commands

Total: 15-50ms depending on pool availability.

Benchmarks

We continuously measure cold start performance:

Compare to alternatives:

Optimizations We Tried (And Rejected)

Not every optimization makes sense. Here's what we tried but didn't adopt:

WASM Instead of VMs

WebAssembly sandboxes start faster (~1ms) but:

• Limited to WASM-compiled code
• No system calls
• Can't run arbitrary Python/Node
• Weaker isolation than hardware virtualization

We chose VMs for flexibility and security.

Container Pooling

Pre-creating containers seemed promising but:

• Security isolation weaker than VMs
• Container escape vulnerabilities exist
• Shared kernel attack surface
• Not suitable for untrusted code

Unikernels

Single-purpose OS images boot incredibly fast but:

• Requires recompiling applications
• No standard tooling
• Debugging is difficult
• Not practical for general use

Real-World Impact

Fast cold starts enable use cases that weren't possible before:

AI Agents

An agent can spawn sandboxes mid-conversation without noticeable delay:

# User asks: "Calculate the Fibonacci sequence"

# Agent decides to run code

sandbox = Sandbox.create(template="code-interpreter")  # <50ms

result = sandbox.commands.run("python fib.py")  # Code runs

# Total latency: imperceptible

# Agent responds with results

Interactive Development

Code execution feels instant, like running locally:

# Each cell execution creates fresh sandbox

for cell in notebook_cells:

    sandbox = Sandbox.create()  # Fast enough for interactive use

    output = sandbox.commands.run(cell.code)

    display(output)

    sandbox.kill()

Parallel Processing

Spawn hundreds of sandboxes without waiting:

import asyncio

from hopx import Sandbox

async def process_item(item):

    sandbox = await Sandbox.create_async()  # Non-blocking

    result = await sandbox.commands.run_async(f"process {item}")

    await sandbox.kill_async()

    return result

# Process 100 items in parallel

items = range(100)

results = await asyncio.gather(*[process_item(i) for i in items])

# All 100 sandboxes started within ~500ms total

Future Improvements

We're continuously working on reducing latency further:

Speculative Execution

Predict sandbox needs before requests arrive:

• Analyze request patterns
• Pre-warm specific templates
• Geographic pre-positioning

Even Smaller Snapshots

Reduce snapshot size through:

• Memory deduplication
• Compression
• Differential snapshots

Edge Deployment

Place sandboxes closer to users:

• Edge locations worldwide
• Sub-10ms network latency
• Local snapshot caches

Conclusion

Achieving sub-100ms cold starts required innovation at every layer:

1. Firecracker micro-VMs for minimal overhead
2. Memory snapshots to skip boot entirely
3. Copy-on-write restore for instant page mapping
4. Pre-warmed pools for immediate availability
5. Optimized rootfs for smaller images
6. Custom kernel for faster boots

The result: sandboxes that feel instant, enabling new categories of applications that require on-demand isolated execution.

When latency drops below human perception thresholds, the technology becomes invisible. That's our goal—making sandboxes so fast you forget they're not local processes.

Further Reading

• Firecracker Design
• VM Snapshotting
• The Cost of Virtualization
• HopX Architecture Documentation