Skip to content

Instantly share code, notes, and snippets.

@louis030195

louis030195/hyper-v-sandbox-internals.md

Created December 6, 2025 22:16
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save louis030195/6d219d50eb9302ea91cdeefb34acecc2 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save louis030195/6d219d50eb9302ea91cdeefb34acecc2 to your computer and use it in GitHub Desktop.
Download ZIP
Windows Sandbox & Hyper-V Internals: Reverse Engineering Guide - HCS API, hypercalls, kernel drivers
Raw
hyper-v-sandbox-internals.md

Windows Sandbox & Hyper-V Internals: Reverse Engineering Guide

A comprehensive technical reference for understanding and reverse engineering Windows Sandbox, HCS APIs, and Hyper-V hypervisor internals.

Architecture Overview

┌─────────────────────────────────────────────────────────────────┐
│                         USER MODE                                │
├─────────────────────────────────────────────────────────────────┤
│  WindowsSandbox.exe → vmcompute.dll (HCS) → hns.dll (HNS)       │
│                              ↓                                   │
│                         vmwp.exe (VM Worker Process)             │
├─────────────────────────────────────────────────────────────────┤
│                        KERNEL MODE                               │
├─────────────────────────────────────────────────────────────────┤
│  vid.sys ←→ wcifs.sys ←→ bindflt.sys ←→ dxgkrnl.sys            │
│     │                                                            │
│     └──→ Hypercalls via vmcall/vmmcall                          │
├─────────────────────────────────────────────────────────────────┤
│                        HYPERVISOR                                │
├─────────────────────────────────────────────────────────────────┤
│           hvix64.exe (Intel) / hvax64.exe (AMD)                 │
├─────────────────────────────────────────────────────────────────┤
│                      GUEST (Sandbox)                             │
├─────────────────────────────────────────────────────────────────┤
│              vmbus.sys ←→ storvsp.sys                           │
└─────────────────────────────────────────────────────────────────┘

1. Host Compute Service (HCS) API

The HCS API (vmcompute.dll) is the primary interface for managing VMs and containers.

Key Files

C:\Windows\System32\vmcompute.dll      # Main HCS library
C:\Windows\System32\computecore.dll    # Core compute primitives
C:\Windows\System32\vmwp.exe           # VM Worker Process

API Categories

Compute System Operations

Function Call Code Description
HcsCreateComputeSystem - Create VM/container with JSON config
HcsOpenComputeSystem - Open existing system by ID
HcsStartComputeSystem - Launch execution
HcsShutdownComputeSystem - Graceful termination
HcsTerminateComputeSystem - Forced termination
HcsPauseComputeSystem - Suspend execution
HcsResumeComputeSystem - Resume from pause
HcsSaveComputeSystem - Checkpoint state
HcsGetComputeSystemProperties - Query system info
HcsModifyComputeSystem - Hot-add devices, change config

Process Operations

Function Description
HcsCreateProcess Spawn process inside compute system
HcsOpenProcess Open existing process by PID
HcsTerminateProcess End process execution
HcsSignalProcess Send signal (SIGTERM, etc.)
HcsGetProcessInfo Retrieve process metadata

Storage Layer Operations

Function Description
HcsImportLayer Import container layer
HcsExportLayer Export layer for registry upload
HcsDestroyLayer Delete container layer
HcsSetupBaseOSLayer Initialize base OS layer
HcsAttachLayerStorageFilter Mount wcifs filter

JSON Configuration Schema (v2.6)

{
  "Owner": "WindowsSandbox",
  "SchemaVersion": { "Major": 2, "Minor": 6 },
  "VirtualMachine": {
    "Chipset": {
      "Uefi": { "BootThis": { "DeviceType": "ScsiDrive" } }
    },
    "ComputeTopology": {
      "Processor": { "Count": 4 },
      "Memory": { "SizeInMB": 4096 }
    },
    "Devices": {
      "Scsi": {
        "primary": {
          "Attachments": [{
            "Type": "VirtualDisk",
            "Path": "C:\\path\\to\\disk.vhdx",
            "ReadOnly": false
          }]
        }
      },
      "NetworkAdapters": {
        "default": {
          "EndpointId": "guid",
          "MacAddress": "00-15-5D-xx-xx-xx"
        }
      },
      "VideoMonitor": {
        "HorizontalResolution": 1920,
        "VerticalResolution": 1080
      }
    },
    "GuestState": {
      "GuestStateFilePath": "sandbox.vmgs",
      "RuntimeStateFilePath": "sandbox.vmrs"
    }
  }
}

Reverse Engineering Approach

# 1. Export function list
dumpbin /exports C:\Windows\System32\vmcompute.dll > vmcompute_exports.txt

# 2. Download symbols
symchk C:\Windows\System32\vmcompute.dll /s srv*C:\symbols*https://msdl.microsoft.com/download/symbols

# 3. Trace API calls
# Use API Monitor, filter on vmcompute.dll and computecore.dll
# Launch Windows Sandbox and observe call sequence

Key discovery source: microsoft/hcsshim - Official Go bindings reveal JSON schemas and API patterns.

2. Bind Filter Driver (bindflt.sys)

Maps host folders into the sandbox with copy-on-write semantics.

API: BfSetupFilter

HRESULT BfSetupFilter(
    HANDLE JobHandle,                    // Optional job object
    UINT32 Flags,                        // See below
    LPCWSTR VirtualizationRootPath,      // Guest-visible path
    LPCWSTR VirtualizationTargetPath,    // Host source path
    LPCWSTR* ExceptionPaths,             // Paths to exclude
    ULONG ExceptionPathCount
);

Flags

Flag Value Description
BINDFLT_FLAG_READ_ONLY_MAPPING 0x00000001 Read-only bind mount
BINDFLT_FLAG_MERGED_BIND_MAPPING 0x00000002 Merge with existing
BINDFLT_FLAG_USE_CURRENT_SILO_MAPPING 0x00000004 Use current silo context

Reverse Engineering

# Find device objects
WinObj.exe  # Look under \Device\BindFlt

# Enumerate IOCTLs
IDA Pro / Ghidra → bindflt.sys → Find DeviceIoControl handlers

3. Windows Container Isolation FS (wcifs.sys)

Provides copy-on-write filesystem isolation.

Key Concepts

  • Layers: Read-only base layers + writable top layer
  • Reparse Points: Redirect reads to appropriate layer
  • Copy-up: Write to lower layer triggers copy to writable layer

IOCTL Discovery

// Common pattern in filter drivers
NTSTATUS WcifsDeviceControl(
    PDEVICE_OBJECT DeviceObject,
    PIRP Irp
) {
    switch (IoControlCode) {
        case IOCTL_WCIFS_CREATE_LAYER: ...
        case IOCTL_WCIFS_ATTACH_FILTER: ...
        case IOCTL_WCIFS_QUERY_LAYER: ...
    }
}

4. Virtual Infrastructure Driver (vid.sys)

Interface between user-mode HCS and the hypervisor.

Key IOCTLs (discovered via RE)

IOCTL Description
VID_IOCTL_CREATE_PARTITION Create Hyper-V partition
VID_IOCTL_DELETE_PARTITION Destroy partition
VID_IOCTL_CREATE_VP Create virtual processor
VID_IOCTL_MAP_GPA Map guest physical address
VID_IOCTL_RUN_VP Execute virtual processor

Debugging

# WinDbg kernel session
.symfix
.reload /f vid.sys
!devobj \Device\VID

5. Hypervisor Interface (Hypercalls)

Establishing the Hypercall Page

// 1. Check hypervisor presence
CPUID(1) → ECX bit 31 == 1

// 2. Read hypervisor info
CPUID(0x40000000) → Signature "Microsoft Hv"
CPUID(0x40000001) → Interface version

// 3. Configure hypercall page
wrmsr(HV_X64_MSR_GUEST_OS_ID, guest_id);      // 0x40000000
wrmsr(HV_X64_MSR_HYPERCALL, gpfn | 1);        // 0x40000001

Hypercall Input Value Format (64-bit)

┌────────────────────────────────────────────────────────────────┐
│ 63-60 │ 59-48      │ 47-44 │ 43-32     │ 31 │ 30-27 │ 26-17        │ 16   │ 15-0      │
├───────┼────────────┼───────┼───────────┼────┼───────┼──────────────┼──────┼───────────┤
│ Rsvd  │ Rep Start  │ Rsvd  │ Rep Count │ N  │ Rsvd  │ Var Hdr Size │ Fast │ Call Code │
└────────────────────────────────────────────────────────────────┘

N = Is Nested (handle by L0)
Fast = 1 for register-based, 0 for memory-based

Register Mapping (x64)

Register Memory-Based (Fast=0) Register-Based (Fast=1)
RCX Hypercall input value Hypercall input value
RDX Input parameters GPA Input parameter directly
R8 Output parameters GPA Output parameter directly
RAX Result (output) Result (output)

XMM Fast Hypercall (up to 112 bytes input)

RCX, RDX → Hypercall input value
R8, XMM0-XMM5 → Input parameter block

Check availability: CPUID(0x40000003) bit 4

Known Hypercalls

Code Name Description
0x0001 HvSwitchVirtualAddressSpace Switch address space
0x0002 HvFlushVirtualAddressSpace TLB flush
0x0003 HvFlushVirtualAddressList Partial TLB flush
0x0004 HvGetLogicalProcessorRunTime Get VP runtime
0x0008 HvNotifyLongSpinWait Spinlock hint
0x0040 HvCreatePartition Create partition
0x0041 HvInitializePartition Initialize partition
0x0042 HvFinalizePartition Finalize partition
0x0043 HvDeletePartition Delete partition
0x0044 HvGetPartitionProperty Query property
0x0045 HvSetPartitionProperty Set property
0x004E HvCreateVp Create virtual processor
0x004F HvDeleteVp Delete virtual processor
0x0050 HvGetVpRegisters Read VP registers
0x0051 HvSetVpRegisters Write VP registers
0x005C HvInstallIntercept Install exit handler

Status Codes

Code Name Description
0x0000 HV_STATUS_SUCCESS Success
0x0002 HV_STATUS_INVALID_HYPERCALL_CODE Unknown hypercall
0x0005 HV_STATUS_INVALID_PARAMETER Bad parameter
0x0006 HV_STATUS_ACCESS_DENIED Privilege violation
0x000B HV_STATUS_INSUFFICIENT_MEMORY OOM
0x000E HV_STATUS_INVALID_VP_INDEX Bad VP index

Hypercall Continuation

Hypervisor limits execution to 50μs max:

  • Partial completion returns with updated rep_start_index
  • RIP not advanced past hypercall instruction
  • Re-execution resumes from where it left off

6. Key Data Structures (from hvgdk.h)

Address Types

typedef UINT64 HV_SPA;   // System Physical Address
typedef UINT64 HV_GPA;   // Guest Physical Address
typedef UINT64 HV_GVA;   // Guest Virtual Address
typedef UINT64 HV_SPA_PAGE_NUMBER;
typedef UINT64 HV_GPA_PAGE_NUMBER;

Processor Context

typedef struct _HV_X64_CONTEXT {
    UINT64 Rax, Rcx, Rdx, Rbx;
    UINT64 Rsp, Rbp, Rsi, Rdi;
    UINT64 R8, R9, R10, R11, R12, R13, R14, R15;
    UINT64 Rip, Rflags;
    UINT64 Cr0, Cr2, Cr3, Cr4, Cr8;
    UINT64 Dr0, Dr1, Dr2, Dr3, Dr6, Dr7;
    // Segment registers, MSRs, APIC state...
} HV_X64_CONTEXT;

Partition Privileges

typedef union _HV_PARTITION_PRIVILEGE_MASK {
    struct {
        UINT64 AccessVpRunTimeReg : 1;
        UINT64 AccessPartitionReferenceCounter : 1;
        UINT64 AccessSynicRegs : 1;
        UINT64 AccessSyntheticTimerRegs : 1;
        UINT64 AccessIntrCtrlRegs : 1;
        UINT64 AccessHypercallMsrs : 1;
        UINT64 AccessVpIndex : 1;
        UINT64 AccessResetReg : 1;
        UINT64 AccessStatsReg : 1;
        UINT64 AccessPartitionReferenceTsc : 1;
        UINT64 AccessGuestIdleReg : 1;
        UINT64 AccessFrequencyRegs : 1;
        UINT64 AccessDebugRegs : 1;
        // ... more privileges
    };
    UINT64 AsUINT64;
} HV_PARTITION_PRIVILEGE_MASK;

7. Reverse Engineering Toolkit

Static Analysis

Tool Target Notes
IDA Pro All binaries Best for kernel drivers
Ghidra All binaries Free, good decompiler
x64dbg User-mode Live debugging vmwp.exe

Dynamic Analysis

Tool Target Notes
WinDbg Kernel + Hypervisor Requires 2-machine setup
API Monitor User-mode DLLs Trace HCS calls without attaching
Process Monitor I/O operations Filter by vmwp.exe
ETW All components Microsoft-Windows-Hyper-V-* providers

Symbol Acquisition

# Download all Hyper-V related symbols
$bins = @(
    "vmcompute.dll", "computecore.dll", "vmwp.exe",
    "vid.sys", "wcifs.sys", "bindflt.sys", "vmsif.sys"
)

foreach ($bin in $bins) {
    symchk "C:\Windows\System32\$bin" `
        /s srv*C:\symbols*https://msdl.microsoft.com/download/symbols
}

ETW Tracing

# Start Hyper-V trace
logman create trace HyperVTrace -p "Microsoft-Windows-Hyper-V-Worker" 0xFFFFFFFF 5 -o hyperv.etl
logman start HyperVTrace

# Do sandbox operations...

logman stop HyperVTrace
# Analyze with Windows Performance Analyzer

8. References

Official Documentation

Source Code & Tools

Research Papers & Talks

  • "Hyper-V Memory Internals" - Black Hat USA 2019
  • Alex Ionescu's Windows Internals talks
  • Microsoft Research papers on Drawbridge/Picoprocess

Quick Start: Trace a Sandbox Launch

# 1. Open API Monitor, set breakpoint on HcsCreateComputeSystem

# 2. Launch Windows Sandbox
WindowsSandbox.exe

# 3. Observe call sequence:
#    - HcsCreateComputeSystem (JSON config)
#    - HcsStartComputeSystem
#    - HcsCreateProcess (for shell)

# 4. Extract JSON config from HcsCreateComputeSystem call
#    This reveals complete VM configuration

Generated from research on HCS APIs, TLFS documentation, and community RE efforts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment