CPU Pinning & NUMA Topology

Introduction

CPU pinning maps vCPUs to specific physical CPU cores for better performance. Infinibay implements BaseCpuPinningStrategy (abstract base), BasicStrategy (NUMA-aware distribution), and HybridRandomStrategy (randomized load balancing).

System reads NUMA topology from /sys/devices/system/node/ and distributes vCPUs across nodes for local memory access.

Why CPU Pinning Matters

Performance: Reduces cache thrashing, improves cache locality, eliminates migration overhead. Typical improvement: 10-30% for CPU-intensive workloads.

NUMA: Local memory access 2-3x faster than remote. Avoids cross-socket QPI/UPI latency.

Guest OS: Proper CPU topology helps OS schedule threads correctly.

CPU Pinning Architecture

Strategy Pattern

File: backend/app/utils/VirtManager/CpuPinning/BasePinningStrategy.ts

• BaseCpuPinningStrategy: Abstract base class
• BasicStrategy: NUMA-aware proportional distribution
• HybridRandomStrategy: Randomized distribution

CpuPinningConfig Interface

Defines cputune (vcpupin mappings), cpu (mode, topology, cache, maxphysaddr, NUMA cells).

Base Class

Provides: Read NUMA topology from /sys/devices/system/node/, expand CPU lists ("0-3,5" → array), create config objects, add topology/cache/maxphysaddr. Abstract method: setCpuPinning(vcpuCount).

Host NUMA Topology Detection

getNumaTopology: Reads /sys/devices/system/node/, filters node directories, reads cpulist files, expands to arrays.

getHostNumaMemory: Reads /sys/devices/system/node/nodeX/meminfo, parses MemTotal, converts kB to MiB.

expandCpuList: Expands "0-3,5,7-9" → ["0","1","2","3","5","7","8","9"].

BasicStrategy - NUMA-Aware Pinning

File: backend/app/utils/VirtManager/CpuPinning/BasicStrategy.ts

NUMA-aware, proportional distribution. Falls back to round-robin when NUMA not available.

setCpuPinning: Gets topology, checks if NUMA optimization possible, calls optimizeNumaPinning() or optimizeRoundRobinPinning().

canOptimizeNumaPinning: Returns true if topology has at least one node with CPUs.

optimizeNumaPinning Method

Calculates proportional vCPU distribution across nodes, assigns vCPUs to physical CPUs (uses modulo for oversubscription), generates NUMA cells with memory, adds topology/cache/maxphysaddr.

optimizeRoundRobinPinning Method

Distributes vCPUs round-robin across all CPUs. Uses modulo for oversubscription. Creates single NUMA cell.

generateNumaCells Method

Validates VM memory ≤ host memory, calculates proportional vCPU/memory distribution, creates NUMA cells with vCPU ranges and memory allocation.

HybridRandomStrategy - Randomized Distribution

File: backend/app/utils/VirtManager/CpuPinning/HybridRandom.ts

Shuffles NUMA nodes and CPUs for load balancing. Better for multiple VMs.

setCpuPinning: Shuffles nodes (Fisher-Yates), shuffles CPUs within nodes, assigns vCPUs to shuffled CPUs, generates NUMA cells, adds topology/cache/maxphysaddr.

generateNumaCells: Sorts vCPUs, formats as compact ranges ("0-2,5,7-8"), calculates memory proportionally.

formatCpuRanges: Groups consecutive CPUs into ranges.

shuffleArray: Fisher-Yates shuffle for uniform distribution.

CPU Configuration

addCpuTopology: Default 1 socket/N cores. For 4+ vCPUs (divisible by 2): 2 sockets/N/2 cores. CPU mode: host-model.

addCpuCache: L3 cache, emulate mode.

addCpuMaxPhysAddr: 42 bits, emulate mode.

Helpers

getHostCpuInfo: Reads /proc/cpuinfo for model, vendor, feature flags.

createCpuPinningConfig: Creates config with vcpupin elements.

addNumaToConfig: Adds NUMA cells to config.

XMLGenerator Integration

setCpuPinningOptimization (xmlGenerator.ts): Default HybridRandomStrategy, gets vCPU count from XML, calls setCpuPinning(), applies cputune and cpu config to domain XML.

Examples

4 vCPUs, 2 nodes (BasicStrategy): vCPU 0-1 → node0, vCPU 2-3 → node1. 2 NUMA cells, 2 sockets/2 cores.

12 vCPUs, 1 node (oversubscription): vCPU 0-7 → CPU 0-7, vCPU 8-11 → CPU 0-3 (shared). 1 NUMA cell.

HybridRandomStrategy: Shuffles nodes/CPUs for randomized assignment.

Troubleshooting

Guest OS doesn't detect cores: Check topology in XML, NUMA cells match vCPU count. Use lscpu (Linux) or Task Manager (Windows).

Poor performance: Check NUMA locality (numastat -p <pid>), verify pinning (virsh vcpupin), avoid oversubscription.

NUMA not detected: Check /sys/devices/system/node/ exists, NUMA enabled in BIOS, kernel support (dmesg | grep NUMA).

Mermaid Diagrams

CPU Pinning Strategy Selection

flowchart TD
    Start([XMLGenerator.setCpuPinningOptimization]) --> CheckStrategy{Strategy<br/>provided?}
    CheckStrategy -->|Yes| UseProvided[Use provided strategy]
    CheckStrategy -->|No| CreateDefault[Create HybridRandomStrategy]
    UseProvided --> GetVCPU[Get vCPU count from domain XML]
    CreateDefault --> GetVCPU
    GetVCPU --> CallStrategy[Call strategy.setCpuPinning]
    CallStrategy --> ReadNUMA[Strategy: Read host NUMA topology]
    ReadNUMA --> CheckOptimize{Can optimize<br/>NUMA?}
    CheckOptimize -->|BasicStrategy + Yes| OptimizeNUMA[optimizeNumaPinning]
    CheckOptimize -->|BasicStrategy + No| RoundRobin[optimizeRoundRobinPinning]
    CheckOptimize -->|HybridRandomStrategy| ShuffleNUMA[Shuffle NUMA nodes]
    ShuffleNUMA --> ShuffleCPUs[Shuffle CPUs within nodes]
    ShuffleCPUs --> AssignShuffled[Assign vCPUs to shuffled CPUs]
    OptimizeNUMA --> GenCells[Generate NUMA cells]
    RoundRobin --> GenCells
    AssignShuffled --> GenCells
    GenCells --> AddTopology[Add CPU topology<br/>sockets, cores, threads]
    AddTopology --> AddCache[Add CPU cache<br/>L3, emulate]
    AddCache --> AddMaxPhys[Add maxphysaddr<br/>42 bits]
    AddMaxPhys --> ReturnConfig[Return CpuPinningConfig]
    ReturnConfig --> ApplyCputune[XMLGenerator: Apply cputune]
    ApplyCputune --> ApplyCPU[XMLGenerator: Apply cpu config]
    ApplyCPU --> LogConfig[XMLGenerator: Log configuration]
    LogConfig --> End([End])

NUMA-Aware Pinning Process

sequenceDiagram
    participant BasicStrategy
    participant FileSystem
    participant XMLGenerator

    BasicStrategy->>FileSystem: Read /sys/devices/system/node/
    FileSystem-->>BasicStrategy: node0, node1 directories
    BasicStrategy->>FileSystem: Read node0/cpulist
    FileSystem-->>BasicStrategy: "0-7,16-23"
    BasicStrategy->>FileSystem: Read node1/cpulist
    FileSystem-->>BasicStrategy: "8-15,24-31"
    BasicStrategy->>BasicStrategy: Expand CPU lists
    BasicStrategy->>BasicStrategy: Calculate vCPUs per node (proportional)
    BasicStrategy->>BasicStrategy: Assign vCPUs to physical CPUs
    BasicStrategy->>FileSystem: Read node0/meminfo
    FileSystem-->>BasicStrategy: MemTotal: 67108864 kB
    BasicStrategy->>FileSystem: Read node1/meminfo
    FileSystem-->>BasicStrategy: MemTotal: 67108864 kB
    BasicStrategy->>BasicStrategy: Calculate memory per NUMA cell
    BasicStrategy->>BasicStrategy: Generate NUMA cells
    BasicStrategy->>BasicStrategy: Create CpuPinningConfig
    BasicStrategy->>BasicStrategy: Add topology, cache, maxphysaddr
    BasicStrategy-->>XMLGenerator: Return CpuPinningConfig

NUMA Topology Structure

graph TD
    Host[Host System]
    Host --> Node0[NUMA Node 0]
    Host --> Node1[NUMA Node 1]
    Node0 --> CPUs0[CPUs 0-7,16-23]
    Node0 --> Mem0[Memory 64GB]
    Node1 --> CPUs1[CPUs 8-15,24-31]
    Node1 --> Mem1[Memory 64GB]

    Guest[Guest VM]
    Guest --> GCell0[Guest NUMA Cell 0]
    Guest --> GCell1[Guest NUMA Cell 1]
    GCell0 --> vCPUs0[vCPUs 0-3]
    GCell0 --> GMem0[Memory 4GB]
    GCell1 --> vCPUs1[vCPUs 4-7]
    GCell1 --> GMem1[Memory 4GB]

    vCPUs0 -.Pinned.-> CPUs0
    vCPUs1 -.Pinned.-> CPUs1
    GMem0 -.Allocated from.-> Mem0
    GMem1 -.Allocated from.-> Mem1