perf-cache-line-false-sharing

Answer

The Problem: False Sharing on CPU Cache Lines

False sharing is a performance degradation that occurs when two or more threads running on different CPU cores modify different variables that reside on the same cache line. Despite the variables being logically independent, the cache coherence protocol (MESI) treats them as a single unit, causing unnecessary cache invalidations and memory traffic.

How it works:

1. CPU caches are organized into fixed-size cache lines (64 bytes on x86_64).
2. When core 0 increments A, it loads the cache line containing [A, B] into its Exclusive (E) state.
3. When core 1 increments B (on the same line), it must also load that line. MESI transitions the line to Shared (S) on both cores.
4. On write, core 1 must Invalidate (I) core 0's copy via a coherence message before going Modified (M).
5. Core 0 then invalidates core 1's copy when it writes next, and so on — ping-ponging the cache line back and forth.

This creates a M→I→M→I cycle on every write pair, costing hundreds of cycles per invalidation instead of a few cycles for a local cache hit.

The fix: Pad each variable to occupy its own cache line, so concurrent writes never force invalidations on each other's data.

Code (main.go)

package main

import (
    "fmt"
    "sync"
    "time"
)

const (
    cacheLineSize = 64
    iterations    = 10_000_000
)

// SharedCounters: A and B share a cache line (distance=8 bytes)
type SharedCounters struct {
    A int64
    B int64
}

// PaddedCounters: A and B are on separate cache lines (distance=64 bytes)
type PaddedCounters struct {
    A    int64
    _    [cacheLineSize - 8]byte // pad to fill cache line
    B    int64
    _    [cacheLineSize - 8]byte
}

func runFalseSharing() time.Duration {
    counters := &SharedCounters{}
    var wg sync.WaitGroup
    start := time.Now()

    wg.Add(2)
    go func() {
        defer wg.Done()
        for i := 0; i < iterations; i++ {
            counters.A++
        }
    }()
    go func() {
        defer wg.Done()
        for i := 0; i < iterations; i++ {
            counters.B++
        }
    }()
    wg.Wait()
    return time.Since(start)
}

func runNoFalseSharing() time.Duration {
    counters := &PaddedCounters{}
    var wg sync.WaitGroup
    start := time.Now()

    wg.Add(2)
    go func() {
        defer wg.Done()
        for i := 0; i < iterations; i++ {
            counters.A++
        }
    }()
    go func() {
        defer wg.Done()
        for i := 0; i < iterations; i++ {
            counters.B++
        }
    }()
    wg.Wait()
    return time.Since(start)
}

func main() {
    fmt.Println("=== CPU Cache Line False Sharing Demo ===")
    fmt.Println("Arch: amd64 (cache line: 64 bytes)")
    fmt.Printf("Iterations per goroutine: %d\n\n", iterations)

    // Warmup
    runFalseSharing()
    runNoFalseSharing()

    const runs = 5
    var fsTotal, nfsTotal time.Duration

    for i := 0; i < runs; i++ {
        fsTotal += runFalseSharing()
    }
    for i := 0; i < runs; i++ {
        nfsTotal += runNoFalseSharing()
    }

    fsAvg := fsTotal / runs
    nfsAvg := nfsTotal / runs
    ratio := float64(fsAvg) / float64(nfsAvg)

    fmt.Printf("False sharing (adjacent int64):  %v\n", fsAvg)
    fmt.Printf("No false sharing (padded int64):  %v\n", nfsAvg)
    fmt.Printf("False sharing overhead: %.2fx slower\n", ratio)
}

MESI Protocol Explanation

Coherence transactions during false sharing: 1. Core 0 reads A → line enters E (fetch from memory) 2. Core 1 reads B → line enters S on both cores (snoop → shared) 3. Core 1 writes B → sends BusRdX (read-for-ownership), invalidates core 0's copy → M on core 1 4. Core 0 writes A → sends BusRdX, invalidates core 1's copy → M on core 0 5. Repeat steps 3–4 for every write pair — each write costs a full cache-to-cache transfer (~100–200 cycles) instead of a local L1 hit (~4 cycles)

With padding: A and B are on different lines. Each core owns its line exclusively, writes hit L1 directly, and no coherence traffic is generated.