Go 1.27 SIMD Benchmarked — Lookup Drops From 76 ns to 14.5 ns, and the Portable Package Costs 2.3x
Two comparisons on one M5 — 5x between scalar code and SIMD, another 2.3x between archsimd and the portable SIMD package
GOLANG
Go 1.27 SIMD Benchmarked — Lookup Drops From 76 ns to 14.5 ns, and the Portable Package Costs 2.3x
Two comparisons on one M5–5x between scalar code and SIMD, another 2.3x between archsimd and the portable SIMD package
Go 1.26 shipped official SIMD support, but only for amd64. Go 1.27, due out shortly, is the first release in which arm64 can write SIMD directly: simd/archsimd now extends to arm64 and wasm, and a portable, vector-width-agnostic simd package arrives alongside it.
The portable one sits atop the architecture-specific one. On amd64 or arm64 it goes straight to native SIMD, and on architectures without hardware support it falls back to pure Go emulation.
To find out what that costs, I ran a lookup benchmark on 10 million int32 values on my own Mac laptop, with two comparisons in mind: how far ordinary code is from SIMD, and how far the two SIMD packages are from each other.
Test setup: Apple M5 / 10 cores / 16 GB, darwin/arm64, go1.27rc2, benchstat count=10 benchtime=400ms. Every implementation has the same semantics: lower_bound, returning the first element ≥ x.Without SIMD and With It, a 5x Gap
Ten million int32 (40 MB, far past the last-level cache), random queries:
implementation ns/op vs slices
slices.BinarySearch 76.10 1.00x
hand-written binary (CSEL) 49.77 1.53x
blocked layout + scalar rank 88.69 0.86x
blocked layout + portable simd 33.12 2.30x
blocked layout + arm64 Neon 14.55 5.23xThe third row uses exactly the same layout as the last one, except that the comparison is written as a scalar loop, and it runs slower than plain binary search. Vector instructions are what make this layout viable at all — without them it is a pessimization.
Shrink the data to one thousand elements (4 KB, small enough to sit in L1):
2 hand-written binary (branchless) 6.912 ± 2%
6 blocked layout + arm64 Neon 4.847 ± 3%Same SIMD code: 1.4x faster when the data fits in L1, and only at 40 MB does the gap widen to 3.4x against the best scalar implementation. So: what SIMD buys here is not arithmetic; it is memory traffic.
What the SIMD Version Looks Like
The layout goes like this: spread the sorted array out in blocks of 16 int32, in layers, where element i of a layer holds the maximum of block i in the layer below. A query does one rank per layer (count how many of those 16 values are less than x), and that tells you which block to descend into next.
Put the two access patterns side by side, and the difference is not in the arithmetic:
Every step of a binary search has to wait for the previous comparison before it knows the next address, and its 23 jumps scatter across 40 MB, hauling in 128 bytes each time, using only 4 of them. The blocked layout cuts that chain to 6 steps, and every one of the 64 bytes it pulls in takes part in a comparison.
The whole structure’s performance rests on rank. Sixteen comparisons, six layers, ninety-six comparisons in total.
The Neon version squeezes those ninety-six into twelve instructions. A vector compare produces a mask; converting that mask to an integer vector turns true into -1, so subtraction does the per-lane counting:
func rankNeon(node []int32, vx archsimd.Int32x4) int {
n := (*[B]int32)(node)
var acc archsimd.Int32x4
// Less yields a mask, ToInt32x4 turns true into -1, so acc.Sub adds one per lane
acc = acc.Sub(archsimd.LoadInt32x4Array((*[4]int32)(n[0:4])).Less(vx).ToInt32x4())
acc = acc.Sub(archsimd.LoadInt32x4Array((*[4]int32)(n[4:8])).Less(vx).ToInt32x4())
acc = acc.Sub(archsimd.LoadInt32x4Array((*[4]int32)(n[8:12])).Less(vx).ToInt32x4())
acc = acc.Sub(archsimd.LoadInt32x4Array((*[4]int32)(n[12:16])).Less(vx).ToInt32x4())
return int(acc.ReduceSum()) // ADDV, one instruction down to a scalar
}
That ToInt32x4 gives -1 rather than 1 is not spelled out in the docs. I printed it to be sure:
mask v<5: {1,1,0,0}
mask.ToInt32x4(): {-1,-1,0,0}The disassembly is textbook clean:
FMOVQ (R5), F2 ← four 128-bit loads
FMOVQ 16(R5), F3
FMOVQ 32(R5), F4
FMOVQ 48(R5), F5
VCMGT V2.S4, V0.S4, V2.S4 ← four vector compares
VSUB V2.S4, V1.S4, V2.S4 ← four accumulates
VCMGT V3.S4, V0.S4, V3.S4
VSUB V3.S4, V2.S4, V2.S4
VCMGT V4.S4, V0.S4, V3.S4
VSUB V3.S4, V2.S4, V2.S4
VCMGT V5.S4, V0.S4, V3.S4
VSUB V3.S4, V2.S4, V2.S4
VADDV V2.S4, V2 ← horizontal sum
VMOV V2.S[0], R5No spilled registers, nothing on the stack, and the whole function inlines into the search loop. Less compiles to VCMGT with the operands swapped — arm64 has no standalone "less than", so the compiler flips it around.
That last VADDV folds four lanes into a single scalar. The portable package has no such instruction.
2.3x Between the Two Packages
The portable version reads almost the same. The width is a runtime variable, and the final step has no ReduceSum to call:
func rankSIMD(node []int32, vx simd.Int32s) int {
var acc simd.Int32s
w := acc.Len()
for j := 0; j < B; j += w {
acc = acc.Sub(simd.LoadInt32s(node[j:]).Less(vx).ToInt32s())
}
// no horizontal reduction, so spill to the stack and sum with scalars
var buf [16]int32
acc.Store(buf[:w])
r := 0
for i := 0; i < w; i++ {
r += int(buf[i])
}
return r
}Why it is missing is written down in the proposal #78902:
The supported vector methods are those in the intersection of the wasm SIMD API and the current amd64 SIMD API
Only the intersection of WebAssembly SIMD and amd64 is included; horizontal reduction is not. Go through all 367 exported functions and methods of the simd package, and there is no ReduceSum, no GetElem, no movemask; archsimd's Int32x4 has every one of them.
To price the missing horizontal reduction on its own, I set up a control: the Neon version with everything else untouched, except that ReduceSum is replaced by the same spill-and-scalar-sum.
6 Neon + ReduceSum 14.55 ns
6b Neon + spill and sum 19.74 ns +36%One ADDV is worth 36%. Inside a dependency chain a dozen nanoseconds long, an extra store followed by a load puts store-to-load forwarding latency straight on the critical path.
The remaining gap, from 19.74 ns to 33.12 ns, comes from the portable abstraction itself, and the disassembly offers three clues.
First, the function gets cloned:
lab/stree.SearchSIMD
lab/stree.SearchSIMD@simd0 ← generic / emulated path
lab/stree.SearchSIMD@simd128 ← 128-bit specializationIn the @simd0 copy, rank is a real CALL and never inlined; only the @simd128 copy inlines it. The hardware picks one at runtime.
Second, that buf [16]int32 gets zeroed on every call:
STP (ZR, ZR), (R3) ← all 64 bytes cleared
STP (ZR, ZR), 16(R3)
STP (ZR, ZR), 32(R3)
STP (ZR, ZR), 48(R3)The buffer has to be declared for the widest possible 512-bit vector while only the first 16 bytes ever get used. Hoisting it out of the query took ten million elements from 35.58 down to 33.12 — barely worth mentioning, because at this size the bottleneck is memory, not those few STPs.
Third, the slice form LoadInt32s(node[j:]) carries a bounds check, whereas on the archsimd side I used the array-pointer form LoadInt32x4Array, where the check disappears at compile time.
The three together make 2.3x. “Portable” does not come cheap.
Three Potholes Along the Way
The first is a compiler ICE. Call a constructor from the simd package inside a method body, and go1.27rc2 crashes outright:
type T struct{ a []int32 }
func (t *T) M(x int32) int32 {
v := simd.BroadcastInt32s(x) // this line alone
...
}
./a.go:7:6: internal compiler error: missing Types entry: M@simd0(x)The minimal reproduction lives in simd/icebug, as its own module, because it fails to compile on purpose. The boundary is clear: free functions are fine, archsimd methods are fine, and only the combination of a method with a portable simd constructor blows up. Value receivers and pointer receivers both blow up. A method taking a simd.Int32s parameter does not — it only happens when you build a vector inside the method body.
Judging by the @simd0 suffix in the error, the rewrite pass that specializes on width does not handle methods. The workaround is easy enough: move the SIMD part into a free function. Every implementation in this article is written that way, not as a matter of style but because there was no choice.
The second is build tags. Both packages sit behind //go:build goexperiment.simd. Without GOEXPERIMENT set, your SIMD files drop out of the build entirely and the functions simply do not exist. A scalar fallback is therefore something you have to write and organize with build tags.
The good news is the portable version really is portable. I cross-compiled it to linux/riscv64 (no hardware support, pure Go emulation) and to linux/amd64, and both built. Portable code and guaranteed performance are two different things.
The third is API stability. The archsimd package docs still say "It currently supports AMD64" even though it already supports arm64 and wasm. Documentation lagging behind is a sign this API is still moving. Neither package is covered by the Go 1 compatibility promise.
Is It Worth It
A few conditions have to hold at once:
- the lookup sits on a hot path
- the data is large enough to overflow cache
- The structure is essentially static (my implementation does not support insertion at all)
- You are willing to accept the extra memory, and a copy of the code that is only fast on arm64
When they do not hold, that branchless binary search (49.77 ns, reachable by changing one line) is a far better deal.
Which package to pick? My advice is to write the architecture-specific archsimd version first, measure the ceiling, and only then decide whether portability is worth that 2.3x. For operators like vector dot products, the portable package is enough from the start; for algorithms with a horizontal reduction in them, you have to measure it yourself.
As for that ADDV, we wait for it to enter the intersection. Whenever wasm SIMD gets a horizontal add, Go's portable package can hand you one.
Full Code
All eleven implementations, the cross-check tests, and the raw benchmark data are in blog-example/go1.27/simd/stree:
stree/
baseline.go binary search (three variants) + Eytzinger
btree.go blocked layout construction + scalar rank
rank_simd.go portable simd version (three variants)
rank_neon_arm64.go arm64 Neon version (two variants)
stree_test.go cross-check + benchmark
probe/ runtime probe, prints VectorBitSize and the integer form of a maskHow to run it:
cd go1.27/simd
GOEXPERIMENT=simd go1.27rc2 test -run TestAll ./stree/ # all eleven cross-checked
GOEXPERIMENT=simd go1.27rc2 test -run XXX -bench . -count=10 ./stree/Every number in this article has its matching benchstat output in results/simd-stree-arm64.txt.
