Methodology — the validation toolkit

A codec or hash is not "done" because it round-trips on one input and is fast on one machine. In go-compressions it is done when it is proven compatible with the reference it interoperates with, proven safe on adversarial input, and measured honestly on representative hardware. Every repo follows the same pipeline.

1. Wire-compatibility with the reference

These are not new formats — they are clean-room pure-Go implementations of existing formats, so the bar is byte-level interoperability with the canonical reference:

repo	reference	compatibility proven
lz4	the LZ4 block format / pierrec/lz4	blocks are mutually decodable with pierrec both directions, verified on arm64 and amd64
deflate	DEFLATE (RFC 1951) / the standard library's compress/flate	streams from flate.NewWriter decode through our NewReader and streams from our Writer decode through flate.NewReader, both back to the original bytes — bidirectional, differential-fuzzed
lzfse	Apple's liblzfse C library	data from liblzfse round-trips through Decompress; Compress output decodes under liblzfse unmodified; every block magic Apple emits (bvx-/bvx1/bvx2/bvxn/bvx$) is handled
blake3	the official BLAKE3 test vectors	every digest (scalar, multi-core, and each SIMD path) is verified bit-identical against the published vectors
b3sum	the reference b3sum CLI	output format <hex> <name> matches; --check parses reference SUMS files

cgo wrappers of the C references (faster still, but needing a C toolchain) are explicitly out of scope — this is the pure-Go tier.

2. SIMD where it earns its place (and only there)

Acceleration is layered behind build tags, never at the cost of portability or correctness:

• matchlen — the LZ common-prefix kernel that lz4 delegates LZ4_count to (via go-simd/matchlen) — ships real SIMD on all six of Go's 64-bit targets (amd64 SSE2, arm64 NEON, riscv64 RVV, loong64 LSX, ppc64le VSX, s390x vector facility), generated by go-asmgen. lz4 needs no code change to benefit.
• blake3 — the default build is SIMD-accelerated on all six arches via a go-asmgen-generated mix4 round function, bit-identical to scalar and falling back to scalar for small inputs. A second, experimental path uses Go's simd/archsimd intrinsics (amd64 on Go 1.26+, arm64 on Go 1.27+) and is opt-in behind GOEXPERIMENT=simd.
• The pure-Go optimisations that need no assembly are applied first: blake3 folds the per-round message permutation into a lookup table and hashes independent chunks across all cores; lz4 uses a 6-byte hash, 3-position probe and lazy matching.

Where SIMD does not earn its place — e.g. the entropy-coding stage of a codec, where time goes to FSE/Huffman rather than to a vectorizable inner loop — it is left out, and the page says why.

3. Fuzz against the reference and against garbage

Two kinds of fuzzing run on every codec:

• Differential / round-trip fuzz against the reference: arbitrary input is compressed and decompressed (and cross-decoded with the reference where applicable), asserting byte-identical recovery.
• Corruption / random-garbage fuzz asserting no panic: a decoder fed adversarial or truncated bytes must return an error, never crash. lzfse's decoder is proven safe to call on arbitrary input this way.

Hashes are fuzzed for streaming/one-shot equivalence (the multi-core Sum256/Sum512 must equal the single-threaded Hasher) and verified bit-identical to the official vectors on every code path, including each SIMD kernel.

4. Validate and benchmark on real hardware

Correctness is proven against the reference; the headline benchmarks come from native runs, not emulation:

• Native arm64 (the Apple-silicon dev box) for the NEON kernels and the reported throughput figures.
• Native / QEMU amd64 for cross-checks — on QEMU x86_64 (TCG) absolute MB/s is low and noisy and is never quoted as a headline; it is used only to prove the compressed output is byte-identical to the arm64 run and decodes both ways with the reference.
• Native ppc64le on real POWER10 silicon (GCC Compile Farm, VSX, Go 1.26.4, June 2026). The "native POWER perf pending" caveat is now resolved for ppc64le: lz4 encode (matchlen-accelerated) runs 1.8× scalar (1174 vs 644 MB/s) and beats pierrec/lz4 (1174 vs 1012 MB/s), and blake3 mix4 runs 4.5× scalar.
• Native riscv64 on a real SpacemiT X60 (GCC Compile Farm, RVV 1.0, Go 1.26.4, June 2026). The X60 is a low-power in-order core and the only widely-available RVV silicon; an out-of-order RVV core would likely do better. The "native RVV perf pending" caveat is now resolved for riscv64: lz4 encode (matchlen-accelerated) runs 1.45× scalar (110 vs 76 MB/s) and beats pierrec/lz4 (110 vs 83 MB/s, ~1.32×), blake3 mix4 runs 2.9× scalar (3024 vs 8782 ns) with FillChunk ~2.0× (275 vs 137 MB/s), and matchlen runs ~5.8× scalar (1236 vs 214 MB/s).
• s390x stays qemu-validated for correctness, native perf pending. The matchlen and blake3 kernels run the official vectors and the byte-identical differential suites under QEMU, and on big-endian s390x the vector output is proven bit-exact — but there is no GitHub-hosted IBM Z runner, so native throughput is not measured and is deliberately not invented from emulated runs.
• Seventh architecture: ppc64 (big-endian), build + test validated. On real POWER9 silicon, every library (lz4, lzfse, blake3 / b3sum) builds and passes its tests bit-exact via the portable fallback path — distinct from the s390x vector kernel. SIMD stays six targets; correctness is validated on seven architectures.

5. 100% statement coverage, enforced as a CI gate

Every repo gates 100% Go statement coverage in CI — the build fails below 100%. Force tests drive every dispatch branch (each SIMD path and the scalar fallback) directly on the native runner, regardless of what the runtime CPU would otherwise select, so no branch is left unmeasured.

What coverage measures

The figure is of the Go code only. The generated .s SIMD kernels are not measured by go test -cover; they are validated by the differential tests against the reference plus the fuzz targets. "100% coverage" therefore means every Go statement, including every fallback branch — and the assembly is covered by the correctness suite, not the coverage counter.

Why this matters

The combination — byte-level interoperability with the canonical reference, fuzz that proves both round-trip fidelity and no-panic safety, real-hardware validation that refuses to trust emulation for headline numbers (and qemu-correctness validation where no native runner exists), and a coverage gate that refuses to ship an unexercised branch — is what lets go-compressions publish honest numbers: lz4 says plainly that it beats pierrec on ratio and decodes at parity with its arm64-asm decoder, while trailing on encode speed, and blake3's pure-Go default openly trails hand-written AVX2 on a single core, while both keep portability and bit-exact correctness as non-negotiables.