Skip to content

Reproducibility

The estimator is a pure function of (family, n, seed). This page says what that buys you, and where it stops.

What is guaranteed

Given the same family, the same n and the same seed, on any host backend:

  • the same Report.value, bit for bit;
  • the same serialised JSON, byte for byte;
  • independent of core count, thread scheduling, CPU vector width, operating system, and whether the parallel or simd features were compiled in at all.

tests/determinism.rs and tests/backends.rs check this directly, and CI re-checks it on every push across six native OS/architecture combinations: Linux, macOS and Windows, each on x86-64 and aarch64. That covers both vector paths on real silicon, AVX2 and NEON, under three operating systems.

How

One seed, no ambient randomness. Nothing in the crate calls thread_rng(). Every random bit comes from a ChaCha20 stream keyed by the 64-bit seed in the config.

Per-draw substreams. Draw i uses fork(seed, i), a fresh ChaCha20 keyed by a SplitMix64 mix of the pair. Draw 5 does not depend on whether draws 0 through 4 ran, or in what order, or on how many threads. This is what lets the loop be parallel without being nondeterministic.

A fixed reduction tree. Floating-point addition is not associative, so any sum whose grouping depends on the thread count gives different bits on different machines. The reduction always collapses buf[k] = buf[2k] + buf[2k+1] with an odd tail carried up. The shape is fixed by the index order, which is what reduction: {order: "tree", leaf_order: "index"} in the config records.

Where it stops

The GPU backend. Single precision and a different normal-deviate algorithm. Reproducible on a given device, statistically equivalent to the host, not bit-identical to it. See Backends.

Across major versions of the crate. Bit-identity is maintained against v1.0.0 by a golden-value check in CI, and breaking it would be a major version bump with the change called out. It is not an accident that survives by luck.

Across implementations of the schema. Two independent implementations agree on the wire format and on the reduction policy. They will not agree bit for bit unless they also happen to use the same RNG and the same distribution algorithms, which the schema does not mandate.

The golden check

GOLDEN.txt holds 42 Report.value bit patterns produced by the original v1.0.0 implementation, across three families, seven ensemble sizes and two seeds. It is regenerated by CI on every platform and diffed:

cargo run --release --example golden > golden.actual
diff GOLDEN.txt golden.actual

The file is never regenerated to make a build pass. Every optimisation in this crate, the vectorised reductions, the thread pool, the flat ensemble storage, was accepted only after this diff came back empty.

Publishing a number

If you are putting a value in a paper or a report, publish a receipt alongside it. It is small enough for an appendix and complete enough to re-run:

receipt = {
    "family": family.to_dict(),
    "config": json.loads(config.to_json()),
    "report": json.loads(report.to_json(v1=True)),
    "value_bits": report.value.hex(),
    "library": {
        "package": "perturbation-kernel",
        "version": pk.__version__,
        "schema_version": pk.SCHEMA_VERSION,
    },
}

Two things are worth doing deliberately here.

Store the bits, not the decimal. report.value.hex() is exact. "Reproduces to six decimal places" is a much weaker claim than the engine actually supports, and it hides the difference between a genuine match and a near miss.

Use to_json(v1=True) for the report. That emits the strict SCHEMA v1.0.0 field set, without the additive execution block, so another implementation of the schema can read it without knowing about this one.

Verification re-runs from the receipt and compares:

value = family.run(config).value
assert value.hex() == receipt["value_bits"]

A worked version, including what happens when the receipt has drifted from the code and what happens when you verify on the wrong backend, is example 05.

Checking someone else's number

pip install "perturbation-kernel==2.0.0"
python verify.py receipt.json

If the value comes back and the schema major version still matches, the number is reproduced. If it does not, the failure names which of the two moved. That is a stronger position than "available on request".