Audio harness (observability + validation)

Windows ARM64/ARM64EC adapter boundary

The Windows audio harness remains an offline evidence lane: run it against a deterministic WAV rendered by the plugin host and retain the exact binary, sample rate, block size, and artifact hash. A native ARM64 or ARM64EC/ARM64X plugin load check does not prove DAW compatibility. REAPER ARM64 is an ARM64X/ARM64EC host, so its scan, failed-scan list, real instance, render, and desktop screenshot must be recorded separately; a cache entry or LoadLibrary/factory result is discovery evidence only. If the host instance cannot be observed, report the DAW rung as unverified and do not promote the audio-lab result to plugin acceptance.

Pulp's agent-first way to turn "I can't hear it" / "does this sound right?" into inspectable, deterministic signal evidence — without a device, speakers, or a debugger. You are reading this skill because you need to prove, measure, debug, or regression-guard the audio a Pulp Processor emits.

Everything here is the test/tool layer, driven by pulp::format::HeadlessHost. Nothing in this skill runs on the realtime audio thread — measurements analyze buffers that have already left process(). The realtime probe path is a separate concern (see Roadmap).

It lives in two places, and the split is load-bearing:

• tools/audio/analysis/ — the pulp::audio-analysis lib (headers under <pulp/audio/analysis/…>). The pure buffer/file analysis: metrics, assertions, artifacts, and the FFT spectrum analyzers. It knows nothing about Processor, so the shipped pulp audio validate … CLI links it directly without pulling in any test/ library.
• test/support/ — the layers that need a Processor: signal generators, RenderScenario, contracts, and the scenario-driven Doctor wiring. Test-only; links the lib above.

The analysis target also owns the deterministic modal-extraction compiler (modal_extraction.hpp). It is offline-only: validate finite, sample-rate-qualified input, preserve recipe identity, and gate candidates with confidence/SNR/T60 evidence before emitting a ModalSpec; do not link it into realtime DSP or treat ranked candidates as host-facing parameter state.

Both use namespace pulp::test::audio, so the namespace does NOT tell you which target a symbol comes from — the include path does.

When to rope this skill in

• "There's no sound / it sounds wrong" — render the path offline and read the facts (peak/RMS/silence/frequency/NaN) instead of guessing.
• Building or changing a filter / oscillator / compressor / delay / sampler — state the contract and prove it (frequency response, delay time, decay, bypass-nulls).
• A regression hunt — "64 frames is fine but 128 is silent", "changing the sample rate chipmunked it", "the test tone toggle produces nothing".
• A DSP refactor — compare old vs new renders (exact / numeric / spectral).
• "How distorted is this?" — THD / THD+N and a harmonic breakdown.
• "What does this lowpass actually do at 8 kHz?" — a magnitude-response curve.
• "What's the group delay / latency of this filter?" — a phase + group-delay curve, in samples or seconds, per frequency.

The layering (each layer builds only on the ones below — no back-edges)

                    ── pulp::audio-analysis lib (also linked by the pulp CLI) ──
metrics → assertions → artifacts → spectrum
                    ── test/support (needs a Processor) ──
signal generators → scenarios → contracts → doctor

Layer	Header	Target	What it gives you
Metrics	<pulp/audio/analysis/audio_metrics.hpp>	lib	analyze() → BufferMetrics: peak, RMS, DC, NaN/Inf, clip count, silence-run; estimate_frequency() (zero-crossing, documented limits); to_dbfs; summarize() (agent-readable signal description).
Assertions	<pulp/audio/analysis/audio_assertions.hpp>	lib	assert_no_nan_inf / not_clipped / silent / not_silent / peak_between / rms_between / frequency_near / null_near / channels_independent — each returns CheckResult{passed,message} with dBFS/Hz/cents messages, never a bare float.
Artifacts	<pulp/audio/analysis/audio_artifacts.hpp>	lib	BufferMetrics → JSON (schema_version + provenance) for failing CI/local runs.
Spectrum	<pulp/audio/analysis/audio_spectrum.hpp>	lib	The FFT-bearing core of the Doctor, over already-rendered buffers (no Processor): response_relative_to_input(input, output, …), magnitude_spectrum_curve(), measure_thd(signal, …), measure_group_delay(input, output, …) (phase + group delay), plus the shared ResponseCurve / ThdResult / PhaseCurve / Window types. This is what the pulp audio validate doctor CLI runs over decoded WAVs — and what you call directly when your audio came from somewhere other than a scenario (e.g. rendered through a real format adapter).
Doctor artifacts	<pulp/audio/analysis/audio_doctor_artifacts.hpp>	lib	write_response_artifact() / write_thd_artifact() / write_phase_artifact() + the JSON serializers (phase_curve_to_json); kDoctorCurveSchemaVersion.
Latency evidence	<pulp/audio/analysis/latency_evidence.hpp>	lib	Delayed-null + impulse-marker latency policies as pure functions (see Proving reported latency).
Generators	"support/audio_test_signals.hpp", "support/audio_signal_generators.hpp"	test	Deterministic stimulus: sine/square/saw, impulse(+train), step, DC, multi-sine, swept sine, seeded white/pink/brown noise, stepped automation + MIDI note scripts. No clocks, no random_device.
Reverb	"support/reverb_metrics.hpp"	test	t60_schroeder / band_t60 (Schroeder backward integration, T20-extrapolated), echo_density_curve / mixing_time_seconds (Abel & Huang), range_rms, band_energy. Header-only; measures a reverberant DECAY, which is why it is not in the analysis lib (see Measuring a reverb below).
Scenarios	"support/render_scenario.hpp"	test	RenderScenario builder over HeadlessHost (factory, sample rate, block size, channels, duration, input/MIDI/param scripts); render() → ScenarioResult. run_matrix() (SR × block sweeps) + assert_block_partition_invariant().
Contracts	"support/audio_contracts.hpp"	test	AudioContract — a named claim + scenario + accumulated CheckResults; failures read contract '<name>': .... Family helpers expect_{passthrough,silence_preserved,tone,finite_and_unclipped}.
Doctor (offline)	"support/audio_doctor.hpp"	test	The scenario-driven entry points: response_relative_to_input(scenario, …), measure_thd(scenario, …), and measure_group_delay(scenario, …) (its results gated by defined_at(hz) for group delay and the stricter phase_defined_at(hz) for phase) synthesize the stimulus, drive the Processor, and delegate the math to audio_spectrum.hpp. It re-exports that header's result types, so a test that includes it needs nothing else.

Two traps this table exists to prevent:

• response_relative_to_input and measure_thd are each overloaded across both targets. The scenario overloads (audio_doctor.hpp) drive a Processor and synthesize their own stimulus, overriding the scenario's input/duration. The buffer overloads (audio_spectrum.hpp) analyze audio you already rendered and synthesize nothing. Same names, different layers — pick by where your audio comes from.
• The heavy FFT lives in pulp-audio-analysis and is never linked into a runtime/plugin build. That is enforced by a CMake target boundary, so a violation is a link error rather than a review comment. Do not add FFT code anywhere else.

Read test/support/README.md for the authoritative layering contract.

Run the proofs

Shared-memory GPU timing evidence

Use tools/scripts/gpu_audio_p4_evidence.py for paired shared/staged timing captures. Its evidence contract defines provenance, complete trials, transfer counters, and CPU accounting. Keep raw JSONL and the exact benchmark binary with every summary. Supply expanded direct compiler flags; response/configuration files and forwarded frontend options cannot prove the effective Release settings. Output-name validation uses hidden probes and must never publish empty final artifacts. Run python3 tools/scripts/test_gpu_audio_p4_evidence.py when changing this tool. A valid capture or passing schema test is not a physical performance verdict, and one paired row cannot authorize a product default.

Model-family and scheduling receipts

The model-neutral neural runtime plan defines the intended architecture and the boundary for future TCN, recurrent, compact-SSM, and Mamba work: https://github.com/danielraffel/pulp-planning/blob/main/research/2026-10-01-neural-audio-mlx-first-program-plan.md. The GPU-audio receipt contract remains model-agnostic, but every workload must identify its model family and prove its own state semantics. A WaveNet/NAM receipt does not establish TCN, LSTM/GRU, compact-SSM, or Mamba support.

For each model family, retain an independent CPU oracle and cover state continuity across block boundaries, reset, reprepare, model swap, device loss, and CPU fallback. Causal TCNs must cover receptive-field warmup and block alignment; LSTM/GRU must cover hidden/cell reset; compact SSM/Mamba must cover state/cache continuity and explicit reset. Keep Mamba behind the compact-SSM gate until it has its own parity and scheduling receipts.

The realtime campaign is a complete matrix over slots × lead × batch (or an explicitly documented unsupported cell). Each row binds sample rate, block size, intrinsic latency, transport lead, provider/engine/model identities, microbatch size, predicted execution time, deadline margin, observed completion-to-deadline tails, GPU delivery, late/drop/miss, and CPU fallback counts. Compare every accelerated row with a CPU-only control and a positive saturation/late/drop control. Missing timing, identity, or predictor fields make that row unavailable; they cannot be interpreted as zero cost or a pass. Keep the receipt at the same Release binary and host as the raw logs and benchmark executable. Use the proxy-first-eval template for minimum-lead claims and report intrinsic processor latency separately from transport lead.

# Build + run the whole harness (Release — Debug is meaningless for DSP timing/levels)
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
tools/ci/governed-build.sh cmake --build build --target \
  pulp-test-audio-support pulp-test-render-scenario pulp-test-audio-contracts \
  pulp-test-audio-doctor pulp-test-adapter-audio-parity pulp-test-golden \
  pulp-test-audio-matrix pulp-test-audio-tone-regression pulp-test-latency-contract
ctest --test-dir build -R 'audio|golden|render|contract|doctor' --output-on-failure

That ctest regex matches on test-case NAMES, so it is deliberately wide — it sweeps in several hundred unrelated render/contract cases from other subsystems. It is the right net when you want "did I break anything audio- shaped"; when you want just the harness, run the built binaries directly (./build/test/pulp-test-audio-doctor) or filter by tag (-R won't do tags — pass '[doctor]' to the binary itself).

The /audio-harness slash command wraps this. JSON metric/curve artifacts (on failure or on demand) land under a temp pulp-audio-metrics/ dir and are INFO-logged.

Localize a GPU STFT failure with the bounded recipe

Use gpu-audio.stft.v1 when the question is whether Pulp's offline Dawn STFT dispatch, readback, or magnitude output disagrees with an independent CPU FFT oracle:

mkdir -p "$PWD/artifacts/gpu/stft"
pulp gpu probe --recipe gpu-audio.stft.v1 \
  --artifacts "$PWD/artifacts/gpu/stft" --json

This is a deterministic, bounded CLI/MCP child-process probe. It never runs in the audio callback and does not open an audio device. A pass proves this fixed GPU STFT recipe completed and matched its CPU oracle on the reported adapter; it does not prove a plugin's realtime scheduling, arbitrary audio content, or the wider DSP contract. Keep using RenderScenario, Audio Doctor, and the offline audio-harness checks above for those claims.

Interpret the exit/result contract literally: exit 0 is a completed pass, exit 1 is a completed oracle failure, and exit 2 is unavailable or unverified evidence, never a skip-pass. Add --negative-control when validating the probe itself; the planted FFT mutation must complete GPU work and return the typed failure that proves the CPU oracle can catch it.

Measuring a reverb — two traps that make a working engine look broken

Both of these cost real debugging time on the multirate FDN reverb, and both produce a plausible wrong number rather than an obvious one.

Measure decay IN A BAND, not broadband. A broadband Schroeder integration folds the top two octaves into the number, and any fractional-delay read in a feedback loop — which is what delay modulation requires — costs real high-frequency energy per pass. On that engine the cost is 19% of the 10 kHz decay against 3% at 1 kHz, so the broadband T60 moved 10.6% when modulation was switched on and modulation looked like a decay control. It is not one. Use band_t60(ir, fs, 1000.0); state the probe frequency in the test.

A proportional band's energy is not a level — normalize by bin count. A +/-12% band at 8 kHz is 32x wider than the same band at 250 Hz, so summing its bins builds a ~15 dB tilt into any band-to-band comparison and a perfectly flat tail reads as 16 dB of spread. band_energy returns the RMS across its bins (a density) for exactly this reason. If you hand-roll a band measure, do the same.

A third, cheaper one: use a band, not a bin, for anything granular. A granular pitch shifter spreads its output into grain-rate sidebands, so a single 880 Hz bin under-reports a working octave-up shifter by an order of magnitude and can read as zero.

Proving a freeze / hold — steady input cannot see capture bugs

A hold averages a window of past frames, so every bug in WHICH frames it averaged is invisible on a steady sine: stale and fresh frames look the same. FreezeHold's rapid-toggle test passed for months over a real defect (a quick re-freeze averaged ~7/8 frames from before the previous freeze). Two stimuli make the window observable:

• Change the tone mid-stream. Tone A before the first freeze, tone B from partway through the hold on; the re-freeze must hold B, judged by spectral peak. Keep a control in the same test: the first hold must still read A while the input is already B, or the probe proves nothing.
• Give every frame its own bin in a synthetic frame test (frame f → bin 10 + f). The set of non-zero held bins then names exactly which frames were averaged — an age assertion, not a similarity score (test/test_freeze_hold.cpp).

Two more traps from the same work:

• A phase accumulated in double and converted to float drifts. A high bin advances thousands of radians per hop; unwrapped, the float conversion at mix time loses the stereo offset and the frame-to-frame advance within seconds. Assert the inter-channel phase offset and the per-frame advance over a long hold (thousands of frames at a large hop), not the level.
• Accumulated 1/N fade steps do not reach 1 for every N (N = 12 stops at 0.99999988 in float), so "pure hold" never arrives and a residual live leak stays. Count fade steps as integers and test several N, including 12.
• A hold across a change of material needs a change stimulus, and a level check. With the default newest-frame frequency, a 2 s hold taken 0.4 s after a chord change plays the older chord ~55 dB down: its bins rotate at the newer chord's leakage frequency and cancel frame to frame. energy_weighted frequency brings it to ~-11 dB. A steady tone passes either way. The residual gap is the latched frame's lobe phases (taken from the newer material), which nothing restores yet.
• Judge a hold's level over time, not at one point. An unlocked hold starts phase-coherent and decoheres while audible (about +1 dB over its first 0.5 s on a detuned pad, settling over ~2 s). Compare the first 0.5 s of pure hold with the last second; phase_lock = peak_lobe holds it within 0.1 dB. Short (50 ms) windows still beat on detuned partials, so bound the window deviation loosely and the early-versus-settled mean tightly.
• A per-hop cost test needs the default's burst as its control. The freeze cost tests in test/test_freeze_hold.cpp count operations per frame group (rt::RtWorkCounter) and assert the default configuration shows the burst (a latch summing capture_frames × channels × bins, a polar per bin per held hop) before asserting the bounded configuration does not.

An offline render cannot hear a dropped buffer — measure cost at every transition

Offline has no deadline: a callback that takes 4 ms against a 2.67 ms budget (128 frames at 48 kHz) renders perfectly offline and clicks in every DAW, because the host drops that buffer. A freeze engage that ran ~23 IFFTs and ~94 k std::polar calls in the latching callback shipped with six offline click tests green. Before reporting a transition (parameter edge, engage/release, mode switch, preset load) clean, measure what the callback containing it costs.

• RenderScenario records a BlockCost per processed block (result.block_costs: thread CPU time, wall time, operation counts, and how many parameter steps landed before the block).
• TransitionScenario::standard(factory) (test/support/transition_scenario.hpp) enumerates every parameter from the processor's own define_parameters() and steps it min→max and max→min mid-stream over loud broadband input; .add() a plugin-specific edge (a freeze at each hold length, a mode switch).
• The required gate is operation counts, assert_transition_ops_bounded(): FFT executions (hooked in Fft / MultiBackendFft), transcendental calls made through pulp::signal::rt::polar/sin/cos/exp/arg, and per-bin accumulation a processor reports with rt::count_bins() (core/signal/include/pulp/signal/rt_work_counter.hpp). Counts cannot flake. They exist only when PULP_RT_WORK_COUNTERS is on (it follows PULP_BUILD_TESTS); the gate FAILS rather than passing vacuously without them. DSP that wants its trig visible must call the rt:: wrappers — a bare std::polar is invisible to the count.
• Timing is advisory (assert_transition_cpu_ratio(), performance label): minimum over repeats of worst-after-edge / worst-steady thread CPU time.
• Both compare against the steady maximum, not the median: an STFT whose hop exceeds the block size is bursty in steady state by design. Give warmup_blocks at least one full hop, and settle_blocks enough to contain any staged work.
• Negative control: BurstyProcessor<23> (test/support/bursty_processor.hpp) must fail both gates and BurstyProcessor<0> must pass, in the same suite (test/test_transition_cost.cpp, test/test_transition_cost_timing.cpp).
• Fix a failing transition by amortising the work over the following blocks or precomputing it in prepare(), never by widening the allowance for work that could be staged.

Glitch hunt: one render, audio and trace on one timeline

The fastest loop found for "it clicks / drops out / sounds staticky" in a processor, in this order:

1. Stamp the timeline. Emit one span per host block from process(): PULP_TRACE_SCOPE_NAMED_ARGS("dsp", "process", "stream_pos", pos, "frames", n) with pos the stream sample of the block's first frame (a member counter reset in prepare()). It compiles to nothing in a shipping SDK. Without stream_pos there is no sample <-> time mapping and no tool can make one. Add a span or instant around the suspect work (a renderer swap, a design handoff) so it shows up nested in the block.
2. Render the scenario under a session in a test built against a tracing-enabled SDK (verify with nm <binary> | grep -c perfetto -- a shipping SDK links zero and Tracing::start returns false): pulp::runtime::Tracing::start({"dsp","state"}, "<flow>.pftrace", 256*1024), render, Tracing::stop(), and write the output WAV beside the trace.
3. One command joins them: python3 tools/audio/glitch_trace.py --wav <flow>.wav --trace <flow>.pftrace -> every click (whitened-residual spike) and dropout (5 ms level gap) with its sample, the block that rendered it, that block's wall time against its deadline, and the slices nested in it; plus block-time p50/p99/max and deadline misses for the whole render.
4. Prove the detector with the defect restored (an env-var plant) before trusting a clean run -- a reference-free detector that sees nothing may be looking at the wrong window. Spectr's Latency-switch cut read clean until the scored window was opened 10 ms BEFORE the event: a spike detector with a 4 ms neighbourhood cannot score the first 4 ms of its own window, which is exactly where a cut lands.

Gotchas that cost real time:

• An offline render cannot hear a dropped buffer. A clean WAV with a block past its deadline is a live glitch; read the block-time line, not just the event list. And a real-time-paced render (audio on its own thread, sleeping to each block's deadline, UI edits from another thread) is the only harness that reproduces worker-timing effects -- build one before concluding "cannot reproduce".
• Vary the host's callback size. Hosts split buffers at automation and loop points; render with random per-callback lengths too, not only fixed blocks.
• Compare against an unedited render of the same material. Drum hits score like clicks; the edit's score minus the untouched render's score in the same window is the number that means something.
• Low buffers are a burst problem, not an average one. At 32 samples the deadline is 0.667 ms at 48 kHz and 0.333 ms at 96 kHz; a processor whose p50 is 20 us can still miss it every hop, because work that runs once per analysis hop (an FFT, a capture/average over bins) lands whole in the one callback that crosses the hop. Measure per-callback thread CPU (not wall time -- the machine's load is not the processor) at 16/32/64 samples and 44.1/48/96 kHz, read max and p99 against the deadline, then attribute the burst with dsp spans around each per-hop stage. Gate it with a planted burst of about the size you are guarding against as the negative control; a plant ten times too big proves nothing about the margin.
• Work a value's only reader needs belongs at that reader. A per-bin transcendental computed every hop for a value read only on a rare event (a freeze latch) is a per-hop burst for nothing: keep the raw input and derive at the event, which is bit-identical (FreezeHold takes its increment's argument at the latch, and counts zero trig on capturing hops).

A hop of latency is a hop of slack -- spend it on the burst

An STFT/WOLA processor whose latency is fft_size + hop does not need a frame's resynthesis until a hop after the frame completes, and a consumer whose output is scheduled latency samples after an analysis frame (an estimator emitting gain events) does not need that frame's result until then. Doing the work in the callback that completed the frame is the burst that misses a 32-sample deadline. Spread it inside the slack, by STREAM POSITION (so it stays block-size invariant), and keep at the frame boundary only what must be decided there (layout/control adoption). SpectralFrameEngine::process() does this for its inverse transforms (Config::defer_synthesis, default on, bit-identical). Prove exactness by comparing against the immediate path across block sizes 1..1024, and count transforms per callback with rt::RtWorkCounter rather than timing them.

What the slack cannot buy: the work that must happen AT the frame boundary -- the forward transforms and whatever decides adoption (a staged mask/layout) -- still lands in one callback. Measured on Spectr at 96 kHz: with its resynthesis, freeze capture and estimator frame all spread, a 32-sample buffer (0.333 ms) holds p99 ~90 us and max ~180-240 us, but a 16-sample buffer (0.167 ms) still misses on the callbacks that complete a WOLA frame (two 8192-point forward FFTs plus the mask). Closing that needs either a pruned/incremental forward transform or moving adoption off the boundary (which costs block-size invariance unless adoption is re-keyed to stream position) -- or one more hop of latency. Say which, with the numbers, before claiming "every buffer size".

Copy-this patterns

Describe / debug a render (the "no sound" workflow):

auto m = analyze(rendered, 48000.0);
INFO(summarize(m, estimate_frequency(rendered.channel(0), 48000.0)));
REQUIRE(assert_not_silent(m, -60.0).passed);   // which stage went silent?

State and prove a DSP contract:

auto sc = RenderScenario(create_my_lowpass)
    .name("mylp.attenuates_8k").sample_rate(48000.0).block_size(128)
    .input(Sine{.hz = 8000.0f, .dbfs = -12.0f}).set_param(kCutoff, 200.0f);
AudioContract c("mylp.attenuates_8k", sc);
c.expect(expect_finite_and_unclipped(c.result()))
 .expect(assert_block_partition_invariant(sc, {64,128,256}));
REQUIRE(c.verify().passed);     // failure says `contract 'mylp.attenuates_8k': ...`

Measure it like Plugin Doctor (offline Doctor, scenario-driven):

auto curve = response_relative_to_input(sc, {50.0, 8000.0});
REQUIRE(curve.attenuation_db_at(8000.0) >= 20.0);   // "drops ≥20 dB at 8 kHz"
auto thd = measure_thd(sc, /*fundamental_hz=*/999.0); // steady bin-coherent sine
// thd.thd_percent(), thd.thd_plus_n, thd.harmonics[...]

// Phase / group delay. ALWAYS gate first: a stopband has no phase to measure,
// so the analyzer reports it undefined and the accessors return NaN rather than
// a number read out of the noise floor. TWO gates, and they are not the same —
// defined_at() qualifies group delay, phase_defined_at() qualifies phase.
auto gd = measure_group_delay(sc, {100.0, 8000.0});
if (gd.defined_at(100.0))
    REQUIRE(gd.group_delay_samples_at(100.0) < 8.0); // "≤ 8 samples latency"
if (gd.phase_defined_at(100.0))                      // STRICTER — see below
    REQUIRE(gd.phase_radians_at(100.0) < 0.0);
// gd.group_delay_seconds_at(hz)
// gd.magnitude_db_rel_peak_at(hz) — peak-relative, NOT the absolute out/in
// ratio that ResponseCurve::magnitude_db_at returns. The two curves use
// different names for this reason; a +12 dB passband reads +12 there and 0 here.

Measure audio that did NOT come from a scenario — e.g. rendered through a real format adapter, or decoded from a WAV. Drop to the buffer overloads in audio_spectrum.hpp; you supply the stimulus AND the render, so nothing is synthesized for you:

#include <pulp/audio/analysis/audio_spectrum.hpp>
const auto impulse  = /* your stimulus  */;
const auto rendered = /* what came back */;
const double checkpoints[] = {50.0, 1000.0, 8000.0};
auto curve = response_relative_to_input(impulse.view(), rendered.view(),
                                        48000.0, checkpoints, {.fft_length = 16384});

test/test_adapter_audio_parity.cpp is the worked example of driving the real adapter: it renders one Processor through clap_process() and through HeadlessHost and requires the bits to match.

Measure the path the code under test actually runs. Driving clap_process() is not by itself proof that you measured the adapter. A clap_process_t proc{} leaves out_events null, and the whole output-parameter publication phase sits behind if (out_events) — so a test that never sets it renders audio through the adapter while silently skipping the param path, and stays green no matter how badly that path breaks. Before trusting an adapter measurement, check that the lines you care about are reachable from your fixture, then break them on purpose and confirm your test goes red.

Two things that will bite you when measuring an adapter path:

• THD is blind to anything non-harmonic. It sums energy at integer multiples of the fundamental only. Chop each 256-frame block in half and you amplitude- modulate the tone at 187.5 Hz; the sidebands land between harmonics, so THD reads a serene −170 dB while the signal is visibly destroyed. thd_plus_n counts every non-fundamental bin and reads ~0 dB (100%). Assert THD+N against a stated floor, not just thd_plus_n >= thd — that comparison is a tautology and proves nothing.
• Response and THD are both RATIOS, so they cannot see a broadband gain error. A wrong-by-0.5 dB adapter path passes every Doctor check. Level is a null test's job.

Group delay is a derivative, so the estimator matters. measure_group_delay uses the ramped-signal Fourier identity (τ = Re{X_r·conj(X)}/|X|², X_r = FFT(n·x[n])) rather than differencing an unwrapped phase curve: it needs no unwrapping, has no frequency step to tune, and is exact for any impulse response that fits inside the rectangular analysis window. Budget fft_length so the IR has decayed within it — a truncated IR is the one real error source, and the analyzer will faithfully report the delay of the truncated signal.

phase_defined is stricter than defined, and you want the strict one for phase. Reported phase_rad IS unwrapped, and unwrapping is a walk from DC upward — so a bin's phase is only as trustworthy as every bin below it, and past a stopband null or a gap in the reference spectrum the accumulated offset lands on an unresolved 2πk branch. defined cannot see that: it asks only whether this bin has energy, which is the whole precondition for group delay (estimated per bin, never consulting the unwrap) and not nearly enough for phase. So a bin can be defined — carrying a real, correct group delay — while its phase is unreachable. phase_defined is the gate that says so, and phase_radians_at() is NaN wherever it is false. Expect exactly this on a band-limited or gapped reference: full group delay across the band, phase only up to the first gap. (The separate aliasing bound — phase unwraps correctly only while group_delay < fft_length / 2 — is not gated: per bin the analyzer cannot tell an aliased phase from a true one, so it is your budgeting duty.)

defined is not only a stopband test — it also rejects a near-silent output. The estimator divides by |X|², so it has an energy floor of its own below which a bin has no phase to differentiate. A processor whose output has collapsed (a dead gain stage, a degenerate filter) can still produce a perfectly shaped impulse response, just at −160 dBFS: its peak-relative magnitude looks like a clean 0 dB passband, but every bin is under the estimator's floor. Those bins are defined = false. If a group-delay curve comes back entirely undefined for a processor you expected to measure, check the output's absolute level before suspecting the analyzer — an all-undefined curve usually means the processor emitted (almost) nothing, which is the finding.

Discipline that keeps it trustworthy

• Release only. A Debug DSP/UI build mismeasures levels and timing.

• Do not assume a polynomial is faster than the platform math library. FastMath::exp2 delegates to the standard float implementation because a cubic-plus-ldexp path measured slower on supported Apple hardware while losing NaN, infinity, overflow, underflow, and gradual-underflow semantics. In Pulp callbacks, ScopedFlushDenormals still governs the finite subnormal range and may flush it to zero, matching the platform standard in that FP mode. A replacement needs a repeatable Release benchmark over pitch and audio-rate FM domains as well as dense numerical and product-level error proofs. Reproduce the current comparison with:

  cmake -S . -B build-exp2-bench -DCMAKE_BUILD_TYPE=Release \
    -DPULP_BENCHMARK=ON -DPULP_ENABLE_GPU=OFF -DPULP_BUILD_EXAMPLES=OFF
  tools/ci/governed-build.sh cmake --build build-exp2-bench \
    --target pulp-fast-exp2-benchmark --parallel 4
  ./build-exp2-bench/test/pulp-fast-exp2-benchmark
  

  The optional, non-ctest target covers pitch [-16,16], audio-rate FM [-8,8], and the normal float exponent range [-126,127]. Its prior_pulp_cubic_exp2 is explicitly benchmark-only historical Pulp code; observable checksums prevent either lane from being optimized away. Record repeated medians from a Release binary (-O3 -DNDEBUG on the supported Apple Clang configuration); timing is evidence, never a CI pass/fail threshold.

  Advisory evidence snapshot (2026-08-06, non-portable): three consecutive executions on Daniels-Mac-Studio (Apple M3 Ultra, arm64, macOS 26.5.2, Apple Clang 21.0.0) from the configuration above produced these median-nanoseconds-per-call ranges:

  Domain	FastMath::exp2 (std delegate)	Prior Pulp cubic	Prior / current
  pitch [-16,16]	1.399-1.401	1.678-1.730	1.200-1.235x
  FM [-8,8]	1.427-1.436	1.684-1.709	1.175-1.191x
  normal [-126,127]	1.401-1.480	1.678-1.798	1.195-1.215x

  All three executions reproduced identical per-domain checksums for each lane. These numbers document this host/compiler decision; rerun before drawing a conclusion on another architecture or toolchain.

• Vectorized kernels live in pulp::simd, and pulp::signal can call them. <pulp/simd/simd.hpp> (target pulp-simd) is linked INTERFACE by pulp::signal; its link items are Highway and, on Apple, Accelerate, so a DSP header may call it without dragging in pulp::runtime and its TLS stack (configure fails if that ever changes). <pulp/runtime/simd.hpp>'s simd_* names are inline wrappers kept for existing callers. Gotchas:

  • The unqualified names are a compile-time alias of one backend (PULP_SIMD_BACKEND: auto = Accelerate on Apple, Highway elsewhere). All compiled backends stay callable as pulp::simd::backend::{scalar, highway, accelerate}; pulp-test-simd-parity runs every kernel on each against scalar. A new kernel lands in all three backends and in that suite in the same change, then gets a row in the throughput benchmark.
  • Tolerance classes (the parity suite states the derivations): one-rounding elementwise ops and max/min are bit-exact; multiply-adds get one extra rounding; reductions and FIR correlation get 2 * n * eps * sum|terms| because each backend sums in its own order. Accelerate advances vDSP_vrampmul's gain by repeated addition, so ramp_mul error grows with the index. The measured float FIR null is -115.6 dBFS worst (Accelerate, 256 taps), floored at -105 dBFS.
  • Under flush-to-zero, a scalar std::clamp copies a subnormal through unchanged while a vector min/max returns zero; parity with subnormal input needs a numeric_limits::min()-scale slack, not bit equality.
  • A break-confirm that injects an allocation must let the pointer escape (store it in a volatile global): clang elides a paired new/delete, so the probe sees nothing and the control is void.
  • Inside namespace pulp, an unqualified simd:: now names Pulp's kernel namespace, not Apple's <simd/simd.h> vector library. Spell Apple's as ::simd:: (see fast_math.hpp); the collision only shows up in a translation unit that also includes a header reaching <pulp/simd/simd.hpp>, so it can hide until an unrelated include lands.
  • Signal headers that call pulp::simd are also compiled by hand-listed builds (WAM/WebCLAP, VCV Rack via forge-modular, Android, live kernels). Those add core/simd/include to their include list and get the header's inline scalar fallback (active_backend_name == "inline-scalar"), with no library to link.
  • Accelerate's vDSP_conv/vDSP_desamp are not partition-invariant: the same output can round differently depending on where it falls in the call. Highway and scalar correlate are (fixed per-output order), so a block-invariance test is bit-exact there and bounded on Accelerate.

• Processor-wide throughput lives in one benchmark; extend it, do not fork it. pulp-dsp-throughput-benchmark (PULP_BENCHMARK=ON, Release) times the heavy processors at 48 kHz x {32,128,512} plus the scalar kernel shapes they reduce to, and --json PATH writes pulp-bench-sections/1 for tools/scripts/bench_diff.py baseline.json current.json. Add a row there when a DSP change needs a before/after number. Read it correctly:

  • Mean is the median over repetitions of whole-run time; worst-block p99 and max time each block alone, so they carry one timer read per block and the clock's tick (Apple silicon: ~41.7 ns, i.e. ~1.3 ns/frame at B=32). Judge sub-ns rows by the mean, and judge realtime headroom by p99, not mean: a partitioned convolver's mean hides FFT blocks many times its average.
  • Every case feeds a rolling noise window and publishes output through an asm memory clobber. A row that reads near zero was hoisted, not fast; the JSON notes flag kernel rows under 0.005 ns/element as suspect.
  • --smoke exercises every case in about a second; its numbers mean nothing. --filter SUBSTR runs a subset. The JSON records optimized and ndebug; a baseline without both true is not comparable.
  • The dsp-throughput-bench.yml workflow (weekly + dispatch) uploads the arm64 macOS and x86-64 Linux JSON as artifacts. It is advisory and must never gain a threshold.

• Fast trigonometry is accepted per consumer, not per primitive. Read docs/validation/fast-trigonometry.md before changing a realtime sine/cosine path. It records the shipped profiles, licensing pin, Release methodology, actual FM/additive results, rejected Chebyshev/RLIBM/vForce/recurrence/ASM alternatives, and exact reopen rules. For Apple additive work, enable PULP_BENCHMARK, build pulp-fast-trig-apple-bank-benchmark, verify both cache and target flags say -O3 -DNDEBUG, and repeat the order-balanced consumer matrix. Primitive throughput cannot authorize a call-site sweep, default change, global fast-math flag, or a quadrature/GPU/control claim. For a new local-overlay candidate, run pulp-fast-trig-benchmark --qualification-only --inputs 16777216 first; this emits dense error JSON without timing loops. Exit zero means the measurement completed, not that the candidate passed; compare its fields with the selected profile's numerical contract. If it misses, stop rather than interpreting incidental speed.

• Grade discontinuity correctors against the shipped baseline. A minBLEP table needs a reproducible generator and a freshness check, but table provenance alone does not prove audio quality. Render free-running and synced product paths, measure their in-band alias residual with the shared projection analyzer, and gate the improvement over the current polyBLEP path. Exercise fractional endpoints, collisions, reset, block partitioning, and near-Nyquist overload separately; a low ordinary-rate alias number says nothing about bounded event storage.

• Analyzer Determinism Contract — every spectral/estimator assertion declares its stimulus, window (rectangular for a bin-coherent tone or an impulse; Hann for broadband — a Hann window annihilates an impulse at n=0), warm-up trim, estimator, seed, sample rate, and tolerance class. State them in the test so a red is a real DSP change, not an analyzer artifact.

• Named tolerances, never magic numbers — see the plan's Threshold Policy.

• Layering is one-way. Doctor may use scenarios + spectrum; nothing below scenarios may include audio_doctor/audio_contracts, and the pulp::audio-analysis lib may never depend on the test-only layers (it has no Processor/scenario knowledge).

• Prove the assertion can fail. A green measurement is not evidence until you have seen it go red. Mutate the thing under test — drop the param event, zero the output, route the id wrong — rebuild, and confirm the check you are relying on actually turns red. This is cheap and it repeatedly catches tolerances that are too loose to detect the bug they were written for: a 200 Hz lowpass check at 8 kHz looks decisive but still passes with the filter left at its 1000 Hz default, because 8 kHz is ~36 dB down either way. Only the 1 kHz checkpoint separates them.

• A fixed sample-rate converter does not anti-alias a variable playback rate. Tempo maps, varispeed, and other runtime projections can advance the source position by more than the asset-rate/timeline-rate ratio prepared at compile time. Derive the reconstruction cutoff from the effective source frames per output frame, prepare a bounded kernel bank off the audio thread, and select from it without allocation. Prove the path with both a passband control and a bright source that aliases under linear point sampling.

• Compiling a harness TU by hand? Copy CXX_DEFINES, not just CXX_FLAGS and CXX_INCLUDES. Both live in the target's build/**/flags.make. Adapter structs are gated on defines (PULP_CLAP_GUI adds members to PulpClapPlugin — the header says so at the top), so a TU built without them disagrees with the prebuilt libpulp-format.a about the struct's size and clap_init writes off the end of your object. The symptom is a __stack_chk_fail abort after every assertion has already printed PASSED, and it appears/disappears with unrelated stack layout changes — it will read as a product bug and it is not one.

Live inspection (Audio Inspector window) — landed

For live signal inspection while a standalone app / hosted graph runs, there is a separate developer tool window: pulp::view::AudioInspectorWindow (core/view/.../audio_inspector_window.hpp). It is a sibling of the layout inspector, not a tab — open it via its CommandRegistry command kToggleAudioInspector (default Cmd/Ctrl+Shift+A, rebindable) or the /audio-inspect slash command. It shows meters (peak/RMS/clip/NaN-Inf/silence), the observed probe stage, a copied fixed-capacity waveform, channel balance + an L/R level-match ratio, and a device/runtime summary — all polled once per UI tick from a realtime-safe AudioProbeSnapshot (it never touches the audio thread). It honestly shows a "no probe" / "stale" state rather than faking zeros. Live data requires the standalone output-boundary tap, which is gated behind PULP_ENABLE_AUDIO_PROBES. NOTE: the "L/R match" is a level ratio, not a phase/Pearson correlation (the RT snapshot carries no inter-sample L*R term yet).

This is for watching what is currently flowing; controlled-stimulus measurement (response/THD/etc.) is the offline Doctor above.

Launch the Audio Inspector in the standalone host

StandaloneApp::run_with_editor() wires the inspector to the host's output-boundary probe automatically (behind PULP_ENABLE_AUDIO_PROBES, default ON for dev/examples). At runtime, toggle it with Cmd/Ctrl+Shift+A, or open it on launch by setting PULP_AUDIO_INSPECTOR in the environment:

# Live: feed a test signal so output is non-silent, then open the inspector.
PULP_AUDIO_INSPECTOR=1 ./build/examples/<app>/pulp-<app>

CLI shortcuts (resolve the standalone binary + set the env vars for you):

pulp run --audio-inspector                    # open the live inspector window
pulp run --audio-probe-json /tmp/probe.json   # headless: dump probe JSON + exit
pulp run --audio-scope-json /tmp/scope.json   # headless: dump live sample-window scope JSON + exit
pulp audio scope --input-wav out.wav --json /tmp/scope.json --png /tmp/scope.png

For MCP clients, use the existing pulp-mcp tools instead of creating a new MCP server. pulp_audio_probe_json is a one-shot wrapper around pulp run --audio-probe-json, accepts optional target and frames, and returns scalar probe counters as structuredContent. pulp_audio_scope returns pulp.audio.scope.v1 sample-window acquisition/measurements; live target mode may open the audio device, while input_wav mode is speakerless offline and can also write a PNG artifact.

Agent triage pattern:

1. Start with pulp_audio_probe_json for the target you are debugging.
2. If the tool errors or no JSON is written, report a launch/probe problem, not a DSP conclusion.
3. If callbacks == 0, increase frames or inspect standalone startup/device lifecycle.
4. If callbacks advance but peak_max == 0 and rms_max == 0, treat the observed output boundary as truly silent; debug routing, input stimulus, bypass/mute state, graph wiring, or the processor output branch.
5. If clip_count/clipped_blocks or nan_inf_count/nan_blocks are non-zero, prioritize DSP/gain/state initialization bugs.
6. If the live snapshot is healthy but the user says the sound is wrong, switch to pulp_audio_scope / pulp audio scope --input-wav for sample-window facts, or to an offline render + Audio Doctor for THD/response/residual checks. The scalar probe cannot prove THD, response, phase, latency, or perceptual quality.

--audio-probe-json is the programmatic readout for agents: it writes output_probe().latest() (+ the AudioStats subset) as a flat JSON object — stage, sample_rate, block_size, channel_count, sequence_number, peak_max/rms_max, peak_dbfs/rms_dbfs (null on true silence), clip_count, nan_inf_count, clipped_blocks, nan_blocks, silence_run_blocks, callbacks, then exits. The mapping is the pure pulp::audio::audio_probe_snapshot_to_json() helper (pulp/audio/audio_probe_json.hpp); the frame delay reuses --frames / PULP_FRAMES. This is the live counterpart to the offline pulp audio validate Doctor below. See docs/guides/audio-inspector.md.

PULP_AUDIO_INSPECTOR also enables the probe's capture ring (sized to the panel display width), so the inspector paints a live waveform and not just meters — the default probe config is summary-only. The toggle routes through a shell-owned CommandRegistry via route_global_keys (the root view's on_global_key), which is independent of the layout inspector's on_global_click (Cmd/Ctrl+I) — both tools coexist on one window without clobbering.

Headless proof: a screenshot run with the inspector open also captures the inspector's OWN window surface next to the main screenshot:

# Writes /tmp/x.png (main) AND /tmp/x.audio-inspector.png (the live panel).
PULP_AUDIO_INSPECTOR=1 ./build/examples/<app>/pulp-<app> \
  --screenshot /tmp/x.png --screenshot-frame-delay 90

The sibling <stem>.audio-inspector.png is only written when the inspector window is visible and the host has GPU capture (WindowHost::capture_png()).

CLI: pulp audio validate <verb> (offline, over WAVs / artifact bundles)

Do not hand-roll WAV analysis. If you are about to soundfile.read() two WAVs and compare them, these verbs already do it. They are registered in docs/status/tools.yaml, whose digest is generated into CLAUDE.md, and a PostToolUse hook names them if you start writing the script anyway. The design tools learned this the hard way: guidance that exists but never fires at the moment of need is the same as no guidance.

The harness analyzers are also reachable from the shipped CLI, nested under the existing pulp audio command (the model/excerpt-find/read-bundle verbs are untouched). The CLI is not tied to a plugin, so it analyzes captured audio files and stored audio-run/ artifacts — not live processor instantiation (controlled-stimulus render stays the test-side RenderScenario). It links the reusable pulp::audio-analysis lib (tools/audio/analysis, namespace pulp::test::audio), the same file-analysis code the test harness uses; no test/ library is linked into the CLI and no FFT leaks into a runtime build.

# Agent-readable signal summary (peak/RMS/DC/dominant pitch); --json for machine output
pulp audio validate summarize out.wav [--json]

# Offline Audio Doctor: THD/THD+N and/or spectrum magnitude at checkpoints.
# Writes a schema-versioned JSON curve artifact to the temp dir.
pulp audio validate doctor out.wav --thd [--fundamental 1000]
pulp audio validate doctor out.wav --response 100,1000,8000

# Null/spectral diff verdict (exits nonzero past tolerance)
pulp audio validate compare before.wav after.wav [--mode null|spectral] [--tolerance -120]

# Re-check a stored assertions.json (or an audio-run dir holding one); nonzero on failure
pulp audio validate assert audio-run/assertions.json

assertions.json schema: {"schema_version":1,"assertions":[{...}]} where each entry has a check (not_silent, silent, no_nan_inf, peak_below, frequency_near), a file (relative to the JSON), and the check's named tolerance (min_rms_dbfs, ceiling_dbfs, expected_hz + tolerance_cents, ...). The /audio-harness slash command documents these verbs and points at the offline render path below.

Offline plugin render (pulp audio render)

pulp audio render --plugin <bundle> --out <file.wav> (--duration-ms <n> | --duration-frames <n>) is the scenario-driven counterpart to validate: it loads an explicit plugin bundle through pulp::host::PluginSlot (the generic CLI has no registered factory, so a bundle is the only offline render source), drives it block-by-block from declarative flags, writes an int16 WAV, and emits the same pulp::audio-analysis metrics JSON as validate summarize --json (--manifest <file> / --json). No DAW, no audio device. Drive it with --input-signal silence|sine:<hz>[,<dbfs>] or --input <file.wav> (used as-is at --sample-rate — no resampling), --param <id>=<value>[@frame], and --midi note:<note>,<vel>,<on>[,<off>].

--param values are PLAIN domain (the parameter's native min..max), not normalized [0,1] — matching PluginSlot::set_parameter / ParameterEvent::value. Parameters are delivered sample-accurately: the stepper windows each block's events with per-block sample offsets and the queue is forwarded straight to PluginSlot::process, which every loader applies at the event offset (CLAP/VST3/AU sample-accurate; LV2 block-rate by its control-port contract). We do NOT also call set_parameter — that would double-apply each change (once at offset 0, once at its real offset). MIDI is likewise sample-accurate. (A plugin that reads its own params once per block still steps at block boundaries — that is the plugin's rate, not the CLI's.) The block stepper is a deliberate, callback-driven parallel to OfflineRenderHost::render (PluginSlot has no ProcessContext, so it can't reuse the core renderer directly); a block-partition-invariance test guards the two against drift.

Live capture-to-WAV — two modes, both LANDED

pulp run taps the standalone's output boundary into a WAV the offline pulp audio validate verbs read, then exits. Pick the mode by which window you need:

• --audio-capture-wav <file> (earliest, int16). Dumps the EARLIEST window after the stream starts (drop-on-full FIFO). Robust for validate summarize / assert (presence / level / clip / NaN); the wrong window for steady-state doctor and quantization-limited for compare. --audio-capture-frames <n> sets the window.
• --audio-capture-rolling <file> (last-N, float or int24). Keeps the LAST (steady-state) window in a RollingAudioCaptureBuffer and writes a float WAV (no int16 floor) — the window doctor (THD/response) and compare (sub-−96 dBFS residuals) actually want. --audio-capture-rolling-frames <n> sets the window; --audio-capture-rolling-format int24 swaps the float WAV for int24 (≈ −144 dBFS floor, smaller, universal DAW compatibility). Uses the hold protocol so the off-RT materialize is safe while the audio thread is still appending. One capture mode per invocation (mutually exclusive with --audio-inspector / --audio-scope-json / --audio-capture-wav).

WAV writing is pulp::audio::write_wav_file(path, data, WavBitDepth) — Int16 (default overload), Int24, or Float32.

For streamed sampler assets, build the production mip sidecar with pulp audio sampler-mip build <source.wav|source.aiff>. It accepts strict ranged WAV and uncompressed AIFF/AIFF-C, uses the sampler's shared 140 dB decimator, publishes source-and-payload-hash-addressed float32 WAV levels, and atomically publishes the .pulpmip manifest last. --levels 1|2, --max-source-bytes, and --max-output-bytes bound offline production. --json emits ok, source, manifest, payloads, and on failure error; the command exits 0 on success and 1 on failure. Sidecar production does not waive PulpSampler's runtime limit of one or two channels and at most 192 kHz.

In-tree render → WAV bridge (no plugin bundle)

pulp audio render and pulp run --audio-capture-* both need a built plugin bundle / standalone. To get an in-tree Processor (or a raw oscillator wrapped in one) onto disk for the Python lane WITHOUT a bundle, use the scenario-side bridge in "support/wav_bridge.hpp": write_scenario_wav(result, path, WavBitDepth::Float32) (or the write_buffer_wav(buffer, sample_rate, …) overload) deinterleaves a ScenarioResult / Buffer<float> and calls write_wav_file. Float32 is the default so the file carries the exact rendered samples; the deterministic harness makes the bytes reproducible. The pulp-osc-render-wav test tool (test/osc_render_wav.cpp) is the argv surface: it renders the in-tree VcoOscillator (via "support/osc_wav_scenario.hpp") to a WAV (--shape/--freq/--sr/--dur-ms/--channels/--drift-cents/--jitter-cents/--bits), so an offline Quality-Lab script can analyze in-tree oscillator output with no bundle in the loop. The pytest tools/audio/quality-lab/tests/test_osc_wav_bridge.py is the end-to-end readability proof (soundfile loads it, fundamental matches).

Roadmap

The Phase-7 offline-render and live-capture slices have all landed: pulp audio render (offline plugin render, sample-accurate --param @frame), pulp run --audio-capture-wav (earliest-window int16), and pulp run --audio-capture-rolling (last-N, float or int24). The live realtime output tap they read from is gated behind PULP_ENABLE_AUDIO_PROBES (see Live inspection above). The harness's offline/live capture surface is feature-complete; further work is open-ended (e.g. additional analysis verbs), not a tracked backlog.

Sibling: the Audio Quality Lab (reference-vs-candidate perceptual artifacts)

This skill covers presence / level / THD / response. For reference-vs-candidate perceptual artifact detection — "did this DSP change make it sound worse?" (transient smear, dulling, metallic fizz, graininess) — there is a separate opt-in developer/CI tool, tools/audio/quality-lab/ (Python, numpy + soundfile). It is additive: it does NOT change anything here and is never required to run the basic harness or ctest.

• Install (managed): pulp tool install audio-quality-lab — provisions an isolated venv under ~/.pulp/tools/ (the same pulp tool lane as ffmpeg/uv/importers; pulp tool list shows it). Then pulp tool run audio-quality-lab -- <args>. A plain cd tools/audio/quality-lab && pip install -r requirements.txt venv works identically.

• Subcommands (after python -m quality_lab.cli or pulp tool run audio-quality-lab --): run --case {drum,tonal} --degradation <kind> (synthetic case + listenable clips), engine [--input <wav>] --character <c> (validate the REAL stretch engine, reference-free on a dry input), engine-baseline (regression gate: did an engine change make it worse?), corpus list|add (versioned, license-guarded corpus).

• Synth-guidance experiment: tools/rack/fidelity.py patch.vcv --wav render.wav writes only the speaker-bound audio taps, using one shared peak-normalization gain while refusing a silent source and content-binding original Rack voltage levels in the adjacent .fidelity.json sidecar. Run quality-lab analyze for one-render features, then quality-lab catalogue-ab off.wav on.wav --expectations cited.json --experiment experiment.json --holdout-without off-2.wav --holdout-with on-2.wav. The content-bound experiment manifest records prompt, inventory digest, model, a fixed analysis channel, audibility-gate configuration, matched attempt/seed, distinct generation identity, guidance state, pair identity, exact WAV digests, source RMS, and each Fidelity-sidecar digest. Every expectation must name exactly one target/min/max plus a positive tolerance and cite source.path with a page or locator. The A/B turns those physical expectations into normalized errors and reuses goodhart_guard: a working-pair win is not the headline result until an independent holdout pair agrees, and any regression on another named metric or a source level drop over 3 dB refuses the candidate. This proves a bounded guidance tranche before admitting a larger corpus; it does not make reference-free features universal definitions of quality.

• corpus.seed() ships FIVE synthetic families, not just the two stretch-oriented ones: synthetic_drumbreak / synthetic_tonalpad (time-stretch / tonal) plus three oscillator families (family: "oscillator", material_class: "synth") built from osc_fixtures.py's corpus-render helpers (render_static_shapes, render_sync_sweep, render_tzfm_grid) — synthetic_osc_static_shapes (sine/saw/square/triangle at a few pitches), synthetic_osc_sync_sweep (hard_synced_saw swept across master frequencies), synthetic_osc_tzfm_grid (tzfm_sine over a mod-rate/index grid). They gate on the SAME regression_net ratchet as any other family — no bespoke oscillator ratchet — typically via the added-hf axis (aliasing reads as added HF energy) and dsp.null_residual_db's bit-identical clamp (-160.0) for the determinism check. See tests/test_corpus_osc.py.

• Aligns a candidate to a reference (onset-map + local cross-correlation), runs the detectors (transient_sharpness, spectral_centroid, hf_fizz, spectral_flux, hnr — tonal noise/roughness via autocorrelation HNR; plus the standalone stereo_width for image-collapse/phase damage on (N,2) stereo arrays; and the EXPERIMENTAL onset_drift — timing/groove drift via event-time residual after common-latency removal, advisory-only until promoted), and writes a report.json with per-onset localization, coverage/confidence, and provenance.

• Credibility: detectors are validated against an independent textbook phase vocoder (reference_pv.py) AND the real product engine, not just their own synthetic degradation.

• engine/engine-baseline need a built stretchcli. engine.resolve() finds it via, in order: the PULP_STRETCHCLI env-path, a build/examples/offline-stretch/stretchcli found by walking up from the current directory (so the engine path works even when the lab is pulp tool install-ed into a managed venv, as long as you run it from a Pulp checkout that built stretchcli), then the package-relative repo build. Absent → a clean, actionable skipped (build command + env-path), never a failure. Build it with cmake --build build --target stretchcli.

• Maturity gate: each detector has maturity (experimental/beta/stable). An experimental detector runs and reports under the report's advisory block but its fired/low_coverage are excluded from the verdict AND engine_baseline — route every verdict/gate path through schema.detectors_for_verdict / detectors_for_engine_baseline, never re-implement the filter. New detectors ship experimental. (Plan: planning/2026-06-29-audio-quality-lab-postmvp-detectors-maturity-gate.md.)

• Perceptual models (perceptual.py, opt-in, license-fenced): full-reference, music/general-audio quality predictors, each behind its OWN env-path and each skipped-independently when absent — set only the ones you want. evaluate() consults all: ViSQOL (PULP_VISQOL_BIN, Apache-2.0, MOS-LQO), PEAQ (PULP_PEAQ_BIN, GPL, ITU-R BS.1387 ODG), AQUA-Tk (PULP_AQUATK_BIN, GPL-3.0, ODG). Results land under report["perceptual"], ADVISORY ONLY. Deliberately excludes speech/telephony (PESQ, POLQA) and no-reference neural speech (DNSMOS, NISQA) metrics — wrong domain/contract. Three non-obvious facts about the ViSQOL lane, each of which silently breaks a run:

  • ViSQOL reads 16-bit PCM WAV only. Its own src/wav_reader.cc accepts format tags PCM/extensible and rejects any other bit depth with "Error parsing WAV Header - Expected 16bit samples." The lab writes float32 (audio_io.save_wav), so the pipeline's own exports are unreadable by ViSQOL as written; prepare_for_visqol() transcodes to 16-bit PCM first. Do not "simplify" that away.
  • ViSQOL exits 0 when a comparison FAILS. main.cc logs the error and continues the file-pair loop, returning 0 regardless; only argument-parse and init failures return -1. So the exit code cannot distinguish scored from unscored, and the score line is the only evidence a comparison happened.
  • Parse stdout ONLY. ViSQOL prints MOS-LQO:\t\t<value> to stdout (outside the --verbose block — --verbose adds paths and per-band tables, it does not gate the score) and routes diagnostics to stderr. A bare-float fallback applied across stderr turns an incidental number — a duration, a version, a path segment — into a confident, fabricated MOS for a run that produced no score.
  • Reachability: the perceptual layer runs only from pipeline.run_and_export(). compare and regression_net never touch it, which is where most readers look first.

• MIR structural oracle (mir.py, opt-in, license-fenced): aubio (PULP_AUBIO_BIN, GPL-3.0) — a feature extractor, NOT a quality metric, so it is a SEPARATE layer from perceptual.py. Gives an independent onset-count cross-check under report["advisory"]["mir_oracles"] to non-circularly validate onset_drift. ADVISORY, never a gate/baseline. License fence for both layers: GPL tools (PEAQ/AQUA-Tk/aubio) are reached ONLY via a developer-set env-path over a process boundary, never bundled, fetched, linked, or added to tool-registry.json — so no copyleft obligation attaches and no runtime warning is needed (consistent with how the lab already treats PEAQ).

• Advisory reviewer (reviewer.py, opt-in PULP_QLAB_REVIEWER_CMD subprocess, run --review): an LLM/multimodal model names artifacts in plain language under advisory.reviewers. ADVISORY ONLY — never changes the verdict/gate; skip-when-absent; no network by default. Validate with reviewer.score_agreement (precision/recall vs the synthetic answer key) before trusting it.

• Tuning loop (loop.py, quality-lab loop, EXPERIMENTAL): scores candidates with the gate-participating detectors (experimental detectors excluded), ranks them, and writes PROPOSAL-ONLY label updates to corpus/LABEL_PROPOSALS.json — never MANIFEST.json, never auto-promotes. goodhart_guard refuses a candidate that games one detector while regressing another (normalized Pareto + held-out slice + NEEDS-EAR).

• Agent compare report (compare.py, quality-lab compare before.wav after.wav): the agent-facing measure → compare → judge surface. Level-matches, runs one curated axis (--profile), and emits a typed evidence envelope + an action-oriented verdict (regression_suspected / material_change_detected / no_material_change_detected / inconclusive / invalid). Six axes today — a new axis is one _AXES registry entry in compare.py, the shared _measure/_verdict machinery does the rest:

  --profile	axis	measures	bad direction (bad_sign)
  tonal-balance	tonal_balance	LTAS spectral-centroid shift	duller (−1)
  added-hf	added_hf	band-relative ≥8 kHz fraction ratio (dB)	added fizz (+1)
  noise-roughness	noise_roughness	harmonic-to-noise ratio drop (dB)	rougher/noisier (−1)
  graininess	graininess	relative spectral-flux increase	grainier (+1)
  stereo-width	stereo_width	RMS(side)/RMS(mid) width + interchannel correlation	narrower/collapsed (−1)
  transient-integrity	transient_integrity	per-onset attack-smear deficit (onset-aligned)	softer/smeared (+1)

  noise-roughness (HNR, pitch range 40–1000 Hz so bass fundamentals count) and graininess (relative mean_spectral_flux increase) are meaningful on tonal/sustained material — a caller-declared contract (the caller picks the profile), carried as a standing summary caveat + both per-signal scalars in the payload, NOT an automatic tonal/percussive classifier in the honesty path (which would just be a tunable threshold on the same statistic). Only mathematically degenerate inputs go not_applicable (a flux reference below an epsilon floor — the div-by-zero edge). stereo-width is the one axis fed the ORIGINAL 2-channel signal (via audio_io.load_wav_multichannel, the _Axis.needs_stereo flag, and the _measure_stereo path) rather than the mono downmix; mono input on either side is not_applicable, its metric is level-invariant (no level-match), and a candidate whose correlation goes negative is flagged out of phase (mono-incompatible) in the summary. transient-integrity (the _Axis.needs_onsets flag) is "did my DSP soften/smear the attacks?" — it detects+matches onsets (align.detect_onsets / map_onsets), locks each attack (dsp.local_align), and compares the high-band attack rise (dsp.attack_rise, the SAME primitive the transient_sharpness detector uses — extracted to dsp so there is no forked DSP). One-directional (a softening is the regression; a sharper candidate reads no change); needs onset-bearing material (< 3 matched onsets → not_applicable); self-aligns per onset so --align is a no-op and the global sample-domain advisory is suppressed. That takes compare to 6 of the lab's 7 detectors reachable from the agent-facing surface (up from 2). compare.STEREO_PROFILES / ONSET_PROFILES mark the capability-specific axes, and NET_DEFAULT_PROFILES (the regression net's default) excludes them so they don't emit spurious not_applicable rows on the wrong material — a suite opts in. compare.MONO_PROFILES / compare.STEREO_PROFILES split which axes need 2-channel input — the regression net defaults to the mono set so a mono comparison doesn't emit a spurious not_applicable stereo-width row.

  The added-hf axis measures the ratio of the ≥8 kHz energy fraction (10·log10(frac_cand / frac_ref), default ±3 dB), NOT the absolute fraction delta — the absolute delta is signal-dependent and effectively blind on a bass-heavy source (an amp render's HF fraction is ~1e-4, so even a clearly harsh addition barely moves it; only the null-residual corroboration caught it before). The ratio is invariant to a broadband gain / the level-match, so it flags a real harshness change on a dark source that the absolute delta missed. Both band fractions are floor-clamped, so a zero-HF reference (fizz on a dark source) or candidate (brickwall low-pass) gives a large finite dB delta, never inf → a false invalid. It flags HF loss as material too (a negative dB delta) — only added fizz is the bad direction. A large EQ move outside the band (e.g. a big LF shelf) still leaves some residual sensitivity — far less than the absolute metric, not zero. The threshold is a dB magnitude with its own per-axis range, so the shipped CLI / MCP pass any positive threshold through to the Python registry (the single source of truth for the valid range) rather than a hardcoded fraction bound.

  Intent-safe: regression_suspected needs --reference-role golden AND a change in the axis's bad direction (a duller candidate can't trip the added-hf axis; added fizz can't trip it on a peer compare); otherwise the change is the neutral material_change_detected. Advisory: exits non-zero only on invalid (couldn't measure), never for a judgment — the gate primitive stays pulp audio validate compare. Each axis has its own materiality default (--threshold overrides). Report shape (quality_lab.compare.v1) is owned by schema.py; every envelope carries status/applicable/materiality/provenance. Use this when an agent must weigh in on a DSP change with cited evidence rather than a bare pass/fail.

  Corroboration + raw comparators (report["advisory"], off-gate). When the primary measurement succeeds, the report also carries an advisory namespace: a deterministic null_residual raw comparator (level-matched sample-domain residual RMS relative to the reference, db_rel_reference; lower = more identical) and a corroboration block. Corroboration is a materiality cross-check — NOT a trust score: it reports only whether that independent, algorithm-agnostic residual also registers a material change (corroborated / not_corroborated), under the same level-matched global contract. Both are maturity: experimental, participates_in_verdict: false — they NEVER move the verdict. The valuable signal is disagreement: not_corroborated with axis_exceeds:false, raw_material:true means a real sample-domain difference this axis can't see (e.g. a pure delay — try another profile); axis_exceeds:true, raw_material:false means a marginal/phase-only change to treat with more caution. That disagreement is now PROMOTED to the headline (it used to sit buried in the advisory): a top-level report["headline_flags"] (always present, empty when there's nothing to flag) carries a structured {flag, detail, expected_for} — uncaptured_material_difference or axis_change_without_residual — and a matching sentence is appended to summary. The flag is STRUCTURED, not prose, precisely so the known false-alarm class is machine-suppressible: every time/pitch-variant A/B (chorus, phaser, bendr, any modulated effect) legitimately reads not_corroborated forever because the phase-sensitive residual always disagrees with a tonal axis, so uncaptured_material_difference carries expected_for:["time_variant_processing"] — a caller doing time-variant work filters on that rather than treating the flag as noise. It still NEVER moves the verdict. Length is handled honestly: the candidate is level-matched over the common region and the shorter signal zero-padded to the longer length, so a dropped/added tail with content raises the residual on its own (reads material) while trailing silence contributes ~0 (stays immaterial) — a truncated render can never masquerade as a near-identity match, and the sample counts are reported in the comparator's detail. The residual is phase/delay-sensitive and alignment-free by contract, so it corroborates materiality, never audibility. Cite it — never treat agreement as a confidence score (the plan's explicit non-goal: agreement ≠ trust). Deliberately NOT built: the GPL aubio / AGPL Essentia feature-extractor menu — feature extractors are never verdicts, and an env-path license fence is disproportionate surface for the value; the pure-numpy residual carries the corroboration story license-free.

  Time-alignment (--align latency). The axes are alignment-free, so a constant delay/offset (delay plugin, tempo-sampler start offset) doesn't move the tonal verdict but makes the phase-sensitive null-residual false-alarm not_corroborated. --align latency estimates one constant lag (dsp.estimate_global_lag — normalized cross-correlation of the attack envelopes), trims to a common time base (dsp.apply_lag_trim), and measures the aligned pair: a pure delay then reads no_material_change + corroborated, and a real change behind a delay is measured cleanly. The outcome is disclosed on the envelope (alignment: {policy: "fixed-latency-trim", lag_samples, confidence, applied}) and the summary. It refuses below a confidence floor (records policy: "not_aligned", measures unaligned) — a wrong alignment is worse than none. Default --align none = zero behavior change. --align threads through compare_files → CLI → MCP (pulp_audio_compare's align param) → the /audio-compare slash command.

  The --align grammar + per-mode dispatch live in alignment.py (extracted from compare.py in Tier 3 / T3.1): alignment.parse("mode[:param]") → AlignSpec and alignment.apply(spec, ref, cand, sr, *, needs_stereo, needs_onsets) → (ref, cand, record), dispatched through a _HANDLERS dict so a new warp class is one entry. alignment.py imports schema + dsp (never compare — capability facts arrive as booleans; no import cycle); the record shape stays in schema.py; the prose lives in alignment.describe(record). Not to be confused with align.py (onset primitives) which the onset-anchored warp handlers consume.

  --align varispeed:R (Tier 3 / T3.1) — a caller-declared tape-style speed change (candidate = reference resampled by R, pitch+time coupled). A resample is EXACT for this class, so the handler resamples the candidate back to the reference length (dsp.resample_to_length — a pure-numpy Kaiser-windowed-sinc resampler, no scipy) and the WHOLE alignment-free pipeline including the sample residual applies (clean varispeed → no_material_change + corroborated). It verifies the declared ratio against the observed duration ratio (±2%) and refuses (not_aligned) otherwise — never resampling to a wrong length. Anti-masking holds (a defect behind the varispeed still flags). Honest physical limit: a speed-up (R<1) bandlimits to a lower Nyquist, so near-Nyquist energy is legitimately lost (correct varispeed behavior, matches an ideal brickwall to ~1% — not a resampler artifact).

  --align stretch:R (Tier 3 / T3.2) — a declared PITCH-PRESERVING time-stretch (PV/bendr). No exact inverse exists, so the axes measure the UNWARPED pair directly (LTAS centroid / HF fraction / HNR / width are time-averages → warp-invariant). Two axes are warp-normalized off the ratio, threaded via the AlignSpec: graininess is warp_aware (_Axis.warp_aware) and measures the candidate flux at mean_spectral_flux(..., hop_scale=R) (spectral_flux.v2-warp — else a clean stretch reads a false "smoother"); corroboration binds to dsp.ltas_log_spectral_distance_db (a phase-blind, length-independent ltas_residual comparator) because the sample residual is invalid across a stretch (the null residual is still emitted, marked not-a-corroborator). _align_stretch returns the pair UNCHANGED after verifying §6.1 (duration within ±3% of R, R∈[0.25,4]) and §6.3 that a single uniform ratio fits — onset material: MAD of the per-onset residual from the ref_t·R prediction ≤ 15 ms with coverage ≥ 0.5; sustained: ratio-mapped smooth_energy_env correlation ≥ floor — else REFUSE (not_aligned, "non-uniformly warped"). The _WARP_MODES set gates the capability short-circuit (a warp is applied/routed for onset/stereo axes, not skipped like a constant lag); _MEASURE_UNWARPED_MODES = {stretch, pitch} drives the per-axis warp normalization + corroborator swap (varispeed is NOT in it — it resamples back, so the standard pipeline applies verbatim).

  --align pitch:S (Tier 3 / T3.3) — a declared DURATION-PRESERVING pitch shift (S semitones, parsed with an optional st/semitones suffix; spec.param is the signed semitones). Time base unchanged, so the axes measure the pair directly; tonal-balance (also warp_aware) compensates the LTAS centroid via dsp.pitch_compensated_centroid_shift (a perfect shift moves the centroid by the pitch ratio → the axis reports deviation-from-expected, spectral_centroid.v2-pitch), and the corroborator shifts the reference LTAS to its expected position first (dsp.ltas_logfreq_shift, via the ref_shift_ratio arg on ltas_log_spectral_distance_db). _align_pitch verifies |S| ≤ 24 st

  • duration preserved (a length change ⇒ not a pure pitch shift) and refuses otherwise. The warp-aware kernels take the accepted AlignSpec (spec=) and each reads only its own class off spec.mode/ spec.param — the generalized threading that replaced T3.2's single flux_hop_scale param. Known advisory imperfection: the log-frequency LTAS compensation carries bin-interpolation error on discrete partials, so a clean shift can read not_corroborated (advisory only, never the verdict). a non-constant warp / DTW stays on the engine path, not compare.

  --align ratio:auto (Tier 3 / T3.4) — ESTIMATE a uniform stretch ratio, double-gated (§6.2): two independent estimators — the duration ratio and the onset-time slope (dsp.theil_sen_slope over map_onsets pairs, ≥ 6) — must agree within 2%, else REFUSE (not_aligned, "ratio estimators disagree" / "needs ≥ 6 matched onsets"). On agreement _align_ratio_auto delegates to _align_stretch with the onset-slope estimate (interior evidence) and tags the record estimated=True + both estimator values. The estimated ratio reaches the warp-aware axes via alignment.effective_spec(spec, record), which resolves a ratio:auto spec to stretch:<estimated> so graininess hop-scales off the ACCEPTED transform, not the "auto" request string. Reliable on onset-bearing uniform EXPANSIONS; compressions (map_onsets drift) + non-uniform + sustained refuse (conservative — declare stretch:R). This is the last declared-warp class; a non-constant warp / DTW stays on the engine path, not compare.

  Honesty disclosures (what compare admits it can't see). Every axis EXCEPT stereo-width mean-downmixes to mono, so on those a stereo/spatial change (widener, panner, M/S) is invisible. When a file was multichannel the report says so on the mono axes: provenance.ref_channels/cand_channels, a downmix note on the measurement envelope, and a one-line summary clause — a stereo-widener whose mid is unchanged reads no_material_change on tonal-balance but the report never hides that imaging was discarded, and pointing the same pair at --profile stereo-width catches the change directly (the downmix note is correctly suppressed there, since that axis DID read the stereo image). The added-hf axis reports not_applicable (→ inconclusive) at low sample rates where the ≥8 kHz band collapses to a degenerate handful of LTAS bins (e.g. sr = 16 kHz is the single Nyquist bin) instead of a confident "no change" over nothing. A DC offset concentrates energy in LTAS bin 0 and drags the centroid down — reading as false tonal "dulling" — so the advisory carries a dc_offset raw comparator (per-signal means + present flag) and, when present, a summary sentence telling you to high-pass before comparing rather than chase phantom dullness. All three are presentation/advisory only — like corroboration, they never move the verdict.

• Shipped surfaces (same measurement, three entry points): the Python quality-lab compare is the dev-loop surface; pulp audio compare <ref> <cand> [--profile …] [--reference-role …] [--json …] is the shipped CLI verb (thin delegator — tools/cli/cmd_audio_compare.cpp locates the opt-in managed tool and forwards, so no DSP links into the CLI); pulp_audio_compare is the MCP tool mirroring it (returns the report JSON). All three run the identical axis logic. The /audio-compare slash command wraps the CLI. Install the tool once: pulp tool install audio-quality-lab.

• Golden-render regression net (quality_lab.regression_net, quality-lab regression-net --manifest net.json). The daily-driver loop: keep a golden render per plugin/preset, pulp audio render --plugin … the candidate, and compare across every wired axis, emitting a table (plugin | profile | verdict | corroboration | flags). Fail policy is the contract: the fail signal keys off axis verdicts only — exit 1 iff an axis reports regression_suspected; a pair that couldn't be measured (missing/corrupt render → invalid, or a bad manifest) is a separate ERROR (exit 2), never greenlit as clean; exit 0 = all measured, none regressed. The corroboration column is informational and never affects the exit code — the modulated family (chorus/phaser/flanger/tremolo/ring-mod) is time-variant so it reads not_corroborated forever (the expected_for:["time_variant_processing"] flag). This is advisory reporting attached to a change; gating proper stays pulp audio validate compare. The module is the reusable REFERENCE — a plugin suite (pulp-classic-effects, bendr, GPU NAM) wires its own renders into run_net and commits a portable manifest (paths resolve relative to it). Read the change class, not the backend: timbral + modulated changes are valid through the net; bendr fixed-ratio A/Bs are valid, but bendr ratio changes / tempo-sampler offsets are time-misaligned (the residual false-alarms) → use the reference-free engine path instead. See the guide's "Golden-render regression net" section.

• Live plugin before/after (measure a real plugin, then judge the change). compare needs two WAVs; capture them from a live/hosted plugin with the existing steady-state tap, then judge:

  # capture the KNOWN-GOOD baseline (announce audio + cap duration per the local-dev etiquette)
  pulp run --plugin before.clap --audio-capture-rolling before.wav --headless   # tear down promptly
  # …apply the DSP change, rebuild…
  pulp run --plugin after.clap  --audio-capture-rolling after.wav  --headless
  pulp audio compare before.wav after.wav --reference-role golden               # base = golden
  

  Use --audio-capture-rolling (last-N steady-state window — the region compare wants), NOT --audio-capture-wav (earliest int16 window). --audio-capture-rolling opens the live audio device: announce before launching, cap the wall-clock, and tear down (see the local-dev audio etiquette in CLAUDE.md). For a fully offline capture of an explicit bundle, pulp audio render --plugin <bundle> --out cand.wav … writes a WAV without a live device — feed that to compare the same way. This is the "measure a plugin with the scope/inspector and compare findings" path.

• Guide: docs/guides/audio-quality-lab.md; module map + deferred-detector status: tools/audio/quality-lab/README.md.

Compare/measure design rationale & non-goals (why we built it this way / what we skipped)

Durable "why / watch-out-for" notes so this isn't re-litigated (rationale, not workflow history):

• A vertical agent report before a shared measurement record. The harness + lab + inspector already measure; the gap was orchestration + a defended verdict. A universal {ref,cand,delta} scalar is too lossy to defend a judgment, so measurements are typed evidence envelopes and curves/THD stay first-class. (Two independent adversarial reviews converged on this.)
• Cross-tool "agreement = trust" is NOT a signal and is deliberately absent — different algorithms disagree for legitimate reasons; treating agreement as trust manufactures overconfidence. If ever added, it reads as "corroborated under a comparable-axis contract."
• Doctor --response/--thd are NOT valid on arbitrary before/after WAVs — --response is a peak-normalized self-spectrum (not a transfer response); THD is gate-grade only for a steady bin-coherent sine. They need a controlled stimulus, so they're excluded from compare and join later behind stimulus-supplying profiles.
• Alignment is not_required for both current axes — global LTAS metrics (centroid shift, HF fraction) are alignment-free; the envelope records the policy, we don't build general alignment for them. A future transients/timing axis will need a real alignment policy first.
• Axes are a per-profile registry, added one entry at a time — each axis declares its axis/tool/bad_sign/default_threshold/kernel/summarize; the shared _measure does applicability + level-match + materiality and _verdict does the intent-safe mapping. bad_sign is the sign of the delta that is a regression against a golden reference (−1 = a drop is bad, +1 = an increase is bad). Only axes on the same contract (global, delta-in-(0,1), self- applicability) belong here — stimulus/alignment-dependent measures do NOT.
• regression_suspected requires --reference-role golden — "worse" assumes the reference is the known-good baseline; without that, a change is neutral. It ALSO requires the change to be in the axis's bad direction (bad_sign), so a legitimately-brighter or HF-reduced candidate reads as material_change_detected, not a false regression.
• Axis thresholds are per-axis and unit-bound to (0, 1) — every current axis's delta is a dimensionless ratio in (0, 1), so the threshold guard is 0 < t < 1. A dB-unit axis (e.g. HNR drop) is deliberately deferred until the threshold validation is made per-axis; don't shove a dB metric onto the (0,1) contract.
• Speech/perceptual & feature-extractor scope is bounded — PESQ/POLQA/DNSMOS/NISQA are speech/ telephony (out of scope for full-reference music); aubio/librosa/Essentia are feature extractors, not MOS predictors (advisory raw-comparators behind profiles, never verdicts); scalar-only where the scalar is the calibrated deliverable (PEAQ/AQUA-Tk ODG). GPL/AGPL tools stay behind the env-path license fence.
• audio_io.load_wav mono-downmixes — fine for tonal balance; a future stereo-image axis must NOT use that loader.
• detectors/click.py refuses a REAL BLEP/BLIT oscillator at a fractional period — do not read that refusal as a click. The detector's comb self-reference (delay one period, subtract) nulls a bandlimited oscillator to the interpolator floor ONLY when successive periods are identical up to the fractional delay. That holds for an exact additive fixture (sinc-periodic), which is exactly why a click floor measured on a synthetic fixture does NOT transfer to a real oscillator: on a polyBLEP saw/square, sr/f0 is fractional, so the discontinuity lands at a different sub-sample phase every period and the comb leaves a per-edge approximation residual that reads a false −20 to −30 dB "click" even when the render's alias floor is −55 dB or lower. detect() discriminates that regime with two measurements — template_novelty_db and edge_concentration — and REFUSES (low_coverage, fired=False, a third refusal precondition alongside low period-confidence and pitch drift) rather than false-firing. template_novelty_db aligns the comb residual into per-period frames at the fitted fractional period, builds the median-frame TEMPLATE, and scores the worst period's departure from it over the median period's — a one-off seam is the single period that does NOT match the recurring template (a large outlier), while the smear repeats the same per-edge shape every period so it barely departs. The departure is scored against a HIGH percentile (90th) of the per-period departures, NOT the median — because at a HALF-INTEGER period (sr/f0 frac ~0.5) the BLEP edge alternates sub-sample phase A/B/A/B (true residual period 2P), so HALF the periods depart from the per-P template: a median denominator reads that recurring half as a lone outlier and FALSE-FIRES on a clean oscillator, flipping with period-count parity (render length). A high percentile puts any recurring cluster over ~10% of periods (2P halves, 3P thirds) INTO the denominator, so only a genuine one-off seam stays a large ratio. It is a per-period RATIO, so it is render-length INDEPENDENT (a longer clip only adds more template-matching periods — it moves neither the worst nor the percentile): the same near-floor seam that a global statistic dropped on a multi-second render now fires at any length, at integer AND half-integer periods. edge_concentration (cosine similarity of |residual| to |diff(y)|) is kept, but ONLY to split smear (rides the waveform's own edges) from a block-rate zipper (off the edges) — novelty alone reads a stationary zipper as low-novelty and would wrongly refuse it. A refusal is NOT a pass: the honest gate for a discontinuous real oscillator is a frozen-reference null (dsp.null_residual_db — render the same patch with the offending parameter held frozen), not the comb self-reference. When you add a click fixture for a new oscillator, grow a REAL polyBLEP render (osc_fixtures.py), not just a synthetic sum — the synthetic-only fixture is precisely the blind spot that let the false-fire ship.

Proving reported latency

expect_reported_latency(result) measures the delay actually present in a render and compares it against Processor::latency_samples(). This is the contract for any processor with nonzero latency: the host slides the whole track by the number the plugin reports, and nothing else checks it. RenderScenario collects the report facts on every render (ScenarioResult::latency), so a contract can ask for the proof without re-rendering.

Do not reach for this on a processor that has no latency. Most don't: gain, EQ, compressors without lookahead, waveshapers, delay effects, and ordinary synth paths correctly report zero, and rendering them to prove 0 == 0 is ceremony. PulpSampler is the sampler exception: enabling its synthetic heritage two-leg SRC produces derived, profile-dependent latency, while an all-bypassed profile reports zero. That heritage path belongs in the same measured-latency proof category as FFT size, partition scheme, lookahead window, or oversampling ratio because a refactor can silently change the number. User-facing rationale, including when NOT to use it: docs/guides/latency-proof.md.

The spectral analyzers fail closed — a refusal is the answer, not a bug

measure_thd, measure_aliasing, and magnitude_spectrum_curve throw rather than return a number they cannot stand behind: silence (no fundamental to be relative to), a capture shorter than fft_length (zero-padding would measure the truncation edge), a fundamental at or above Nyquist, and — the one that surprises people — an alias series that never reaches Nyquist. Size num_harmonics from the sample rate: the default 64 models no alias site at all below ~375 Hz at 48 kHz, and that used to read as clean on a maximally aliased saw, because "no aliases among the zero places I looked" is technically true. Through the CLI a refusal is exit 2, distinct from a failed check.

Every one of those paths previously returned a confident number that passed a gate. If a refusal surprises you, the input is usually wrong — not the tool.

Measuring an oscillator's pitch — estimate_pitch, not estimate_frequency

estimate_frequency (audio_metrics.hpp) is a zero-crossing detector and its own doc disclaims harmonically-dense material. A real oscillator IS dense: on a bright tone it locks to a harmonic (measured ~1900 cents high on a formant waveform), so it cannot measure a VCO's drift or a DCO's few-cent quantization error. Reach for estimate_pitch / track_pitch (pitch_track.hpp) instead: a coarse magnitude_spectrum_curve peak seeds a golden-section refine over the shipped fit_tone projection, so it is leakage-free and sub-cent (<0.002 cent on a clean tone, coherent or not) and FFT-backend-stable. It refuses silence and noise rather than octave-guessing — honor the confidence gate. track_pitch gives the f0(t) trajectory; the drift/jitter statistics over it (Allan deviation, Theil-Sen slope) stay in the Quality Lab per the C++/Python split. A steep glide needs the window short relative to the sweep.

Harmonic-dominant material — don't assume the loudest bin is f0. A rolled-off or resonant oscillator can carry MORE energy in an upper partial than in the fundamental, so the loudest FFT bin lands an octave/twelfth/higher above the true pitch. estimate_pitch recovers f0: it ranks the coarse peak against its subharmonics (to coarse/8) and adopts the lowest root that BOTH explains the segment (harmonic-comb energy within a margin of the best) AND carries real energy at its OWN fundamental tooth. That own-tooth test is load-bearing — explained energy alone cannot tell f0 from f0/N (a near-pure tone lets any subharmonic refine one of its harmonics onto the tone), and only the TRUE root has a partial sitting at the root itself. The tooth floor is LEAKAGE-AWARE, and it has to be: a fixed floor is defeated by rectangular-window LSQ leakage from the loud coarse peak into the f0/2 fit, and that leakage GROWS as the analysis window shortens (energy ~ (1/(π·Δf·T))²). With a fixed floor a plain bass saw over the DEFAULT window holds only a handful of periods and reads a confident octave (or two) DOWN — the ordinary case, worse than the octave-up it was meant to fix. So a candidate's tooth must clear max(absolute-noise-floor, expected coarse-peak leakage at its Δf over this window); a genuine fundamental far below a loud harmonic (Δf large) or over a long window (T large) still clears it, a leaky saw subharmonic over a few-period window does not. Corollary: a genuinely faint fundamental is not resolvable below the leakage floor at a short window — use a longer analysis window (the DCO/VCO quantization/drift fixtures already do, n≈2^15) rather than trusting a low bass note over a 4096-sample frame. Do NOT "fix" a wrong octave by lowering the floor — it re-opens the octave trap. One honest residual, NOT a refusal: a genuine missing-fundamental comb (energy only at 2·f0/3·f0, nothing at f0) has no tooth at f0 to lock, so the estimate stays on the loudest PRESENT partial (2·f0) — the frequency actually there, never a fabricated virtual f0; likewise the descent stops at coarse/8, so a 9th-harmonic-dominant signal (rare) reports a lower harmonic. Both return a real present frequency, never fail closed. (Regressions fixed 2026-07: octave-UP on harmonic-dominant material — old guard stopped at coarse/4 on a fixed 10% cliff; then octave-DOWN on a plain short-window saw — fixed-floor leakage, closed with the leakage-aware floor above.)

Never gate on detection_floor_db

It is a ~2σ bound that assumes a white residual. Aliases are discrete tones, so it is wrong exactly where it is most tempting, and asserting it is the analyzer grading its own homework. Its one legitimate use is deciding whether a reading is conclusive at all — a gate means something only while the measured value clears the floor.

Prove a floor with a negative control: a fixture whose alias content is zero by construction must read collapsed, and injected impurities of known level must come back at that level. Prefer tone_residual_db (least-squares projection) to reading a windowed spectrum near a loud tone — it sidesteps leakage entirely. Window floors are measured, not assumed: flat_top is amplitude-accurate at about −93 dB with a flat skirt and cannot gate −100 dBc; only kaiser at β=14 resolves it, and even then not at bins 1–3, where the DC-removal pedestal dominates regardless of window.

The schema and both evaluators live in pulp::audio-analysis (latency_evidence.hpp) as pure functions over buffers, so the test harness, the pulp CLI, and MCP all produce the same verdict. Add a policy there, never in test/support, or the surfaces drift.

Feed it a broadband, aperiodic stimulus. make_white_noise for the delayed-null policy, make_impulse for the marker policy.

The refusals are the feature, not a limitation. A stimulus that cannot pin the delay down returns not_measurable / inconclusive, and an inconclusive result that was asked for fails the gate — an unprovable claim is a failed claim, never a silent pass. Expect a refusal when:

• the stimulus is silent, or the output is not a delayed copy of the input (the delayed-null policy needs the processor in a declared identity / bypass / fully-dry mode — set that up with set_param);
• the stimulus is periodic on an integer sample count. This is the one that bites. Tile-a-cell noise with period 100 against a true delay of 512 makes delays 12, 112, 212… bit-identical; the sweep's argmin walks upward and would confidently return 12, falsely accusing an honest processor of misreporting by 500 samples. The guard catches competing minima and refuses. Note this is about integer-sample periodicity, not tonality — a 440 Hz sine at 48 kHz has a 109.09-sample period, so its delay IS recoverable and is proven;
• the processor's report moved mid-render, including moving away and back. The harness watches consume_latency_changed_flag() as well as polling the value, because a host notified mid-render has already acted on the intermediate number. (Consuming the flag is safe here: a headless render has no adapter to steal it from.)

Read the confidence numbers, not just the verdict. The evidence carries null_depth_db (how completely the delayed input explained the output) and ambiguity_margin_db (how much worse the best competing delay scored). Measured references: a pure delay line nulls to the -200 dB floor with a 203 dB margin; SpectralFrameEngine in identity mode — real STFT, real overlap-add rounding — nulls to -137 dB with a 140 dB margin. Both are nowhere near the -60 dB floor or the 12 dB ambiguity bar. A pass with a small margin technically cleared the bar and is still a coin flip waiting to happen — treat it as a finding and say so.

A pass proves self-consistency, not the right value. The audio is delayed by exactly what the processor reports — which is all the host needs, and is why it sounds correct. It does NOT prove the latency is what it should be: double an FFT size and the processor honestly reports its new, larger delay, passes the contract, and has silently added 43 ms to every session. apply_expected_samples (C++) / --latency-expect (CLI) / latency_expect (MCP) pins the intended value so that drift fails too. It never masks a real mismatch — an audio-vs-report disagreement is still reported as that.

Dogfood reference. test/test_latency_contract.cpp proves the contract against SpectralFrameEngine (true latency fft_size + analysis_hop = 2560), not just against a delay line, and a companion case deliberately misreports by one analysis hop to confirm the catch. Copy that shape when covering real DSP.

A latency contract can pass while the stream start is wrong. The SpectralFrameEngine latency dogfood above passed at -137 dB on a noise stimulus while the engine was swallowing the first fft_size - hop samples after every prepare()/reset() — a contract proves where the audio is, not that the head of the stream survived. That defect shipped: the first ~85 ms of every playback start or seek through SpectralFrameEngine were tapered (impulse at sample 0 → -300 dB, 1024 → -24.8 dB, 2048 → -3.5 dB at FFT 8192 / hop 2048) because the first frames ended at sample fft_size - 1 and the startup normalisation floor attenuated the partial overlap. The engine now starts its frame grid at first_frame_start() (the last hop multiple above -fft_size, implicit silence before 0), so sample 0 has full overlap and the reported latency is unchanged; spectral_ola_window_energy() states the coverage at any position without running audio. Prove stream starts with positioned stimuli — impulses at 0, 13, 512, 1024, 2048, 6000 and a kick at 0, after prepare AND after reset — nulled against the input delayed by the reported latency (test/test_spectral_frame_engine_stream_start.cpp). Do not prime the engine with silence app-side any more: it is redundant, and it now emits real frames, so it costs FFTs on the audio thread's reset. A split-API caller that maps frame ordinals to time itself (RealtimePitchTimeProcessor) opts out with SpectralFrameEngineConfig::full_overlap_stream_start = false and keeps the tapered start.

Proving an SOS cascade executor

For a runtime cascade, do not use its own DF2T recurrence as the oracle. Compare impulses against an independently written high-precision direct-form-I reference, and compare steady-state sine gain against direct complex evaluation of the SOS polynomials. Cover every supported design family, float and double execution, caller ordering/internal headroom, coefficient replacement and reset tails, non-finite/unstable/over-capacity transaction failures, denormal decay, and an allocation probe around the prepared runtime surface. Build and time the largest supported product in Release; Debug timing is not performance evidence.

Sample-region proof surface

For sample-region DSP, pair the declared scalar-kernel contract with a focused render proof through the canonical graph executor. Use an explicit UnitDelay for feedback, test regular and irregular callback schedules, and record the independent scalar oracle, source head, executable, artifact, and log hashes. A registered or generated proof without an executed receipt does not establish reachability; retained-state edits also need a positive unchanged-identity case and a negative changed-identity case.