feat(pipeline): single final normalization just before write and display #49

Merged

ykp merged 1 commit from final-normalization into master

2026-09-30 07:09:33 +02:00

ykp commented

2026-09-29 22:14:49 +02:00

Owner

Operator directive: "Find all normalization steps in the code and optimize for a single final normalization just before write and display."

The normalization inventory (what was found)

Step	Location	Semantics
Double-chirp base normalization	`chirp.generate_double_chirp` (ticket 41b)	the summed base scaled to peak 1.0 before the repeat — only that one waveform
CFR per-pass renormalization	`cfr.cfr_reduce`	each pass scales back to the pre-pass peak (spec §5.3.1 — kept)
Comb block normalization	`comb.generate_comb`	`peak == max(\|scales\|)` — the Level definition for tones (kept)
Writer clip	`sig_io.write_sig`	`clip(round(v·2¹⁵))` — over-full-scale signals clip (distortion)

The gap: only the double chirp normalized — every other over-scale combination (AGWN's ~4σ noise peaks, Level-100 waveforms after the band/notch filter's passband ripple nudged the peak) hit the writer's clip or landed off-scale.

The fix — one normalization point

pipeline.apply_post_chain ends with a scale-down-only peak guard (after windowing, before returning — both the SigV3.1 write and the preview display see it):

peak > 1.0 + 1e-9 → scaled to exactly full scale (no clipping; the 41b semantics generalized to every waveform)
at/below full scale → the array passes through untouched (same object) — Level = % of full scale is preserved (a Level-50 comb stays at 0.5; a Level-100 chirp sits at its post-filter peak, now pinned to 1.0)
the 1e-9 tolerance: a 1-ulp float-noise overshoot (a unit tone peaks at 1 + 2e-16) must not trigger a pointless renormalization — the writer's 1-ulp clip is invisible
all-zero signals skipped; PAPR/PSD measurements are scale-invariant; byte-stable for identical inputs

chirp.generate_double_chirp drops its base-level step (subsumed; the COLA repeat is linear, so the later division is equivalent) — the generator is scale-faithful. CFR's per-pass renormalization stays (its own §5.3.1 semantics).

Test updates (the pipeline contract changed by directive)

test_pipeline: the pinned fixtures move to sub-full-scale amplitudes (0.3·tones, 0.2·noise, /1500 arange) so the no-copy passthrough and bitwise contracts stay meaningful (the guard is a no-op at/below full scale)
test_double_chirp: the single-block test asserts the raw phase-optimised sum (scale-faithful generator); the normalization test moved to the pipeline level (raw over-scale → chained peak == 1.0, single-block and repeated)
AGWN's in-band-RMS tests call the generator directly — unaffected; the full-chain AGWN is now clip-free (the RMS-referenced level yields to no-clipping — flagged for operator awareness)

Spec

New §5.3.5 Final normalization; the §5.3 chain diagram gains the step; §5.2.1c's level semantics point at the single guard.

Gate

pytest: 1288 passed / 64 skipped (headless tier), coverage 93.62 %
ruff + mypy: clean

Operator directive: *"Find all normalization steps in the code and optimize for a single final normalization just before write and display."* ## The normalization inventory (what was found) | Step | Location | Semantics | |---|---|---| | Double-chirp base normalization | `chirp.generate_double_chirp` (ticket 41b) | the summed base scaled to peak 1.0 **before the repeat** — only that one waveform | | CFR per-pass renormalization | `cfr.cfr_reduce` | each pass scales back to the **pre-pass peak** (spec §5.3.1 — kept) | | Comb block normalization | `comb.generate_comb` | `peak == max(\|scales\|)` — the Level **definition** for tones (kept) | | Writer clip | `sig_io.write_sig` | `clip(round(v·2¹⁵))` — **over-full-scale signals clip** (distortion) | The gap: only the double chirp normalized — every other over-scale combination (AGWN's ~4σ noise peaks, Level-100 waveforms after the band/notch filter's passband ripple nudged the peak) hit the writer's clip or landed off-scale. ## The fix — one normalization point **`pipeline.apply_post_chain` ends with a scale-down-only peak guard** (after windowing, before returning — both the SigV3.1 write and the preview display see it): - peak `> 1.0 + 1e-9` → scaled to **exactly full scale** (no clipping; the 41b semantics generalized to every waveform) - at/below full scale → the array passes through **untouched** (same object) — `Level` = % of full scale is preserved (a Level-50 comb stays at 0.5; a Level-100 chirp sits at its post-filter peak, now pinned to 1.0) - the **1e-9 tolerance**: a 1-ulp float-noise overshoot (a unit tone peaks at `1 + 2e-16`) must not trigger a pointless renormalization — the writer's 1-ulp clip is invisible - all-zero signals skipped; PAPR/PSD measurements are scale-invariant; byte-stable for identical inputs `chirp.generate_double_chirp` drops its base-level step (subsumed; the COLA repeat is linear, so the later division is equivalent) — **the generator is scale-faithful**. CFR's per-pass renormalization stays (its own §5.3.1 semantics). ## Test updates (the pipeline contract changed by directive) - `test_pipeline`: the pinned fixtures move to sub-full-scale amplitudes (`0.3·tones`, `0.2·noise`, `/1500` arange) so the no-copy passthrough and bitwise contracts stay meaningful (the guard is a no-op at/below full scale) - `test_double_chirp`: the single-block test asserts the **raw** phase-optimised sum (scale-faithful generator); the normalization test moved to the **pipeline level** (raw over-scale → chained peak == 1.0, single-block and repeated) - AGWN's in-band-RMS tests call the generator directly — unaffected; the full-chain AGWN is now clip-free (the RMS-referenced level yields to no-clipping — flagged for operator awareness) ## Spec New **§5.3.5 Final normalization**; the §5.3 chain diagram gains the step; §5.2.1c's level semantics point at the single guard. ## Gate - `pytest`: **1288 passed / 64 skipped** (headless tier), coverage **93.62 %** - `ruff` + `mypy`: clean

ykp added 1 commit

2026-09-29 22:14:49 +02:00

feat(pipeline): single final normalization just before write and display (operator directive)

test / test (ubuntu-latest) (pull_request) Failing after 8h46m35s

Details

05256dc6ce

Problem: normalization steps were scattered (the double chirp's base-
level step, CFR's per-pass renormalization), so combinations WITHOUT
the dedicated step ended up denormalized at the write/display boundary
— the double chirp with filters, AGWN's ~4-sigma noise peaks (the int16
writer CLIPS above full scale), and any Level-100 waveform after the
band/notch filter's passband ripple nudged the peak.

Fix — ONE normalization point:

- pipeline.apply_post_chain ends with a scale-DOWN-ONLY peak guard
  (after windowing, before returning — so both the SigV3.1 write and
  the preview display see it): a peak above full scale (1.0 + 1e-9
  tolerance — a 1-ulp float-noise overshoot must not trigger a
  pointless renormalization) is scaled to exactly full scale; at-or-
  below full scale the array passes through UNTOUCHED (same object),
  preserving Level = % of full scale (a Level-50 comb stays at 0.5).
- chirp.generate_double_chirp drops its base-level normalization
  (subsumed; the repeat is linear so the later division is equivalent)
  — the generator is scale-faithful.
- CFR's per-pass renormalization STAYS (spec §5.3.1 semantics: keeps
  the absolute scale through passes; scale-invariant for the guard).

Test updates (the pipeline contract changed by directive):

- test_pipeline: the sub-full-scale fixtures (0.3*tones, 0.2*noise,
  /1500 arange) keep the no-copy passthrough and bitwise-pinned
  contracts meaningful (the guard is a no-op at/below full scale); the
  combined-order pin keeps its deliberately-peaky PAPR comment.
- test_double_chirp: the single-block test asserts the RAW phase-
  optimised sum (scale-faithful generator); the normalization test
  moves to the pipeline level (raw over-scale -> chained peak == 1.0,
  single-block and repeated); the module docstring follows.
- spec.md: §5.3.5 Final normalization (new), §5.3 chain diagram, §5.2.1c
  level semantics (generator scale-faithful; the guard handles the
  peak).

Gate: pytest 1288 passed / 64 skipped (headless tier), coverage
93.62 %; ruff + mypy clean.

ykp merged commit 33b362ea4a into master

2026-09-30 07:09:33 +02:00

ykp deleted branch final-normalization

2026-09-30 07:09:35 +02:00

ykp referenced this pull request from a commit

2026-09-30 07:09:35 +02:00

Merge pull request 'feat(pipeline): single final normalization just before write and display' (#49) from final-normalization into master