How few tokens can an audio codec decode at once?

Listen to the measured error bands

Open the interactive listening report →: original, full-decode control and independently windowed reconstructions for every codec, dataset, metric and positive-error band. The listening ladder holds one source per dataset fixed across every comparison. Its displayed percentages are individual clip increases, not the pooled confidence-bound thresholds below.

How to compare these results

Compare both temporal frame counts and durations. Each Fish frame spans four times as much audio as a Descript frame. The joint thresholds above therefore measure different durations even when one codec uses fewer frames.

The full-decode baselines matter. Pooled MR-STFT baseline errors are Descript DAC: 0.987, Fish 400M MorePitchLoss1: 1.162, Fish DAC S2-Pro: 0.980, Qwen3-TTS 12Hz: 1.108. Relative tolerance to windowing describes context sensitivity rather than establishing the best absolute reconstruction quality.

The strict 5% upper-confidence-bound criterion is not established for S2-Pro in the tested sweep. The longest partial-window conditions include only the longest eligible clips. These unresolved results are retained rather than inferring an unmeasured cutoff.

Qwen tokenizer: native geometry and decoder controls

Qwen/Qwen3-TTS-Tokenizer-12Hz is the audio tokenizer, not a text-to-speech language model. Despite its name, its native temporal rate is 12.5 frames/s: 1,920 samples per frame at 24 kHz, or 80 ms. Each frame contains 16 discrete codebook IDs. This study retains every book and uses the pinned checkpoint and implementation.

The public Qwen decode helper normally splits long code sequences into 300-frame chunks with 25 frames of left context. That policy is not an independent-window experiment. Here the underlying model.decoder(codes) receives the complete tensor for the control and independent tensors for the treatments, with no shared context. Its eight-layer transformer has a per-layer 72-frame sliding window (5.76 s); this does not itself determine a safe measured cutoff.

Native durations, not nominal model names or scalar index counts, must be compared: a Qwen frame spans 80 ms, a Fish frame 46.44 ms, and a Descript frame 11.61 ms. Codebook sizes in the measured registry: [2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048].

Sources: official HF model and the pinned Qwen implementation recorded in the protocol.

Full-roundtrip quality in the benchmark workbench

A separate normal workbench run evaluates all four codecs on exactly the same 60 checked source IDs: 240 successful full roundtrips, with no failed or missing metrics. Run ID: 0b4452295400. Reconstructions are saved for playback and comparison in the workbench. The following values are unweighted means of 20 clips per dataset, not decoder-window deterioration percentages.

Speaker similarity uses SpeechBrain ECAPA-TDNN at its native 16 kHz; all other metrics use the workbench’s 24 kHz sample-zero alignment, no loudness normalization. Waveform SNR and SI-SDR are phase/timing sensitive, not MOS. Log1p-mel is reported separately throughout the controlled window study. This run contains no new ASR transcriptions or WER claims. Source hashes, runtime fixes and complete per-clip metrics are stored under data/qwen-paired-benchmark/ and the run database.

The workbench uses CPU resampling and requests deterministic Torch algorithms; the controlled window experiment uses GPU resampling and feature extraction. Their definitions and source files match, but numerical paths can produce slightly different quantized codes and absolute errors. Each window-treatment ratio uses its own measured matched full control, never the separate workbench baseline.

Relationship to the preceding Fish experiment

The earlier 62-frame result used an absolute added log-mel margin of 0.01 nat. Against that sample baseline, this is about a 2.3% increase. The present report asks a different question: how small a window can satisfy budgets from 5% to 50% in each of three errors? A much smaller window at a 50% budget is therefore consistent with the earlier result. The sample, full-encode control and independent hard-join policy are shared.

Relative deterioration and uncertainty

The same comparison in token counts

Individual codebook tokens per decoder call

Absolute error by token count

Dataset breakdown by token count

Token-count distributions across clips

Absolute reconstruction error

Decoder divergence at fixed codes

Source-relative error can decrease for an individual clip even while the windowed waveform differs from full decoding. Divergence from the control and increased error against the source answer different questions.

Error budgets from 5% through 50%

Differences between datasets

Subgroup thresholds use the observed sweep; integer refinements target the pooled primary endpoint. Subgroup brackets can therefore be wider.

Clip-to-clip ranges

Mean error ratios and averages of per-clip percentages are different statistics. The primary endpoint is the ratio of arithmetic means. The distribution plot uses individual clip ratios to show the spread; the raw aggregate file additionally includes minima and maxima.

Descript DAC — mel spectrograms

Fish 400M MorePitchLoss1 — mel spectrograms

Fish DAC S2-Pro — mel spectrograms

Qwen3-TTS 12Hz — mel spectrograms

Coverage and interpretation

There is no universal monotonic threshold: changing window length also moves the joins onto different audio content, and decoder context is reset at each call. Reported pass/fail brackets are measured ranges, not interpolated guarantees. A passing pooled mean does not guarantee that every clip or dataset passes.

Confidence intervals are pointwise exploratory intervals for this fixed sample; they are not simultaneous coverage guarantees across all adaptive comparisons. The metrics describe spectral error, and no listener test or ASR metric was used to interpret these window thresholds.

What the S2-Pro architecture tells us

The pinned S2-Pro configuration includes an eight-layer causal transformer after code lookup, with a 128-frame attention window per layer, followed by upsampling and a waveform decoder with four transformer layers. At 21.53 code frames per second, 128 code frames represent about 5.94 seconds; stacking layers can extend the effective history beyond that per-layer window. An independent decoder call starts a new sequence, so it loses the preceding codes as attention context.

This is a plausible mechanism for the measured S2-Pro sensitivity to short windows. The experiment compares complete inference paths and does not isolate architecture from training. Its thresholds apply to decoding independent code windows using the documented hard-join policy.

Local source evidence: vendor/fish-speech/fish_speech/configs/modded_dac_vq.yaml, models/dac/rvq.py:DownsampleResidualVectorQuantize.decode, and models/dac/modded_dac.py:DAC.from_indices, under the pinned Fish source revision in the protocol.

Exact experiment and metric definitions

Each source is encoded once as a complete clip. The same integer code tensor is decoded in one call for the control and then in contiguous independent windows. Equal-length windows may be batched, while each batch row remains an independent sequence. The final short window is retained. Waveforms are concatenated at hard joins and trimmed to source duration; no overlap, crossfade, decoder state, delay correction, or loudness adjustment is used.

All metrics use sample-zero aligned 24 kHz mono audio. The mel transform uses 80 Slaney magnitude bands, Slaney area normalization, a 1024-sample periodic Hann window, 256-sample hop, 0–12 kHz range, centered constant-padded STFT, and epsilon 10⁻⁵.

• Log-mel L1: mean |ln(Msource + 10⁻⁵) − ln(Mdecode + 10⁻⁵)|.
• Log1p-mel L1: mean |ln(1 + Msource) − ln(1 + Mdecode)|. No epsilon or rescaling is added. It weights low-amplitude differences less than log-mel and remains sensitive to the input amplitude scale.
• MR-STFT: mean over FFT sizes 512, 1024, 2048 of spectral convergence plus log-magnitude L1. Each hop is FFT/4, with centered constant padding and periodic Hann window. Spectral convergence is ||Ssource − Sdecode||F / max(||Ssource||F, 10⁻⁵); log-magnitude L1 uses ln(S + 10⁻⁵).

Relative increase: 100 × (mean treatment error / mean matched baseline error − 1). Negative values are retained. Ten thousand paired clip resamples form the percentile 95% interval. The initial broad sweep is followed by integer refinement of pooled threshold brackets; ranges that remain unsampled are reported explicitly.