Commit 88e8cfc801 for ffmpeg
commit 88e8cfc801be24192b8d0292ee33a30249342970
Author: Lynne <dev@lynne.ee>
Date: Wed Sep 23 20:25:14 2026 +0900
avcodec/aacpsy: gap-onset attacks, HF-novelty veto, level-homogeneous short grouping
Three changes to the LAME-style window switch and short-window grouping.
Gap-onset detection. Two attack classes were invisible to the
fs/4-highpass ratio test: stop-gap slams with no content above the
filter (or a rise too gradual for the 2-sub-block comparison), and both
classes of pulse-train repeat. Detect gap-onsets on broadband and HP
sub-block peak envelopes against a decaying program-level reference:
fire when a sustained quiet stretch ends adjacent to a candidate
comparable to the recent program level. Pulse trains cannot fire it -
their inter-pulse floor never drops far enough below their own running
peak.
HF-novelty veto. A bass onset under sustained playing trips the attack
detector, but a long window codes it better: pre-echo is masked by the
bass itself, while the short excursion is the audible event - the
shorts frame mutes the HF floor, then the stop frame codes the attack's
HF spray into 21ms of silent-HF background at SNR~0 (ENC_TEST_SAMPLE_05
~2.07s, 2x the reference HF energy, rate-independent; Apple stays long
through the same passage). Candidates are gated on a first-difference
sub-block peak envelope, and the veto fires only when all three agree:
- absolutely small (2000): no HF content worth protecting;
- relatively unremarkable (4x the recent derivative envelope):
a quiet transient rising out of near-silence has a small derivative
but maximal novelty, and its long-window pre-echo lands on silence
('rustle of a falling leaf', fatboy 4.35s: 3x the un-vetoed error);
- pre-echo is masked: the minimum envelope over the ~5ms before the
attack is no more than 8x below the candidate. Stop-gap
slams (kick+chug into a beat gap) pass both other bars, but the dip
right before them means the smear lands on quiet.
Gap-detector attacks fire downstream and are unaffected. E5 2.07s
scratch gone (HF ratio 5.13 -> 0.64, Apple 0.63); castanets -11%,
trumpet -17% Zim (their low-derivative spurious shorts were hurting);
velvet/applaud/Linchpin and both corpus rates neutral; artifacts.flac
click-free.
Level-homogeneous grouping. The fixed attack-position grouping patterns
force windows of very different level into one group, where they share
scalefactors and the group's error placement smears across the level
step - the mechanism behind the castanets/velvet transient deficit.
Group instead by level runs: a new group starts where adjacent lookahead
windows differ by more than 2.5x in peak level, with group
length capped at 4; the pair path merges the two channels'
level maps so the grouping (and with it common_window) stays shared.
Castanets -15% Zim, velvet -9%, Linchpin -5.5%, 64k corpus -4.5%, 128k
corpus neutral; applause pays +3% for the fragmentation. Forcing extra
boundaries at detected attacks reversed the wins and is not done -
fragmentation, not boundary placement, is the enemy. A looser 2.0x/cap-8
rule buys castanets and velvet another 20% but costs the corpus at both
rates.
Rejected along the way: capping the group that starts at an attack (the
FDK 2-window discipline) and isolating the pre-attack window close the
pre-attack window gap to FDK on the per-window attack-profile
instrument, but the overhead costs fatboy/velvet corpus Zim.
The window decisions change for every coder: tns-encode re-pinned
637 -> 619. The id3v2 re-encode refs follow the default NMR output.
diff --git a/libavcodec/aacpsy.c b/libavcodec/aacpsy.c
index 04e0c67ba4..55ba412de4 100644
--- a/libavcodec/aacpsy.c
+++ b/libavcodec/aacpsy.c
@@ -112,6 +112,24 @@ enum {
#define PSY_LAME_HIST 32 ///< HP sub-block peak history depth
#define PSY_LAME_NOV_BACK 30 ///< novelty look-back in sub-blocks
+/* HF-novelty veto: an attack candidate stays long when its first-difference
+ * envelope is both relatively and absolutely unremarkable, and pre-echo is
+ * masked by what precedes it. */
+#define PSY_LAME_HFN_REL 4.0f ///< max derivative rise over the recent envelope
+#define PSY_LAME_HFN_ABS 2000.0f ///< max absolute derivative peak
+#define PSY_LAME_HFN_PRE 8.0f ///< max candidate rise over the ~5ms pre-attack minimum
+
+/* Gap-onset detection against a decaying program-level peak-hold */
+#define PSY_LAME_GAP_DEPTH 6.0f ///< a gap is this far below the peak-hold
+#define PSY_LAME_GAP_BACK 24 ///< gap look-back in sub-blocks (<= PSY_LAME_HIST)
+#define PSY_LAME_GAP_LEVEL 0.4f ///< candidate must reach this fraction of the peak-hold
+#define PSY_LAME_GAP_FLOOR 4000.0f ///< and this absolute peak
+#define PSY_LAME_GAP_TOWER 4.0f ///< HP tower: rise over the whole look-back
+
+/* Level-homogeneous short-window grouping */
+#define PSY_LAME_GRP_RATIO 2.5f ///< max adjacent-window level ratio inside a group
+#define PSY_LAME_GRP_MAX 4 ///< max windows per group
+
/**
* @}
*/
@@ -145,7 +163,12 @@ typedef struct AacPsyChannel{
/* LAME psy model specific members */
float attack_threshold; ///< attack threshold for this channel
float prev_energy_subshort[AAC_NUM_BLOCKS_SHORT * PSY_LAME_NUM_SUBBLOCKS];
+ float next_win_level[AAC_NUM_BLOCKS_SHORT]; ///< lookahead short-window peak levels (grouping homogeneity)
+ float dif_env_hist[PSY_LAME_HIST]; ///< rolling first-difference sub-block peak envelope (HF novelty)
float hp_env_hist[PSY_LAME_HIST]; ///< rolling HP sub-block peak envelope
+ float raw_env_hist[PSY_LAME_HIST]; ///< rolling broadband sub-block peak envelope
+ float gap_wall; ///< decaying broadband peak-hold (gap-onset reference)
+ float gap_wall_hp; ///< decaying HP peak-hold (gap-onset reference)
int prev_attack; ///< attack value for the last short block in the previous sequence
int next_attack0_zero; ///< whether attack[0] of the next frame is zero
int frames_since_short; ///< consecutive long frames (pre-echo-aware isolated-onset gate)
@@ -300,7 +323,8 @@ static av_cold void lame_window_init(AacPsyContext *ctx, AVCodecContext *avctx)
for (j = 0; j < AAC_NUM_BLOCKS_SHORT * PSY_LAME_NUM_SUBBLOCKS; j++)
pch->prev_energy_subshort[j] = 10.0f;
for (j = 0; j < PSY_LAME_HIST; j++)
- pch->hp_env_hist[j] = 10.0f;
+ pch->hp_env_hist[j] = pch->raw_env_hist[j] = pch->dif_env_hist[j] = 10.0f;
+ pch->gap_wall = pch->gap_wall_hp = 0.0f;
}
}
@@ -1008,13 +1032,30 @@ static int psy_lame_detect(AacPsyContext *pctx, AacPsyChannel *pch,
attack_intensity[i + PSY_LAME_NUM_SUBBLOCKS] = p;
}
+ for (i = 0; i < AAC_NUM_BLOCKS_SHORT; i++)
+ pch->next_win_level[i] = energy_short[1 + i];
{ /* pre-echo-aware threshold relaxation + periodicity/novelty check
* (a pulse train repeats its peak; a real onset towers) */
float frame_peak = 1.0f;
float env[PSY_LAME_HIST + AAC_NUM_BLOCKS_SHORT * PSY_LAME_NUM_SUBBLOCKS];
+ float denv[PSY_LAME_HIST + AAC_NUM_BLOCKS_SHORT * PSY_LAME_NUM_SUBBLOCKS];
const float nov_gate = 1.25f;
- memcpy(env, pch->hp_env_hist, sizeof(pch->hp_env_hist));
+ /* first-difference peak per sub-block: an attack that shorts can
+ * help has HF novelty; a bass pluck under a long window does not */
+ memcpy(denv, pch->dif_env_hist, sizeof(pch->dif_env_hist));
+ {
+ const int sub = AAC_BLOCK_SIZE_LONG / (AAC_NUM_BLOCKS_SHORT * PSY_LAME_NUM_SUBBLOCKS);
+ for (i = 0; i < AAC_NUM_BLOCKS_SHORT * PSY_LAME_NUM_SUBBLOCKS; i++) {
+ float p = 0.0f;
+ for (int j2 = i*sub + 1; j2 < (i+1)*sub; j2++)
+ p = FFMAX(p, fabsf(la[j2] - la[j2-1]));
+ denv[PSY_LAME_HIST + i] = FFMAX(p * 32768.0f, 1.0f);
+ }
+ }
+ memcpy(pch->dif_env_hist, denv + AAC_NUM_BLOCKS_SHORT * PSY_LAME_NUM_SUBBLOCKS,
+ sizeof(pch->dif_env_hist));
+ memcpy(env,pch->hp_env_hist, sizeof(pch->hp_env_hist));
memcpy(env + PSY_LAME_HIST, energy_subshort + PSY_LAME_NUM_SUBBLOCKS,
AAC_NUM_BLOCKS_SHORT * PSY_LAME_NUM_SUBBLOCKS * sizeof(*env));
for (i = PSY_LAME_NUM_SUBBLOCKS; i < (AAC_NUM_BLOCKS_SHORT + 1) * PSY_LAME_NUM_SUBBLOCKS; i++)
@@ -1044,6 +1085,33 @@ static int psy_lame_detect(AacPsyContext *pctx, AacPsyChannel *pch,
pch->frames_since_short >= PSY_LAME_PE_GAP))
continue; /* periodic, not an onset */
}
+ if (i >= PSY_LAME_NUM_SUBBLOCKS) {
+ /* no HF novelty: pre-echo is masked by the
+ * sustained LF itself, and the short excursion
+ * (HF mute, then the stop frame's noisy HF
+ * hand-back) is the audible event. Both bars
+ * must agree: absolutely small AND relatively
+ * unremarkable - a quiet transient rising out
+ * of silence has a tiny derivative but maximal
+ * novelty, and its pre-echo lands on silence */
+ const int pos = PSY_LAME_HIST + i - PSY_LAME_NUM_SUBBLOCKS;
+ float dmax = 1.0f, premin = 1e30f;
+ for (int k = 1; k <= PSY_LAME_NOV_BACK; k++)
+ dmax = FFMAX(dmax, denv[pos - k]);
+ /* pre-echo audibility: the veto is only safe
+ * when the surroundings mask the smear - a deep
+ * dip right before a LOUD attack (stop-gap
+ * slams) means the long window's pre-echo lands
+ * on quiet. Quiet candidates keep the veto:
+ * their smear is at the noise floor, and shorts
+ * would only fragment the passage */
+ for (int k = 1; k <= 4; k++)
+ premin = FFMIN(premin, env[pos - k]);
+ if (denv[pos] < PSY_LAME_HFN_REL * dmax &&
+ denv[pos] < PSY_LAME_HFN_ABS &&
+ premin * PSY_LAME_HFN_PRE > energy_subshort[i])
+ continue;
+ }
attacks[i / PSY_LAME_NUM_SUBBLOCKS] = (i % PSY_LAME_NUM_SUBBLOCKS) + 1;
}
}
@@ -1067,6 +1135,74 @@ static int psy_lame_detect(AacPsyContext *pctx, AacPsyChannel *pch,
att_sum += attacks[i];
}
+ { /* Gap-onset detection on the broadband envelope: a slam that ends
+ * a quiet gap (stop-start riffing, kick after a break) can be
+ * invisible to the HP path - no content above fs/4, or a rise too
+ * gradual for the 2-sub-block ratio - yet pre-echo into the gap
+ * is maximally audible (no forward masking there). Fire when the
+ * candidate towers over a recent dip. Pulse trains cannot fire:
+ * their inter-pulse floor never dips far enough below the pulse. */
+ float renv[PSY_LAME_HIST + AAC_NUM_BLOCKS_SHORT * PSY_LAME_NUM_SUBBLOCKS];
+ /* scan both envelopes: broadband (kick+chug slams with LF
+ * dominance) and HP (events whose gap only exists above the
+ * sustained bass) */
+ float henv[PSY_LAME_HIST + AAC_NUM_BLOCKS_SHORT * PSY_LAME_NUM_SUBBLOCKS];
+ float *walls[2] = { &pch->gap_wall, &pch->gap_wall_hp };
+ const float *envs[2] = { renv, henv };
+ memcpy(renv, pch->raw_env_hist, sizeof(pch->raw_env_hist));
+ for (i = 0; i < AAC_NUM_BLOCKS_SHORT * PSY_LAME_NUM_SUBBLOCKS; i++) {
+ float p = 0.0f;
+ for (int j2 = 0; j2 < 64; j2++)
+ p = FFMAX(p, fabsf(la[i*64 + j2]));
+ renv[PSY_LAME_HIST + i] = FFMAX(p * 32768.0f, 1.0f);
+ }
+ memcpy(henv, pch->hp_env_hist, sizeof(pch->hp_env_hist));
+ memcpy(henv + PSY_LAME_HIST, energy_subshort + PSY_LAME_NUM_SUBBLOCKS,
+ AAC_NUM_BLOCKS_SHORT * PSY_LAME_NUM_SUBBLOCKS * sizeof(*henv));
+ for (int e = 0; e < 2; e++) {
+ const float *ev = envs[e];
+ float wall = *walls[e];
+ for (i = 0; i < AAC_NUM_BLOCKS_SHORT * PSY_LAME_NUM_SUBBLOCKS; i++) {
+ const int b = (i + PSY_LAME_NUM_SUBBLOCKS) / PSY_LAME_NUM_SUBBLOCKS;
+ const int pos = PSY_LAME_HIST + i;
+ const float cand = ev[pos];
+ const float quiet = wall / PSY_LAME_GAP_DEPTH;
+ int run = 0, k0 = 0;
+ /* the gap must end adjacent to the candidate (<= 5
+ * rising sub-blocks) and hold >= 4 sub-blocks (~6ms).
+ * A pulse train's inter-pulse floor never drops this
+ * far below its own running peak, so it cannot fire. */
+ for (int k = 3; k <= 8; k++)
+ if (ev[pos - k] < quiet) {
+ k0 = k;
+ break;
+ }
+ if (k0)
+ for (int k = k0; k <= PSY_LAME_GAP_BACK && ev[pos - k] < quiet; k++)
+ run++;
+ if (e == 1 && !(run >= 4)) {
+ /* HP tower: an onset rising far above everything
+ * in the look-back, even without a silent gap
+ * (cymbal-less slams leave the bass sustaining) */
+ float dmax = 1.0f;
+ for (int k = 3; k <= PSY_LAME_GAP_BACK; k++)
+ dmax = FFMAX(dmax, ev[pos - k]);
+ if (cand > PSY_LAME_GAP_TOWER * dmax)
+ run = 4; /* qualify via the same fire path */
+ }
+ if (!attacks[b] && run >= 4 && cand > PSY_LAME_GAP_LEVEL * wall &&
+ cand > PSY_LAME_GAP_FLOOR) {
+ attacks[b] = (i + PSY_LAME_NUM_SUBBLOCKS) % PSY_LAME_NUM_SUBBLOCKS + 1;
+ att_sum += attacks[b];
+ }
+ wall = FFMAX(wall * 0.996f, cand);
+ }
+ *walls[e] = wall;
+ }
+ memcpy(pch->raw_env_hist, renv + AAC_NUM_BLOCKS_SHORT * PSY_LAME_NUM_SUBBLOCKS,
+ sizeof(pch->raw_env_hist));
+ }
+
/* roll the HP sub-block peak history */
memmove(pch->hp_env_hist,
pch->hp_env_hist + AAC_NUM_BLOCKS_SHORT * PSY_LAME_NUM_SUBBLOCKS,
@@ -1154,6 +1290,24 @@ static FFPsyWindowInfo psy_lame_apply(AacPsyContext *pctx, AacPsyChannel *pch,
}
pch->next_grouping = window_grouping[grouping];
+ {
+ /* energy-homogeneous grouping: fixed attack-position patterns force
+ * disparate windows to share scalefactors; regroup on level jumps */
+ uint8_t bits = 0;
+ int glen = 1;
+ for (i = 1; i < AAC_NUM_BLOCKS_SHORT; i++) {
+ float a = pch->next_win_level[i], b = pch->next_win_level[i-1];
+ float hi = FFMAX(a, b), lo = FFMAX(FFMIN(a, b), 1.0f);
+ if (hi <= lo * PSY_LAME_GRP_RATIO && glen < PSY_LAME_GRP_MAX) {
+ bits |= 1 << i;
+ glen++;
+ } else {
+ glen = 1;
+ }
+ }
+ pch->next_grouping = bits;
+ }
+
pch->prev_attack = attacks[AAC_NUM_BLOCKS_SHORT - 1];
return wi;
@@ -1204,6 +1358,11 @@ static void psy_lame_window_pair(FFPsyContext *ctx,
for (int i = 0; i < AAC_NUM_BLOCKS_SHORT + 1; i++)
merged[i] = att0[i] ? att0[i] : att1[i];
+ /* grouping must also match across the pair: merge the level maps */
+ for (int i = 0; i < AAC_NUM_BLOCKS_SHORT; i++)
+ pch0->next_win_level[i] = pch1->next_win_level[i] =
+ FFMAX(pch0->next_win_level[i], pch1->next_win_level[i]);
+
wi[0] = psy_lame_apply(pctx, pch0, u, merged, prev_type0, !!la0);
wi[1] = psy_lame_apply(pctx, pch1, u, merged, prev_type1, !!la1);
}
diff --git a/tests/fate/aac.mak b/tests/fate/aac.mak
index 071fc8ae17..7bbcc6d5f6 100644
--- a/tests/fate/aac.mak
+++ b/tests/fate/aac.mak
@@ -238,7 +238,7 @@ FATE_AAC_ENCODE += fate-aac-tns-encode
fate-aac-tns-encode: CMD = enc_dec_pcm mp4 wav s16le $(TARGET_SAMPLES)/audio-reference/luckynight_2ch_44kHz_s16.wav -c:a aac -aac_coder fast -aac_tns 1 -aac_is 0 -aac_pns 0 -aac_ms 0 -b:a 128k -cutoff 22050 -fflags +bitexact -flags +bitexact
fate-aac-tns-encode: CMP = stddev
fate-aac-tns-encode: REF = $(SAMPLES)/audio-reference/luckynight_2ch_44kHz_s16.wav
-fate-aac-tns-encode: CMP_TARGET = 637
+fate-aac-tns-encode: CMP_TARGET = 619
fate-aac-tns-encode: FUZZ = 7
FATE_AAC_ENCODE += fate-aac-is-encode
diff --git a/tests/ref/fate/id3v2-reenc-delete-metadata b/tests/ref/fate/id3v2-reenc-delete-metadata
index a3c9b7b9b9..2d19a3071b 100644
--- a/tests/ref/fate/id3v2-reenc-delete-metadata
+++ b/tests/ref/fate/id3v2-reenc-delete-metadata
@@ -1,5 +1,5 @@
-10ccc863d23c9f43b7fa94dd42702d61 *tests/data/fate/id3v2-reenc-delete-metadata.nut
-2794 tests/data/fate/id3v2-reenc-delete-metadata.nut
+54b8e2b9f3d83cbd601ae625978a5c4a *tests/data/fate/id3v2-reenc-delete-metadata.nut
+2683 tests/data/fate/id3v2-reenc-delete-metadata.nut
[FORMAT]
TAG:title=7rk
[/FORMAT]
diff --git a/tests/ref/fate/id3v2-reenc-delete-metadata-keep b/tests/ref/fate/id3v2-reenc-delete-metadata-keep
index ac33b3bdcd..98caa8c82b 100644
--- a/tests/ref/fate/id3v2-reenc-delete-metadata-keep
+++ b/tests/ref/fate/id3v2-reenc-delete-metadata-keep
@@ -1,5 +1,5 @@
-0a35ba4d86e0bcf69513d03019336af6 *tests/data/fate/id3v2-reenc-delete-metadata-keep.nut
-2934 tests/data/fate/id3v2-reenc-delete-metadata-keep.nut
+46e8ca22aa8be69d14bfad4f4ed3cefc *tests/data/fate/id3v2-reenc-delete-metadata-keep.nut
+2823 tests/data/fate/id3v2-reenc-delete-metadata-keep.nut
[FORMAT]
TAG:title=7rk
TAG:comment-iTunSMPB-eng= 00000000 00000210 0000086A 0000000000066486 00000000 0002DA9D 00000000 00000000 00000000 00000000 00000000 00000000
diff --git a/tests/ref/fate/id3v2-reenc-delete-metadata-keep-format b/tests/ref/fate/id3v2-reenc-delete-metadata-keep-format
index 86dc5c75b0..a3e9fbd819 100644
--- a/tests/ref/fate/id3v2-reenc-delete-metadata-keep-format
+++ b/tests/ref/fate/id3v2-reenc-delete-metadata-keep-format
@@ -1,5 +1,5 @@
-0a35ba4d86e0bcf69513d03019336af6 *tests/data/fate/id3v2-reenc-delete-metadata-keep-format.nut
-2934 tests/data/fate/id3v2-reenc-delete-metadata-keep-format.nut
+46e8ca22aa8be69d14bfad4f4ed3cefc *tests/data/fate/id3v2-reenc-delete-metadata-keep-format.nut
+2823 tests/data/fate/id3v2-reenc-delete-metadata-keep-format.nut
[FORMAT]
TAG:title=7rk
TAG:comment-iTunSMPB-eng= 00000000 00000210 0000086A 0000000000066486 00000000 0002DA9D 00000000 00000000 00000000 00000000 00000000 00000000
diff --git a/tests/ref/fate/id3v2-reenc-delete-metadata-keep-stream b/tests/ref/fate/id3v2-reenc-delete-metadata-keep-stream
index 5aa028cd1e..6415519a53 100644
--- a/tests/ref/fate/id3v2-reenc-delete-metadata-keep-stream
+++ b/tests/ref/fate/id3v2-reenc-delete-metadata-keep-stream
@@ -1,5 +1,5 @@
-10ccc863d23c9f43b7fa94dd42702d61 *tests/data/fate/id3v2-reenc-delete-metadata-keep-stream.nut
-2794 tests/data/fate/id3v2-reenc-delete-metadata-keep-stream.nut
+54b8e2b9f3d83cbd601ae625978a5c4a *tests/data/fate/id3v2-reenc-delete-metadata-keep-stream.nut
+2683 tests/data/fate/id3v2-reenc-delete-metadata-keep-stream.nut
[FORMAT]
TAG:title=7rk
[/FORMAT]
diff --git a/tests/ref/fate/id3v2-reenc-delete-metadata-map-metadata b/tests/ref/fate/id3v2-reenc-delete-metadata-map-metadata
index 0494d2e00d..62a9411e4e 100644
--- a/tests/ref/fate/id3v2-reenc-delete-metadata-map-metadata
+++ b/tests/ref/fate/id3v2-reenc-delete-metadata-map-metadata
@@ -1,5 +1,5 @@
-10ccc863d23c9f43b7fa94dd42702d61 *tests/data/fate/id3v2-reenc-delete-metadata-map-metadata.nut
-2794 tests/data/fate/id3v2-reenc-delete-metadata-map-metadata.nut
+54b8e2b9f3d83cbd601ae625978a5c4a *tests/data/fate/id3v2-reenc-delete-metadata-map-metadata.nut
+2683 tests/data/fate/id3v2-reenc-delete-metadata-map-metadata.nut
[FORMAT]
TAG:title=7rk
[/FORMAT]