[FFmpeg-devel] [PATCH] h264: fix reference reordering

[averne]

Fixes hardware decoders depending on long_term_pic_idx.
Relevant ITU conformance test vectors: MR6_BT_B, MR8_BT_B

---
According to the specification (8.2.4.3 Modification process for
reference picture lists), the reference reordering process does not
affect frame indices (PicNum for short refs and LongTermPicNum for long
ones).

Hardware decoders commonly use this values in order to identify refs,
so modifying the frame idx passed to the the hardware would throw off
the decode engine.

Fixes: two ITU conformance test samples (MR6_BT_B and MR8_BT_B, found at
https://www.itu.int/net/itu-t/sigdb/spevideo/VideoForm-s.aspx?val=102002641),
on my WIP hwaccel backend (https://github.com/averne/FFmpeg/tree/envideo).
Potentially fixes more hwaccels using this value (dxva2, vaapi, vdpau,
vulkan).

This patch was verified to not cause any regression of the software
decoder.

Signed-off-by: averne <averne381 at gmail.com>
---
 libavcodec/h264_refs.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/libavcodec/h264_refs.c b/libavcodec/h264_refs.c
index 050ca9229218d68c1f1c5acb8fed8e24c61fd5a1..74840e5909212d6a750518acc416221ffe90f722 100644
--- a/libavcodec/h264_refs.c
+++ b/libavcodec/h264_refs.c
@@ -337,7 +337,7 @@ int ff_h264_build_ref_list(H264Context *h, H264SliceContext *sl)
                         break;
                 }
                 if (i >= 0)
-                    ref->pic_id = pred;
+                    pic_id = pred;
                 break;
             }
             case 2: {
@@ -354,7 +354,6 @@ int ff_h264_build_ref_list(H264Context *h, H264SliceContext *sl)
                 ref = h->long_ref[long_idx];
                 assert(!(ref && !ref->reference));
                 if (ref && (ref->reference & pic_structure)) {
-                    ref->pic_id = pic_id;
                     assert(ref->long_ref);
                     i = 0;
                 } else {
@@ -376,7 +375,7 @@ int ff_h264_build_ref_list(H264Context *h, H264SliceContext *sl)
                 for (i = index; i + 1 < sl->ref_count[list]; i++) {
                     if (sl->ref_list[list][i].parent &&
                         ref->long_ref == sl->ref_list[list][i].parent->long_ref &&
-                        ref->pic_id   == sl->ref_list[list][i].pic_id)
+                        pic_id        == sl->ref_list[list][i].pic_id)
                         break;
                 }
                 for (; i > index; i--) {

---
base-commit: 26f2f03e0de278f296fbd1e8a09c43245b65f5e3
change-id: 20250323-h264-reorder-38465068ea4e

Best regards,
-- 
averne <averne381 at gmail.com>