[FFmpeg-trac] #9350(avcodec:new): HEVC decoder applies alternative transfer function SEI in presentation order with leading pictures
FFmpeg
trac at avcodec.org
Tue Jul 27 14:42:11 EEST 2021
#9350: HEVC decoder applies alternative transfer function SEI in presentation
order with leading pictures
------------------------------------+-----------------------------------
Reporter: jeeb | Owner: (none)
Type: defect | Status: new
Priority: normal | Component: avcodec
Version: git-master | Resolution:
Keywords: hevc | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
------------------------------------+-----------------------------------
Description changed by jeeb:
Old description:
> Summary of the bug:
>
> In HEVC there is a concept called "leading pictures" (NAL unit types
> RASL, RADL), which can come after - among others - CRA ("clean random
> access") NAL units. This is where the pictures have a PTS before the
> random access sample, but come after it.
>
> Sample:
> https://megumin.fushizen.eu/samples/2021-07-23-broadcast_with_alternative_transfer_sei.ts
>
> Now, additionally - this sample utilizes a way of flagging both a
> "compatible" (as in, does not look too incorrect if interpreted as such)
> as well as the preferred (actual) transfer function with having:
> 1. A more compatible value in the VUI information (bt.2020, which is
> effectively the same as BT.709 etc - effectively BT.1886 on the rendering
> side).
> 2. The actual value in a separate SEI message at the CRA IRAP (ARIB
> STD-B67 aka HLG).
>
> This SEI is defined that it should be applied in decoding order for the
> following sequence: ''"The alternative transfer characteristics SEI
> message persists for the current layer in decoding order from the current
> picture until the end of the CLVS"''
>
> Unfortunately, it seems like the libavcodec HEVC decoder applies these in
> presentation order, so the leading pictures which come after the CRA IRAP
> (but get output first in presentation order) will only get the
> "compatible" value, which leads to workflows that configure their output
> based on the first received frame (such as ffmpeg.c) to output the
>
> How to reproduce:
> {{{
> % ffprobe -v verbose -i
> 'https://megumin.fushizen.eu/samples/2021-07-23-broadcast_with_alternative_transfer_sei.ts'
> -of json -show_frames -select_streams v > rasl_stream_frames.json
> }}}
> {{{
> head -92 rasl_stream_frames.json
> {
> "frames": [
> {
> "media_type": "video",
> "stream_index": 0,
> "key_frame": 0,
> "pkt_pts": 7988680456,
> "pkt_pts_time": "88763.116178",
> "pkt_dts": 7988696656,
> "pkt_dts_time": "88763.296178",
> "best_effort_timestamp": 7988680456,
> "best_effort_timestamp_time": "88763.116178",
> "pkt_duration": 1800,
> "pkt_duration_time": "0.020000",
> "pkt_pos": "50384",
> "pkt_size": "38001",
> "width": 1920,
> "height": 1080,
> "pix_fmt": "yuv420p10le",
> "sample_aspect_ratio": "1:1",
> "pict_type": "B",
> "coded_picture_number": 0,
> "display_picture_number": 0,
> "interlaced_frame": 0,
> "top_field_first": 0,
> "repeat_pict": 0,
> "color_range": "tv",
> "color_space": "bt2020nc",
> "color_primaries": "bt2020",
> "color_transfer": "bt2020-10",
> "chroma_location": "left"
> },
> {
> "media_type": "video",
> "stream_index": 0,
> "key_frame": 0,
> "pkt_pts": 7988684056,
> "pkt_pts_time": "88763.156178",
> "pkt_dts": 7988698456,
> "pkt_dts_time": "88763.316178",
> "best_effort_timestamp": 7988684056,
> "best_effort_timestamp_time": "88763.156178",
> "pkt_duration": 1800,
> "pkt_duration_time": "0.020000",
> "pkt_pos": "117312",
> "pkt_size": "38000",
> "width": 1920,
> "height": 1080,
> "pix_fmt": "yuv420p10le",
> "sample_aspect_ratio": "1:1",
> "pict_type": "B",
> "coded_picture_number": 0,
> "display_picture_number": 0,
> "interlaced_frame": 0,
> "top_field_first": 0,
> "repeat_pict": 0,
> "color_range": "tv",
> "color_space": "bt2020nc",
> "color_primaries": "bt2020",
> "color_transfer": "bt2020-10",
> "chroma_location": "left"
> },
> {
> "media_type": "video",
> "stream_index": 0,
> "key_frame": 1,
> "pkt_pts": 7988700256,
> "pkt_pts_time": "88763.336178",
> "pkt_dts": 7988700256,
> "pkt_dts_time": "88763.336178",
> "best_effort_timestamp": 7988700256,
> "best_effort_timestamp_time": "88763.336178",
> "pkt_duration": 1800,
> "pkt_duration_time": "0.020000",
> "pkt_pos": "184992",
> "pkt_size": "38000",
> "width": 1920,
> "height": 1080,
> "pix_fmt": "yuv420p10le",
> "sample_aspect_ratio": "1:1",
> "pict_type": "I",
> "coded_picture_number": 0,
> "display_picture_number": 0,
> "interlaced_frame": 0,
> "top_field_first": 0,
> "repeat_pict": 0,
> "color_range": "tv",
> "color_space": "bt2020nc",
> "color_primaries": "bt2020",
> "color_transfer": "arib-std-b67",
> "chroma_location": "left"
> },
> }}}
>
> Expected result: all output frames should be flagged as
> ''"color_transfer": "arib-std-b67"''.
> Received result: leading pictures get flagged with the base "compatible"
> transfer function.
New description:
Summary of the bug:
In HEVC there is a concept called "leading pictures" (NAL unit types RASL,
RADL), which can come after - among others - CRA ("clean random access")
NAL units. This is where the pictures have a PTS before the random access
sample, but come after it.
Sample:
https://megumin.fushizen.eu/samples/2021-07-23-broadcast_with_alternative_transfer_sei.ts
Now, additionally - this sample utilizes a way of flagging both a
"compatible" (as in, does not look too incorrect if interpreted as such)
as well as the preferred (actual) transfer function with having:
1. A more compatible value in the VUI information (bt.2020, which is
effectively the same as BT.709 etc - effectively BT.1886 on the rendering
side).
2. The actual value in a separate SEI message at the CRA IRAP (ARIB
STD-B67 aka HLG).
This SEI is defined that it should be applied in decoding order for the
following sequence: ''"The alternative transfer characteristics SEI
message persists for the current layer in decoding order from the current
picture until the end of the CLVS"''
Unfortunately, it seems like the libavcodec HEVC decoder applies these in
presentation order, so the leading pictures which come after the CRA IRAP
(but get output first in presentation order) will only get the
"compatible" value, which leads to workflows that configure their output
based on the first received frame (such as ffmpeg.c) to output the
compatible value instead of the actual value.
How to reproduce:
{{{
% ffprobe -v verbose -i
'https://megumin.fushizen.eu/samples/2021-07-23-broadcast_with_alternative_transfer_sei.ts'
-of json -show_frames -select_streams v > rasl_stream_frames.json
}}}
{{{
head -92 rasl_stream_frames.json
{
"frames": [
{
"media_type": "video",
"stream_index": 0,
"key_frame": 0,
"pkt_pts": 7988680456,
"pkt_pts_time": "88763.116178",
"pkt_dts": 7988696656,
"pkt_dts_time": "88763.296178",
"best_effort_timestamp": 7988680456,
"best_effort_timestamp_time": "88763.116178",
"pkt_duration": 1800,
"pkt_duration_time": "0.020000",
"pkt_pos": "50384",
"pkt_size": "38001",
"width": 1920,
"height": 1080,
"pix_fmt": "yuv420p10le",
"sample_aspect_ratio": "1:1",
"pict_type": "B",
"coded_picture_number": 0,
"display_picture_number": 0,
"interlaced_frame": 0,
"top_field_first": 0,
"repeat_pict": 0,
"color_range": "tv",
"color_space": "bt2020nc",
"color_primaries": "bt2020",
"color_transfer": "bt2020-10",
"chroma_location": "left"
},
{
"media_type": "video",
"stream_index": 0,
"key_frame": 0,
"pkt_pts": 7988684056,
"pkt_pts_time": "88763.156178",
"pkt_dts": 7988698456,
"pkt_dts_time": "88763.316178",
"best_effort_timestamp": 7988684056,
"best_effort_timestamp_time": "88763.156178",
"pkt_duration": 1800,
"pkt_duration_time": "0.020000",
"pkt_pos": "117312",
"pkt_size": "38000",
"width": 1920,
"height": 1080,
"pix_fmt": "yuv420p10le",
"sample_aspect_ratio": "1:1",
"pict_type": "B",
"coded_picture_number": 0,
"display_picture_number": 0,
"interlaced_frame": 0,
"top_field_first": 0,
"repeat_pict": 0,
"color_range": "tv",
"color_space": "bt2020nc",
"color_primaries": "bt2020",
"color_transfer": "bt2020-10",
"chroma_location": "left"
},
{
"media_type": "video",
"stream_index": 0,
"key_frame": 1,
"pkt_pts": 7988700256,
"pkt_pts_time": "88763.336178",
"pkt_dts": 7988700256,
"pkt_dts_time": "88763.336178",
"best_effort_timestamp": 7988700256,
"best_effort_timestamp_time": "88763.336178",
"pkt_duration": 1800,
"pkt_duration_time": "0.020000",
"pkt_pos": "184992",
"pkt_size": "38000",
"width": 1920,
"height": 1080,
"pix_fmt": "yuv420p10le",
"sample_aspect_ratio": "1:1",
"pict_type": "I",
"coded_picture_number": 0,
"display_picture_number": 0,
"interlaced_frame": 0,
"top_field_first": 0,
"repeat_pict": 0,
"color_range": "tv",
"color_space": "bt2020nc",
"color_primaries": "bt2020",
"color_transfer": "arib-std-b67",
"chroma_location": "left"
},
}}}
Expected result: all output frames should be flagged as
''"color_transfer": "arib-std-b67"''.
Received result: leading pictures get flagged with the base "compatible"
transfer function.
--
--
Ticket URL: <https://trac.ffmpeg.org/ticket/9350#comment:1>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
More information about the FFmpeg-trac
mailing list