#9088(undetermined:new): HEVC bitstream with alpha layer to mp4
#9088: HEVC bitstream with alpha layer to mp4 -------------------------------------+------------------------------------- Reporter: johnhe4 | Type: | enhancement Status: new | Priority: normal Component: | Version: undetermined | unspecified Keywords: hevc h.265 | Blocked By: alpha transparency missing access | Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- == Summary == I used NVIDIA's Video Codec SDK to create an HEVC bitstream containing an alpha layer. I tried to mux this bitstream into an MP4 container with no success. I was hoping this would work because: 1. It was a copy, not a re-encode (no need to involve libx265) 2. This change gave me hope: https://github.com/FFmpeg/FFmpeg/commit/ad326379c6634505c9ebc469640574417610... No cigar. FFmpeg ''did'' create an MP4 file but the MP4 will not play in QuickTime and does not seem to retain the alpha layer. Whenever a `nuh_layer_id > 0` is encountered FFmpeg outputs: {{{ missing picture in access unit }}} I've attached the `ffmpeg -v trace ...` output to this request as well as the HEVC bitstream containing the alpha layer. == How to reproduce == {{{ % ffmpeg -i bsWithAlpha.265 -c:v copy -tag:v hvc1 outputWithAlpha.mp4 }}} -- Ticket URL: <https://trac.ffmpeg.org/ticket/9088> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9088: HEVC bitstream with alpha layer to mp4 -------------------------------------+------------------------------------- Reporter: johnhe4 | Owner: Type: enhancement | Status: new Priority: normal | Component: | undetermined Version: unspecified | Resolution: Keywords: hevc h.265 | Blocked By: alpha transparency missing access | Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Changes (by johnhe4): * Attachment "trace.txt" added. -- Ticket URL: <https://trac.ffmpeg.org/ticket/9088> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9088: HEVC bitstream with alpha layer to mp4 -------------------------------------+------------------------------------- Reporter: johnhe4 | Owner: Type: enhancement | Status: new Priority: normal | Component: | undetermined Version: unspecified | Resolution: Keywords: hevc h.265 | Blocked By: alpha transparency missing access | Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Changes (by johnhe4): * Attachment "bsWithAlpha.265" added. HEVC bitstream with alpha layer -- Ticket URL: <https://trac.ffmpeg.org/ticket/9088> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9088: HEVC bitstream with alpha layer to mp4 -------------------------------------+------------------------------------- Reporter: johnhe4 | Owner: Type: enhancement | Status: new Priority: normal | Component: | undetermined Version: git-master | Resolution: Keywords: hevc h.265 | Blocked By: alpha transparency missing access | Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Changes (by johnhe4): * version: unspecified => git-master -- Ticket URL: <https://trac.ffmpeg.org/ticket/9088#comment:1> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9088: HEVC bitstream with alpha layer to mp4 -------------------------------------+------------------------------------- Reporter: johnhe4 | Owner: Type: defect | Status: new Priority: normal | Component: avformat Version: git-master | Resolution: Keywords: hevc mov | Blocked By: alpha | Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Changes (by cehoyos): * keywords: hevc h.265 alpha transparency missing access => hevc mov alpha * type: enhancement => defect * component: undetermined => avformat Comment: Please test current FFmpeg git head, the only version supported on this bug tracker. -- Ticket URL: <https://trac.ffmpeg.org/ticket/9088#comment:2> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9088: HEVC bitstream with alpha layer to mp4 -------------------------------------+------------------------------------- Reporter: johnhe4 | Owner: Type: defect | Status: new Priority: normal | Component: avformat Version: git-master | Resolution: Keywords: hevc mov | Blocked By: alpha | Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Comment (by Balling): It does not even ffplay correctly. What do you want from it? -- Ticket URL: <https://trac.ffmpeg.org/ticket/9088#comment:3> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9088: HEVC bitstream with alpha layer to mp4 -------------------------------------+------------------------------------- Reporter: johnhe4 | Owner: Type: defect | Status: new Priority: normal | Component: avformat Version: git-master | Resolution: Keywords: hevc mov | Blocked By: alpha | Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Comment (by johnhe4): Replying to [comment:2 cehoyos]:
Please test current FFmpeg git head, the only version supported on this bug tracker.
I tested again this morning with the same results {{{ % date Tue Feb 2 08:24:29 PST 2021 % git pull Already up to date. % git remote -v origin https://git.ffmpeg.org/ffmpeg.git (fetch) origin https://git.ffmpeg.org/ffmpeg.git (push) }}} -- Ticket URL: <https://trac.ffmpeg.org/ticket/9088#comment:4> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9088: HEVC bitstream with alpha layer to mp4 -------------------------------------+------------------------------------- Reporter: johnhe4 | Owner: Type: defect | Status: open Priority: normal | Component: avformat Version: git-master | Resolution: Keywords: hevc mov | Blocked By: alpha | Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Changes (by Balling): * status: new => open Comment: Realted: https://forums.developer.nvidia.com/t/hevc-with-alpha- layer/167353 Please, next time provide all the links. -- Ticket URL: <https://trac.ffmpeg.org/ticket/9088#comment:5> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9088: HEVC bitstream with alpha layer to mp4 -------------------------------------+------------------------------------- Reporter: johnhe4 | Owner: Type: defect | Status: open Priority: normal | Component: avformat Version: git-master | Resolution: Keywords: hevc mov | Blocked By: alpha | Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Comment (by cehoyos): Replying to [comment:5 Balling]:
Related: https://forums.developer.nvidia.com/t/hevc-with-alpha- layer/167353 https://github.com/johnhe4/nvenc_h265_transparency
Both look unrelated to this ticket.
Should not this be closed as a duplicate of #7965?
This ticket is about an issue with libavformat, the other ticket is libavcodec-only. -- Ticket URL: <https://trac.ffmpeg.org/ticket/9088#comment:6> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9088: HEVC bitstream with alpha layer to mp4 -------------------------------------+------------------------------------- Reporter: johnhe4 | Owner: Type: defect | Status: open Priority: normal | Component: avformat Version: git-master | Resolution: Keywords: hevc mov | Blocked By: alpha | Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Comment (by Balling):
Both look unrelated to this ticket.
Both (the code and nvidia comment) were done by the same user that reported this issue... -- Ticket URL: <https://trac.ffmpeg.org/ticket/9088#comment:7> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9088: HEVC bitstream with alpha layer to mp4 -------------------------------------+------------------------------------- Reporter: johnhe4 | Owner: Type: defect | Status: open Priority: normal | Component: avformat Version: git-master | Resolution: Keywords: hevc mov | Blocked By: alpha | Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Comment (by johnhe4): Yes, those links are related but I don't think they affect the scope. Sorry for not posting. No, this ticket is not a duplicate of #7965. Perhaps the best way to explain why is to detail my target use case. I am looking to segment out video using a Linux instance in the cloud. This segmentation process will result in two layers: a foreground layer and a background layer. I want to stream the foreground layer over the Internet to an Apple device that will render this foreground layer over whatever background the app chooses. I know of two traditional methods to accomplish this that make inefficient use of bandwidth and/or push complexity to the end device. Traditional method 1 is to send both layers as separate streams, or if possible, as separate tracks in a single stream. This requires the device to spin up two decoders, perform post and or pre-decode synchronization, and possibly deal with multiple-multiple bitrates. Traditional method 2 is to have the Linux server overwrite the background with a predefined chroma key. This requires a carefully chosen chroma key that will not unexpectedly occur in the foreground, and requires custom shaders on the end device. Alpha support in HEVC seems (to me) a much better solution than the traditional methods, if it is supported. This ticket is different than #7965 because it specifically asks for muxing support only. I would like FFmpeg to mux an existing HEVC bitstream which includes alpha (encoded with whatever, in this case NVidia but can be anything) into an MP4 (or TS file) that Apple can decode. From my limited understanding of FFmpeg, any changes relating to this ticket wouldn't need to touch the encoder or decoder but would instead touch muxers and parsers. Here is a link from Apple that includes more information for HEVC with alpha, including the profile constraints on Apple devices: https://developer.apple.com/videos/play/wwdc2019/506/ I noticed some activity on #7965, that is great to see. I imagine this ask will grow as more non-Apple encoders begin to support HEVC with alpha. -- Ticket URL: <https://trac.ffmpeg.org/ticket/9088#comment:8> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9088: HEVC bitstream with alpha layer to mp4 -------------------------------------+------------------------------------- Reporter: John | Owner: (none) Type: defect | Status: closed Priority: normal | Component: avformat Version: git-master | Resolution: Keywords: hevc mov | worksforme alpha | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Changes (by quinkblack): * status: open => closed * resolution: => worksforme Comment: Remux works fine with samples generated from Apple's videotoolbox. The reason to current issue is that nvenc alpha layer encoding is broken. http://ffmpeg.org/pipermail/ffmpeg-devel/2022-December/305204.html Nothing we can do until Nvidia fixed their SDK. -- Ticket URL: <https://trac.ffmpeg.org/ticket/9088#comment:9> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
participants (1)
-
FFmpeg