[FFmpeg-devel] [FFmpeg-cvslog] libavcodec/libx264: add user data unregistered SEI encoding

[James Almer]

On 10/18/2021 11:29 PM, Brad Hards wrote:
> On Tuesday, 19 October 2021 9:53:50 AM AEDT James Almer wrote:
>> On 10/18/2021 7:38 PM, James Almer wrote:
>>> On 10/18/2021 7:34 PM, Derek Buitenhuis wrote:
>>>> On 10/18/2021 11:05 PM, Michael Niedermayer wrote:
>>>>> This code breaks bitexact mode for some files, i have not looked into
>>>>> this
>>>>> any deeper yet.
>>>>
>>>> [23:18] <@jamrial_> Daemon404: that libx264 patch conflicts with the
>>>> closed caption code. it overwrites the extra_sei struct and leaks it
>>>>
>>>> ^ presumably this is why
>>>
>>> Actually no. It calls av_fast_realloc() on a (potentially) av_malloc'd
>>> buffer, which i think is a valid scenario. Also, these samples have no
>>> closed captions, so the relevant code is never run.
>>>
>>> The problem i guess is in the following line:
>>>
>>> sei_payload->payload = side_data->data;
>>>
>>> Which should be an av_memdup() or similar, because side_data->data is
>>> the frame's side data, and by the time the encoder tries to do something
>>> with it, it may have been freed.
>>> Also, this code should set x4->pic.extra_sei.sei_free to av_free().
>>
>> The following should fix it
> My mail client mangled the patch, but manually applying it worked with my test case. I
> don't know what the issue is with the bitexact issue identifier by Michael, but will keep
> looking.

I assume that bitexact issue was libx264 reading uninitialized memory, 
since you were passing it a pointer to a buffer that was no longer valid.
I fixed it last night in 66f8055c89 and it now works as intended.