[FFmpeg-devel] [PATCHv2] avutil/file_open: avoid file handle inheritance on Windows

[Tobias Rapp]

On 30.10.2015 15:06, wm4 wrote:
> On Fri, 30 Oct 2015 11:30:40 +0100
> Tobias Rapp <t.rapp at noa-archive.com> wrote:
>
>> On 29.10.2015 09:38, Tobias Rapp wrote:
>>> Attached patch fixes file lock issues in my Windows application when a
>>> child process is started with handle inheritance enabled (standard
>>> input/output redirection) while a FFmpeg transcoding is running in the
>>> parent process.
>>>
>>> BTW: It would be great if the patch could also be applied to the 2.7/2.8
>>> release branches.
>>
>> Forgot to add links relevant to the subject.
>>
>> [1] https://msdn.microsoft.com/en-us/library/w7sa2b22.aspx
>>
>> Describes the _wsopen() function and the O_NOINHERIT flag. File handles
>> opened by _wsopen() are inheritable by default.
>>
>> [2]
>> https://msdn.microsoft.com/en-us/library/windows/desktop/aa363858%28v=vs.85%29.aspx
>>
>> Describes the CreateFile() function. Not directly relevant here, often
>> used in cases outside of C/libc. Opens file without inheritance by
>> default (lpSecurityAttributes is NULL).
>>
>> [3]
>> https://msdn.microsoft.com/en-us/library/windows/desktop/ms682425%28v=vs.85%29.aspx
>>
>> Describes handle inheritance when creating new processes. Handle
>> inheritance must be enabled (bInheritHandles = TRUE) e.g. when you want
>> to pass handles for stdin/stdout via lpStartupInfo.
>
> Would be great if you could put all this stuff into the commit message.

Have attached a more verbose version of the patch.

> Would it make sense to define O_CLOEXEC to O_NOINHERIT on Windows?

Although to my knowledge O_CLOEXEC and O_NOINHERIT cause the same 
behavior it would be confusing to trace when O_CLOEXEC maps to the 
original Linux definition and when it maps to the Windows override when 
reading source code. If someone finds it useful a definition like 
FF_O_CLOEXEC could be added (to which file?) that maps to the correct 
flag depending on the platform.

Regards,
Tobias
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-avutil-file_open-avoid-file-handle-inheritance-on-Wi.patch
Type: text/x-patch
Size: 1537 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20151030/59062829/attachment.bin>