[FFmpeg-devel] Network IO in FFmpeg (was: abstractable threading api)

[Olivier Langlois]

On Mon, 2014-04-07 at 14:56 -0600, Roger Pack wrote:
> On 12/19/13, Nicolas George <george at nsup.org> wrote:
> > Le sextidi 26 frimaire, an CCXXII, Hendrik Leppkes a écrit :
> >> The compat isn't complete, it just contains the functions needed by
> >> avcodec et al, well short of pthread_cancel.
> >> However, providing pthread_cancel with win32 threading api is also
> >> non-trivial, if not impossible to find a "drop in" compat
> >> functionality.
> >>
> >> Canceling threads from the outside is a no-no in the win32 API. Most
> >> blocking functions usually have the option to give it a unblock event
> >> that you can trigger to release it, and safely shut down the thread
> >> from within, however as you can imagine thats not something that ports
> >> well over different APIs.
> >>
> >> The best course of action in this case would probably be to find a way
> >> to re-write this worker thread without pthread_cancel, other solutions
> >> seem rather ugly.
> >
> > Regarding the pthread_cancel thing, I am the one responsible for this bit
> > of
> > code, I can explain how it is needed.
> >
> > The basic idea is that we have a thread blocked inside a system call:
> >
> > 	read(udp_sock_fd, buffer, size);
> >
> > and we want to terminate it, because reading on the socket is finished.
> > Note
> > that no packet may arrive there for a long time, so waiting for the system
> > call to return by itself is not an option.
> >
> > First, a small discussion: in this particular case, we want to stop the
> > whole thread. If we had a way of interrupting just the system call,
> > stopping
> > the thread is just a matter of checking a flag afterwards. If the situation
> > were inverted: we knew how to cancel the thread but want to interrupt just
> > the system call, then we could start a new thread just for the system call:
> > more work, but that does the trick. In short: whether we cancel the system
> > call or the whole thread, we are good.
> >
> > Now, let us see the options:
> >
> > * Signals
> >
> >   Using a signal to interrupt a system call is a long tradition, but
> >   handling signals reliably is a portability nightmare, and mixing threads
> >   with signals is that squared. Let us avoid it.
> >
> > * pthread_cancel()
> >
> >   With true POSIX threads, it is perfect: pthread_cancel() only interrupts
> >   at specific points of the code, so we do not need to care about
> > atomicity,
> >   and reading on a socket is guaranteed to be one.
> >
> > * Closing the socket
> >
> >   ... does not work.
> >
> > * The asynchronous I/O API aio_*
> >
> >   It could work, but I am afraid it does not. This is a seldom used API,
> > and
> >   thus it will have a lot of pitfalls. On GNU/Linux, aio_* is implemented
> >   with hidden threads; furthermore, a blocking operation is not really
> >   cancelled.
> >
> > * Non-blocking I/O + usleep() + retry loop / poll() with periodic timeout
> >
> >   The "poll() with periodic timeout" solution was used before I changed it
> > to
> >   pthread_cancel(). Both solution work, but they are universally frowned
> >   upon.
> 

Hi,

This thread has caught my attention and I went to see how
pthread_cancel() was used in libavformat/udp.c and I might have spotted
a race condition in it.

cancellation points are implemented inside glibc not in the kernel. That
means that once the cancellation point check is done and the thread is
blocked inside the kernel, cancelling the thread will have no effect.

In order to have the code behave as designed,

pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL);

should be used. asynchronous cancels are implemented with RT signals.
For that reason only async cancels can interrupt and cancel a thread
that is already inside a system call.

https://github.com/lano1106/glcs/blob/master/src/glc/common/signal.c

is an example where I have implemented a timed_waitpid() by using a
thread and pthread_cancel().

if there is interest, I can point you to source files in glibc to look
to understand how pthread_cancel is implemented in glibc. Just let me
know,

Greetings,
Olivier