[PATCH 0/5] FIFO meta muxer related improvements
The primary parts of this are patches 1,4,5. 2 and 3 were just noticed when poking at the recovery timestamp logic, where the stream-time comparison logic seemed somewhat weird (such as comparing the pts to last_recovery_ts only if last_recovery_ts == AV_NOPTS_VALUE). If they seem incorrectly understood, they can be left out. As for the rest of the patches, they address the following issues: 1. Even if the header and the first packet get written out, the muxer might still fail at writing to a server when it actually decides to do I/O. Currently (for example with the MP4 muxer) this leads to a constant retry loop as each recovery is actually "successful". This patch makes sure that even if a recovery is "successful", the following recovery will only happen after the configured time, easing the load on any receiving server. 4. In case of avformat_write_header failing, the fifo muxer until now would not close the avformat context and its underlying I/O. This is now added, so that file sockets do not keep creeping up. 5. Unset a configured codec_tag if it is not supported by the underlying muxer, as the API client has no visibility regarding whether a codec tag is acceptable in cases of meta-muxers such as fifo. Jan Bernard Boulay (1): avformat/fifo: always wait recovery_wait_time between recoveries Jan Ekström (4): avformat/fifo: fix handling of stream-time non-NOPTS recovery avformat/fifo: cause immediate stream-time recovery if time went backwards avformat/fifo: close IO in case header writing fails avformat/fifo: unset codec tag if unsupported by underlying muxer libavformat/fifo.c | 77 ++++++++++++++++++++++++++++++---------------- 1 file changed, 51 insertions(+), 26 deletions(-) -- 2.29.2
From: Bernard Boulay <bernard.boulay@24i.com> With various formats/protocols - for example fragmented mp4 over HTTP - success/failure might not appear right away. The header and first packet might get successfully written into the muxer, but a response of failure might appear only N packets later. In such a case, a recovery attempt would always be the "first time", and it would thus be executed without delay, causing effectively a DoS towards the server on the other side of the connection. Thus, modify the logic of the module so that it takes into account the last time recovery was attempted - even if it seemed to be successful: 1. Refactor update of last_recovery_ts to its own helper function. 2. Call this function even if the recovery is successful, thus ensuring that the time is always set. 3. Move initialization of last_recovery_ts to be handled together with the thread initialization. 4. Do not limit sleeping in recovery based on recovery_nr being nonzero ("not the first time of attempted recovery"). Signed-off-by: Jan Ekström <jan.ekstrom@24i.com> --- libavformat/fifo.c | 54 +++++++++++++++++++++++++--------------------- 1 file changed, 30 insertions(+), 24 deletions(-) diff --git a/libavformat/fifo.c b/libavformat/fifo.c index 17748e94ce5..55cbf14ae93 100644 --- a/libavformat/fifo.c +++ b/libavformat/fifo.c @@ -279,15 +279,10 @@ static void free_message(void *msg) av_packet_unref(&fifo_msg->pkt); } -static int fifo_thread_process_recovery_failure(FifoThreadContext *ctx, AVPacket *pkt, - int err_no) +static void fifo_thread_set_last_recovery_ts(FifoThreadContext *ctx, AVPacket *pkt) { AVFormatContext *avf = ctx->avf; FifoContext *fifo = avf->priv_data; - int ret; - - av_log(avf, AV_LOG_INFO, "Recovery failed: %s\n", - av_err2str(err_no)); if (fifo->recovery_wait_streamtime) { if (pkt->pts == AV_NOPTS_VALUE) @@ -297,6 +292,19 @@ static int fifo_thread_process_recovery_failure(FifoThreadContext *ctx, AVPacket } else { ctx->last_recovery_ts = av_gettime_relative(); } +} + +static int fifo_thread_process_recovery_failure(FifoThreadContext *ctx, AVPacket *pkt, + int err_no) +{ + AVFormatContext *avf = ctx->avf; + FifoContext *fifo = avf->priv_data; + int ret; + + av_log(avf, AV_LOG_INFO, "Recovery failed: %s\n", + av_err2str(err_no)); + + fifo_thread_set_last_recovery_ts(ctx, pkt); if (fifo->max_recovery_attempts && ctx->recovery_nr >= fifo->max_recovery_attempts) { @@ -329,27 +337,22 @@ static int fifo_thread_attempt_recovery(FifoThreadContext *ctx, FifoMessage *msg ctx->header_written = 0; } - if (!ctx->recovery_nr) { - ctx->last_recovery_ts = fifo->recovery_wait_streamtime ? - AV_NOPTS_VALUE : 0; - } else { - if (fifo->recovery_wait_streamtime) { - if (ctx->last_recovery_ts == AV_NOPTS_VALUE) { - AVRational tb = avf->streams[pkt->stream_index]->time_base; - time_since_recovery = av_rescale_q(pkt->pts - ctx->last_recovery_ts, - tb, AV_TIME_BASE_Q); - } else { - /* Enforce recovery immediately */ - time_since_recovery = fifo->recovery_wait_time; - } + if (fifo->recovery_wait_streamtime) { + if (ctx->last_recovery_ts == AV_NOPTS_VALUE) { + AVRational tb = avf->streams[pkt->stream_index]->time_base; + time_since_recovery = av_rescale_q(pkt->pts - ctx->last_recovery_ts, + tb, AV_TIME_BASE_Q); } else { - time_since_recovery = av_gettime_relative() - ctx->last_recovery_ts; + /* Enforce recovery immediately */ + time_since_recovery = fifo->recovery_wait_time; } - - if (time_since_recovery < fifo->recovery_wait_time) - return AVERROR(EAGAIN); + } else { + time_since_recovery = av_gettime_relative() - ctx->last_recovery_ts; } + if (time_since_recovery < fifo->recovery_wait_time) + return AVERROR(EAGAIN); + ctx->recovery_nr++; if (fifo->max_recovery_attempts) { @@ -372,6 +375,7 @@ static int fifo_thread_attempt_recovery(FifoThreadContext *ctx, FifoMessage *msg } } else { av_log(avf, AV_LOG_INFO, "Recovery successful\n"); + fifo_thread_set_last_recovery_ts(ctx, pkt); ctx->recovery_nr = 0; } @@ -389,7 +393,7 @@ static int fifo_thread_recover(FifoThreadContext *ctx, FifoMessage *msg, int err int ret; do { - if (!fifo->recovery_wait_streamtime && ctx->recovery_nr > 0) { + if (!fifo->recovery_wait_streamtime) { int64_t time_since_recovery = av_gettime_relative() - ctx->last_recovery_ts; int64_t time_to_wait = FFMAX(0, fifo->recovery_wait_time - time_since_recovery); if (time_to_wait) @@ -420,6 +424,8 @@ static void *fifo_consumer_thread(void *data) memset(&fifo_thread_ctx, 0, sizeof(FifoThreadContext)); fifo_thread_ctx.avf = avf; fifo_thread_ctx.last_received_dts = AV_NOPTS_VALUE; + fifo_thread_ctx.last_recovery_ts = fifo->recovery_wait_streamtime ? + AV_NOPTS_VALUE : 0; while (1) { uint8_t just_flushed = 0; -- 2.29.2
From: Jan Ekström <jan.ekstrom@24i.com> Until now the proper diff between last_recovery_ts and packet's pts was only done when last_recovery_ts was AV_NOPTS_VALUE. This is kind of an oxymoron as the result of comparing that difference to AV_NOPTS_VALUE most likely always be relatively large. Thus, move the immediate recovery to when last_recovery_ts is AV_NOPTS_VALUE, and do a proper difference between the values otherwise. Signed-off-by: Jan Ekström <jan.ekstrom@24i.com> --- libavformat/fifo.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/libavformat/fifo.c b/libavformat/fifo.c index 55cbf14ae93..b940d0dafe1 100644 --- a/libavformat/fifo.c +++ b/libavformat/fifo.c @@ -339,12 +339,12 @@ static int fifo_thread_attempt_recovery(FifoThreadContext *ctx, FifoMessage *msg if (fifo->recovery_wait_streamtime) { if (ctx->last_recovery_ts == AV_NOPTS_VALUE) { + /* Enforce recovery immediately */ + time_since_recovery = fifo->recovery_wait_time; + } else { AVRational tb = avf->streams[pkt->stream_index]->time_base; time_since_recovery = av_rescale_q(pkt->pts - ctx->last_recovery_ts, tb, AV_TIME_BASE_Q); - } else { - /* Enforce recovery immediately */ - time_since_recovery = fifo->recovery_wait_time; } } else { time_since_recovery = av_gettime_relative() - ctx->last_recovery_ts; -- 2.29.2
From: Jan Ekström <jan.ekstrom@24i.com> This way discontinuities backwards could also get handled (albeit they generally tend to be incorrect to begin with on this layer of avformat). Signed-off-by: Jan Ekström <jan.ekstrom@24i.com> --- libavformat/fifo.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/libavformat/fifo.c b/libavformat/fifo.c index b940d0dafe1..9fc524adbdf 100644 --- a/libavformat/fifo.c +++ b/libavformat/fifo.c @@ -338,7 +338,8 @@ static int fifo_thread_attempt_recovery(FifoThreadContext *ctx, FifoMessage *msg } if (fifo->recovery_wait_streamtime) { - if (ctx->last_recovery_ts == AV_NOPTS_VALUE) { + if (ctx->last_recovery_ts == AV_NOPTS_VALUE || + pkt->pts < ctx->last_recovery_ts) { /* Enforce recovery immediately */ time_since_recovery = fifo->recovery_wait_time; } else { -- 2.29.2
From: Jan Ekström <jan.ekstrom@24i.com> Otherwise each new attempt will leave its connection open. Signed-off-by: Jan Ekström <jan.ekstrom@24i.com> --- libavformat/fifo.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/libavformat/fifo.c b/libavformat/fifo.c index 9fc524adbdf..bc363e458cd 100644 --- a/libavformat/fifo.c +++ b/libavformat/fifo.c @@ -142,8 +142,14 @@ static int fifo_thread_write_header(FifoThreadContext *ctx) avf2->streams[i]->cur_dts = 0; ret = avformat_write_header(avf2, &format_options); - if (!ret) - ctx->header_written = 1; + if (ret < 0) { + av_log(avf2, AV_LOG_ERROR, + "Failed to write format header: %s\n", av_err2str(ret)); + ff_format_io_close(avf2, &avf2->pb); + goto end; + } + + ctx->header_written = 1; // Check for options unrecognized by underlying muxer if (format_options) { -- 2.29.2
From: Jan Ekström <jan.ekstrom@24i.com> The API client has no visibility into the values provided by the underlying muxer, so we either do this always, or when an option is presented. In this case, we do it unconditionally, as this is probably more useful. Signed-off-by: Jan Ekström <jan.ekstrom@24i.com> --- libavformat/fifo.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/libavformat/fifo.c b/libavformat/fifo.c index bc363e458cd..c9b782c2309 100644 --- a/libavformat/fifo.c +++ b/libavformat/fifo.c @@ -513,6 +513,18 @@ static int fifo_mux_init(AVFormatContext *avf, ff_const59 AVOutputFormat *oforma ret = ff_stream_encode_params_copy(st, avf->streams[i]); if (ret < 0) return ret; + + if ((st->codecpar->codec_tag && oformat->codec_tag) && + av_codec_get_id(oformat->codec_tag, st->codecpar->codec_tag) != st->codecpar->codec_id) { + av_log(avf, AV_LOG_ERROR, + "Codec tag '%s' is unsupported for codec %s in the %s muxer! " + "Unsetting codec tag so that auto-configuration can be " + "attempted.\n", + av_fourcc2str(st->codecpar->codec_tag), + avcodec_get_name(st->codecpar->codec_id), + avf2->oformat->name); + st->codecpar->codec_tag = 0; + } } return 0; -- 2.29.2
On Mon, Dec 7, 2020 at 12:08 PM Jan Ekström <jeebjp@gmail.com> wrote:
The primary parts of this are patches 1,4,5. 2 and 3 were just noticed when poking at the recovery timestamp logic, where the stream-time comparison logic seemed somewhat weird (such as comparing the pts to last_recovery_ts only if last_recovery_ts == AV_NOPTS_VALUE). If they seem incorrectly understood, they can be left out.
As for the rest of the patches, they address the following issues: 1. Even if the header and the first packet get written out, the muxer might still fail at writing to a server when it actually decides to do I/O. Currently (for example with the MP4 muxer) this leads to a constant retry loop as each recovery is actually "successful". This patch makes sure that even if a recovery is "successful", the following recovery will only happen after the configured time, easing the load on any receiving server. 4. In case of avformat_write_header failing, the fifo muxer until now would not close the avformat context and its underlying I/O. This is now added, so that file sockets do not keep creeping up. 5. Unset a configured codec_tag if it is not supported by the underlying muxer, as the API client has no visibility regarding whether a codec tag is acceptable in cases of meta-muxers such as fifo.
Jan
Bernard Boulay (1): avformat/fifo: always wait recovery_wait_time between recoveries
Jan Ekström (4): avformat/fifo: fix handling of stream-time non-NOPTS recovery avformat/fifo: cause immediate stream-time recovery if time went backwards avformat/fifo: close IO in case header writing fails avformat/fifo: unset codec tag if unsupported by underlying muxer
libavformat/fifo.c | 77 ++++++++++++++++++++++++++++++---------------- 1 file changed, 51 insertions(+), 26 deletions(-)
Ping for this patch set. Jan
On Mon, Jan 11, 2021 at 4:46 PM Jan Ekström <jeebjp@gmail.com> wrote:
On Mon, Dec 7, 2020 at 12:08 PM Jan Ekström <jeebjp@gmail.com> wrote:
The primary parts of this are patches 1,4,5. 2 and 3 were just noticed when poking at the recovery timestamp logic, where the stream-time comparison logic seemed somewhat weird (such as comparing the pts to last_recovery_ts only if last_recovery_ts == AV_NOPTS_VALUE). If they seem incorrectly understood, they can be left out.
As for the rest of the patches, they address the following issues: 1. Even if the header and the first packet get written out, the muxer might still fail at writing to a server when it actually decides to do I/O. Currently (for example with the MP4 muxer) this leads to a constant retry loop as each recovery is actually "successful". This patch makes sure that even if a recovery is "successful", the following recovery will only happen after the configured time, easing the load on any receiving server. 4. In case of avformat_write_header failing, the fifo muxer until now would not close the avformat context and its underlying I/O. This is now added, so that file sockets do not keep creeping up. 5. Unset a configured codec_tag if it is not supported by the underlying muxer, as the API client has no visibility regarding whether a codec tag is acceptable in cases of meta-muxers such as fifo.
Jan
Bernard Boulay (1): avformat/fifo: always wait recovery_wait_time between recoveries
Jan Ekström (4): avformat/fifo: fix handling of stream-time non-NOPTS recovery avformat/fifo: cause immediate stream-time recovery if time went backwards avformat/fifo: close IO in case header writing fails avformat/fifo: unset codec tag if unsupported by underlying muxer
libavformat/fifo.c | 77 ++++++++++++++++++++++++++++++---------------- 1 file changed, 51 insertions(+), 26 deletions(-)
Ping for this patch set.
Ping for this patch set. Jan
On Mon, Jan 25, 2021 at 10:01 AM Jan Ekström <jeebjp@gmail.com> wrote:
On Mon, Jan 11, 2021 at 4:46 PM Jan Ekström <jeebjp@gmail.com> wrote:
On Mon, Dec 7, 2020 at 12:08 PM Jan Ekström <jeebjp@gmail.com> wrote:
The primary parts of this are patches 1,4,5. 2 and 3 were just noticed when poking at the recovery timestamp logic, where the stream-time comparison logic seemed somewhat weird (such as comparing the pts to last_recovery_ts only if last_recovery_ts == AV_NOPTS_VALUE). If they seem incorrectly understood, they can be left out.
As for the rest of the patches, they address the following issues: 1. Even if the header and the first packet get written out, the muxer might still fail at writing to a server when it actually decides to do I/O. Currently (for example with the MP4 muxer) this leads to a constant retry loop as each recovery is actually "successful". This patch makes sure that even if a recovery is "successful", the following recovery will only happen after the configured time, easing the load on any receiving server. 4. In case of avformat_write_header failing, the fifo muxer until now would not close the avformat context and its underlying I/O. This is now added, so that file sockets do not keep creeping up. 5. Unset a configured codec_tag if it is not supported by the underlying muxer, as the API client has no visibility regarding whether a codec tag is acceptable in cases of meta-muxers such as fifo.
Jan
Bernard Boulay (1): avformat/fifo: always wait recovery_wait_time between recoveries
Jan Ekström (4): avformat/fifo: fix handling of stream-time non-NOPTS recovery avformat/fifo: cause immediate stream-time recovery if time went backwards avformat/fifo: close IO in case header writing fails avformat/fifo: unset codec tag if unsupported by underlying muxer
libavformat/fifo.c | 77 ++++++++++++++++++++++++++++++---------------- 1 file changed, 51 insertions(+), 26 deletions(-)
Ping for this patch set.
Ping for this patch set.
As there were no comments... I think that while this is a behavioral change (you no longer get an immediate retry in case you have recovery_wait_time set with the first attempt): 1. the current way of doing recovery waits can cause DoS with muxers such as movenc - which can return success initially (and thus the retry counter is reset), but then fail to actually write actual data out 2. I do not think we can ask from the muxer whether it actually sent data out through the I/O or not. Thus, just making the fifo meta muxer always wait for the recovery_wait_time makes sense. Additionally, there are some bug fixes that I found, such as the forgotten closing of I/O, or unsetting the codec tag if it is not supported by the actual underlying format (since the API client has no visibility of the underlying format). Jan
participants (1)
-
Jan Ekström