[FFmpeg-trac] #9371(undetermined:new): filter_complex doesn't work with HLS inputs - problems with handling "keep alive" error

FFmpeg trac at avcodec.org
Wed Aug 11 14:06:47 EEST 2021


#9371: filter_complex doesn't work with HLS inputs - problems with handling "keep
alive" error
-------------------------------------+-------------------------------------
             Reporter:               |                    Owner:  (none)
  codedigger_s9                      |
                 Type:  defect       |                   Status:  new
             Priority:  normal       |                Component:
                                     |  undetermined
              Version:  unspecified  |               Resolution:
             Keywords:               |               Blocked By:
  filter_complex HLS "keep alive"    |
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
Description changed by codedigger_s9:

Old description:

> Summary of the bug:
> It looks like "keep alive" doesn't work with HLS inputs used in
> filter_complex.
> How to reproduce:
> {{{
> ffmpeg -abort_on empty_output -y -nostats -progress pipe:1 -report \
>  -analyzeduration 20M -f lavfi -i color=c=black:s=1920x1080:r=30 \
>  -analyzeduration 20M -f lavfi -i anullsrc,volume=0 \
>  -analyzeduration 20M -t 61.582 -i
> "https://cfcdn.screen9.com/media/U/Q/UQWviTqgugtibCIToGKSkA_720p_hls/playlist.ssl.m3u8?auth
> =IsedbhZXM2HsxFy30RrWruXBqY-61BnmOuK9DGYcpGUghe6pUPHd60zxT-
> var346uFHOJBZ0v1wdLLg_rhBihw" \
>  -analyzeduration 20M -t 187.663 -i
> "https://cfcdn.screen9.com/media/U/Q/UQWviTqgugtibCIToGKSkA_480p_hls/playlist.ssl.m3u8?auth
> =IsedbhZXM2HsxFy30RrWruXBqY-61BnmOuK9DGYcpGUghe6pUPHd60zxT-
> var346uFHOJBZ0v1wdLLg_rhBihw" \
>  -analyzeduration 20M -t 107.393 -i
> "https://cfcdn.screen9.com/media/G/B/GBMMRHhJGktXgeOHHXaBzA_720p_hls/playlist.ssl.m3u8?auth=p2t92R0jDnleK5zkmUBEknDKXSRQ9fGt0lwM4vy2e0LCdGWdVnZTSd11lJEl9MfrO2JnvNKE2GyAm58q7qfFnA"
> \
>  -analyzeduration 20M -i
> "https://cfcdn.screen9.com/media/K/W/KWoaSWLrDMHrYjv5oN4lJQ_720p_png.png?auth
> =NX_fO9i2nwumtMxlpksfNZvGCTW-
> kxFoo6tqa_icMnERrl3mvYv1BHMttIZgVP905R96JkVoOQUMXYA2_MT3lg&Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9jZmNkbi5zY3JlZW45LmNvbS9tZWRpYS9LL1cvS1dvYVNXTHJETUhyWWp2NW9ONGxKUV83MjBwX3BuZy5wbmc~YXV0aD1OWF9mTzlpMm53dW10TXhscGtzZk5adkdDVFcta3hGb282dHFhX2ljTW5FUnJsM212WXYxQkhNdHRJWmdWUDkwNVI5NkprVm9PUVVNWFlBMl9NVDNsZyIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MjE0NzQ3MjAwMH19fV19&Signature=gOM3ajOjQMXGJzLknh019cau
> ~0P1H7xvs01PE93Ub1XviDSrm8FDy4YmydS60MsoosbVdGm83HFbuf12ZBrxtxTVssttYhW4w0pDUGx5fiS5sMjU46hPYpOz5mGjgNmTOqlvQpsiEPuqVck7uq7BM5gHl2avcyn8qSWCK5NlO9NxMzOpgi4SDi0xHeiK68Qu-
> YEkU~7j1Vmi2zDGf4J8I5zZem1UbRckE~BaBHcCdhF932gODtdzgyTNaqTt8yrB4nFxgL~S9kObvIrFkoF6BhS94ile7HKCm~Vfv1FnRQ8pRPxzDmBwSIS6Gnec1PTCzWcegiYLKV5Uz8H61MrHpQ__
> &Key-Pair-Id=APKAJLADG3ZP7IR3HDDQ" \
>  -analyzeduration 20M -f lavfi -i
> color=c=black at 0.0:s=1920x1080:r=30,format=yuva420p \
>  -analyzeduration 20M -i
> "https://cfcdn.screen9.com/media/w/m/wmlmZoCSrxGoBZRpYqTPFw_1440p_png.png?auth=jaKM4jqqg9VwPqNtFwN8lwfiEveo5
> -LbaKymG_w7w6NHf6T7qhNmdfg-
> ASsHrDFhh0sOudR1yVPrsfEaFbckxg&Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9jZmNkbi5zY3JlZW45LmNvbS9tZWRpYS93L20vd21sbVpvQ1NyeEdvQlpScFlxVFBGd18xNDQwcF9wbmcucG5nP2F1dGg9amFLTTRqcXFnOVZ3UHFOdEZ3Tjhsd2ZpRXZlbzUtTGJhS3ltR193N3c2TkhmNlQ3cWhObWRmZy1BU3NIckRGaGgwc091ZFIxeVZQcnNmRWFGYmNreGciLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjIxNDc0NzIwMDB9fX1dfQ__&Signature=Ouc77dneQBj~oIQwQnS3UjwmMw7M9-wpFS3Dm3WSvNm12wD~F4KowOsDVFYGrk-h
> -euQaSUjoAUf7jusSF70RTtTeXEcW-xcrEF2T3haAna73c5vX2Jj~pbq-
> ZpWFbqD2edGZWUmgldy-
> sZ4dxFhajrwhXNKFtllOzAKoQSsZjrORam6tKgeNNc5wYq6rbjEwKWL1B7T9QTe3ptYslHQOUY81C4V9vSI8qrJ35YGzvFZE33GN3mBB3TG5bly4-rKfH8NPVaC9yPpjtrUyh8gFDTmZNf3amyYgCFLD39eyGOjlrBFw
> ~sbGgDybpV7mpVYXH-zN~b8VxRa5reUzRzWNA__&Key-Pair-Id=APKAJLADG3ZP7IR3HDDQ"
> \
>  -analyzeduration 20M -f lavfi -i
> color=c=black at 0.0:s=1920x1080:r=30,format=yuva420p \
>  -map [o7] -t 364.756 -map [amix] -profile:v high -level 4.1 -async 1
> -map_metadata -1 -map_chapters -1 -sn -max_muxing_queue_size 10000 -g 150
> -keyint_min 150 -threads 0 -f hls -hls_time 5 -hls_list_size 0 -vcodec
> libx264 \
>  -filter_complex "\
>     [2:v]trim=start=0,setpts=PTS-
> STARTPTS+0/TB,scale=1920:1080:force_original_aspect_ratio=decrease,format=yuva420p,pad=1920:1080
> :(ow-iw)/2:(oh-ih)/2:#00000000,fps=30[v2]; \
>     [0:v][v2]overlay=x=0:y=0:enable='between(t,0,61.5815925505)'[o2]; \
>     [2:a]atrim=start=0,volume=1.0,adelay=delays=0|0,apad[a2]; \
>     [3:v]trim=start=0,setpts=PTS-
> STARTPTS+65.6544529109/TB,scale=1920:1080:force_original_aspect_ratio=decrease,format=yuva420p,pad=1920:1080
> :(ow-iw)/2:(oh-ih)/2:#00000000,fps=30[v3]; \
> [o2][v3]overlay=x=0:y=0:enable='between(t,65.6544529109,253.316931915)'[o3];
> \
>     [3:a]atrim=start=0,volume=1.0,adelay=delays=65654|65654,apad[a3]; \
>     [4:v]trim=start=0,setpts=PTS-
> STARTPTS+257.362866879/TB,scale=1920:1080:force_original_aspect_ratio=decrease,format=yuva420p,pad=1920:1080
> :(ow-iw)/2:(oh-ih)/2:#00000000,fps=30[v4]; \
> [o3][v4]overlay=x=0:y=0:enable='between(t,257.362866879,364.755866879)'[o4];
> \
>     [4:a]atrim=start=0,volume=1.0,adelay=delays=257363|257363,apad[a4]; \
> [5:v]scale=1920:1080:force_original_aspect_ratio=decrease,format=yuva420p,pad=1920:1080
> :(ow-iw)/2:(oh-ih)/2:#00000000[v5]; \
>     [6:v][v5]overlay[v6]; \
> [v6]fade=t=in:start_time=61.3297419729:duration=1:color=#efe8e8,fade=t=out:start_time=64.6544529109:duration=1:color=#ebebeb[v6f2];
> \
> [o4][v6f2]overlay=x=0:y=0:enable='between(t,61.3297419729,65.6544529109)'[o5];
> \
> [7:v]scale=1920:1080:force_original_aspect_ratio=decrease,format=yuva420p,pad=1920:1080
> :(ow-iw)/2:(oh-ih)/2:#00000000[v7]; \
>     [8:v][v7]overlay[v8]; \
> [v8]fade=t=in:start_time=253.269526335:duration=1:color=#f7f2f2,fade=t=out:start_time=256.362866879:duration=1:color=#f2eeee[v8f2];
> \
> [o5][v8f2]overlay=x=0:y=0:enable='between(t,253.269526335,257.362866879)'[o7];
> \
>     [1:a][a2][a3][a4]amix=inputs=4,volume=4[amix]" \
>  -b:v 4150k -maxrate 5187k -bufsize 5187k -pix_fmt yuv420p -preset fast
> -x264opts no-scenecut -acodec libfdk_aac -ac 2 -ab 256k  -ar 44100
> -profile:a aac_low \
>  -hls_segment_filename '%04d.ts' playlist.m3u8
> }}}
> Full command line is in the log. Probably any HLS inputs and similar
> filter will work.
>
> What was tested:
> * master with `--enable-gnutls`
> * master with `--enable-openssl`
> * different HLS demuxer options: `http_persistent`, `http_multiple`,
> `http_seekable` and probably more
> * high `thread_queue_size` seems to resolve the issue, but only because
> in some cases whole streams are cached in the memory and no http requests
> are made after initialization
> * different providers (local files are working correctly)
>
> With openssl error is slightly different:
> {{{
> [tls @ 0x2806ac0] error:00000000:lib(0):func(0):reason(0)
> [https @ 0x2830340] Opening 'https://some.playlist/0002.ts' for reading
> [tls @ 0x2806ac0] error:00000000:lib(0):func(0):reason(0)
> [hls @ 0x27add80] keepalive request failed for
> 'https://some.playlist/0002.ts' with error: 'Input/output error' when
> opening url, retrying with new connection
> [hls @ 0x27add80] Opening 'https://some.playlist/0002.ts' for reading
> [mpegts @ 0x2835700] Packet corrupt (stream = 0, dts = 390000).
> [mpegts @ 0x2835700] PES packet size mismatch
> [mpegts @ 0x2835700] Packet corrupt (stream = 1, dts = 384865).
> [aac @ 0x2896280] Number of bands (56) exceeds limit (44).
> Error while decoding stream #3:1: Invalid data found when processing
> input
> [aac @ 0x2896280] channel element 3.3 is not allocated
> Error while decoding stream #3:1: Invalid data found when processing
> input
> [tls @ 0x290c9c0] error:00000000:lib(0):func(0):reason(0)
> [https @ 0x2909e00] Opening 'https://some.playlist/0003.ts' for reading
> [tls @ 0x290c9c0] error:00000000:lib(0):func(0):reason(0)
> [hls @ 0x27add80] keepalive request failed for
> 'https://some.playlist/0003.ts' with error: 'Input/output error' when
> opening url, retrying with new connection
> [hls @ 0x27add80] Opening 'https://some.playlist/0003.ts' for reading
> [mpegts @ 0x2835700] Packet corrupt (stream = 0, dts = 660000).
> [h264 @ 0x2b10a40] concealing 7521 DC, 7521 AC, 7521 MV errors in B frame
> https://some.playlist/playlist.m3u8: corrupt decoded frame in stream 0}}}
> but the result is the same - output with broken audio and video.

New description:

 Summary of the bug:
 It looks like "keep alive" doesn't work with HLS inputs used in
 filter_complex.
 How to reproduce:
 {{{
 ffmpeg -abort_on empty_output -y -nostats -progress pipe:1 -report \
  -analyzeduration 20M -f lavfi -i color=c=black:s=1920x1080:r=30 \
  -analyzeduration 20M -f lavfi -i anullsrc,volume=0 \
  -analyzeduration 20M -t 61.582 -i
 "https://cfcdn.screen9.com/media/U/Q/UQWviTqgugtibCIToGKSkA_720p_hls/playlist.ssl.m3u8?auth
 =IsedbhZXM2HsxFy30RrWruXBqY-61BnmOuK9DGYcpGUghe6pUPHd60zxT-
 var346uFHOJBZ0v1wdLLg_rhBihw" \
  -analyzeduration 20M -t 187.663 -i
 "https://cfcdn.screen9.com/media/U/Q/UQWviTqgugtibCIToGKSkA_480p_hls/playlist.ssl.m3u8?auth
 =IsedbhZXM2HsxFy30RrWruXBqY-61BnmOuK9DGYcpGUghe6pUPHd60zxT-
 var346uFHOJBZ0v1wdLLg_rhBihw" \
  -analyzeduration 20M -t 107.393 -i
 "https://cfcdn.screen9.com/media/G/B/GBMMRHhJGktXgeOHHXaBzA_720p_hls/playlist.ssl.m3u8?auth=p2t92R0jDnleK5zkmUBEknDKXSRQ9fGt0lwM4vy2e0LCdGWdVnZTSd11lJEl9MfrO2JnvNKE2GyAm58q7qfFnA"
 \
  -analyzeduration 20M -i
 "https://cfcdn.screen9.com/media/K/W/KWoaSWLrDMHrYjv5oN4lJQ_720p_png.png?auth
 =NX_fO9i2nwumtMxlpksfNZvGCTW-
 kxFoo6tqa_icMnERrl3mvYv1BHMttIZgVP905R96JkVoOQUMXYA2_MT3lg&Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9jZmNkbi5zY3JlZW45LmNvbS9tZWRpYS9LL1cvS1dvYVNXTHJETUhyWWp2NW9ONGxKUV83MjBwX3BuZy5wbmc~YXV0aD1OWF9mTzlpMm53dW10TXhscGtzZk5adkdDVFcta3hGb282dHFhX2ljTW5FUnJsM212WXYxQkhNdHRJWmdWUDkwNVI5NkprVm9PUVVNWFlBMl9NVDNsZyIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MjE0NzQ3MjAwMH19fV19&Signature=gOM3ajOjQMXGJzLknh019cau
 ~0P1H7xvs01PE93Ub1XviDSrm8FDy4YmydS60MsoosbVdGm83HFbuf12ZBrxtxTVssttYhW4w0pDUGx5fiS5sMjU46hPYpOz5mGjgNmTOqlvQpsiEPuqVck7uq7BM5gHl2avcyn8qSWCK5NlO9NxMzOpgi4SDi0xHeiK68Qu-
 YEkU~7j1Vmi2zDGf4J8I5zZem1UbRckE~BaBHcCdhF932gODtdzgyTNaqTt8yrB4nFxgL~S9kObvIrFkoF6BhS94ile7HKCm~Vfv1FnRQ8pRPxzDmBwSIS6Gnec1PTCzWcegiYLKV5Uz8H61MrHpQ__
 &Key-Pair-Id=APKAJLADG3ZP7IR3HDDQ" \
  -analyzeduration 20M -f lavfi -i
 color=c=black at 0.0:s=1920x1080:r=30,format=yuva420p \
  -analyzeduration 20M -i
 "https://cfcdn.screen9.com/media/w/m/wmlmZoCSrxGoBZRpYqTPFw_1440p_png.png?auth=jaKM4jqqg9VwPqNtFwN8lwfiEveo5
 -LbaKymG_w7w6NHf6T7qhNmdfg-
 ASsHrDFhh0sOudR1yVPrsfEaFbckxg&Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9jZmNkbi5zY3JlZW45LmNvbS9tZWRpYS93L20vd21sbVpvQ1NyeEdvQlpScFlxVFBGd18xNDQwcF9wbmcucG5nP2F1dGg9amFLTTRqcXFnOVZ3UHFOdEZ3Tjhsd2ZpRXZlbzUtTGJhS3ltR193N3c2TkhmNlQ3cWhObWRmZy1BU3NIckRGaGgwc091ZFIxeVZQcnNmRWFGYmNreGciLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjIxNDc0NzIwMDB9fX1dfQ__&Signature=Ouc77dneQBj~oIQwQnS3UjwmMw7M9-wpFS3Dm3WSvNm12wD~F4KowOsDVFYGrk-h
 -euQaSUjoAUf7jusSF70RTtTeXEcW-xcrEF2T3haAna73c5vX2Jj~pbq-
 ZpWFbqD2edGZWUmgldy-
 sZ4dxFhajrwhXNKFtllOzAKoQSsZjrORam6tKgeNNc5wYq6rbjEwKWL1B7T9QTe3ptYslHQOUY81C4V9vSI8qrJ35YGzvFZE33GN3mBB3TG5bly4-rKfH8NPVaC9yPpjtrUyh8gFDTmZNf3amyYgCFLD39eyGOjlrBFw
 ~sbGgDybpV7mpVYXH-zN~b8VxRa5reUzRzWNA__&Key-Pair-Id=APKAJLADG3ZP7IR3HDDQ"
 \
  -analyzeduration 20M -f lavfi -i
 color=c=black at 0.0:s=1920x1080:r=30,format=yuva420p \
  -map [o7] -t 364.756 -map [amix] -profile:v high -level 4.1 -async 1
 -map_metadata -1 -map_chapters -1 -sn -max_muxing_queue_size 10000 -g 150
 -keyint_min 150 -threads 0 -f hls -hls_time 5 -hls_list_size 0 -vcodec
 libx264 \
  -filter_complex "\
     [2:v]trim=start=0,setpts=PTS-
 STARTPTS+0/TB,scale=1920:1080:force_original_aspect_ratio=decrease,format=yuva420p,pad=1920:1080
 :(ow-iw)/2:(oh-ih)/2:#00000000,fps=30[v2]; \
     [0:v][v2]overlay=x=0:y=0:enable='between(t,0,61.5815925505)'[o2]; \
     [2:a]atrim=start=0,volume=1.0,adelay=delays=0|0,apad[a2]; \
     [3:v]trim=start=0,setpts=PTS-
 STARTPTS+65.6544529109/TB,scale=1920:1080:force_original_aspect_ratio=decrease,format=yuva420p,pad=1920:1080
 :(ow-iw)/2:(oh-ih)/2:#00000000,fps=30[v3]; \
 [o2][v3]overlay=x=0:y=0:enable='between(t,65.6544529109,253.316931915)'[o3];
 \
     [3:a]atrim=start=0,volume=1.0,adelay=delays=65654|65654,apad[a3]; \
     [4:v]trim=start=0,setpts=PTS-
 STARTPTS+257.362866879/TB,scale=1920:1080:force_original_aspect_ratio=decrease,format=yuva420p,pad=1920:1080
 :(ow-iw)/2:(oh-ih)/2:#00000000,fps=30[v4]; \
 [o3][v4]overlay=x=0:y=0:enable='between(t,257.362866879,364.755866879)'[o4];
 \
     [4:a]atrim=start=0,volume=1.0,adelay=delays=257363|257363,apad[a4]; \
 [5:v]scale=1920:1080:force_original_aspect_ratio=decrease,format=yuva420p,pad=1920:1080
 :(ow-iw)/2:(oh-ih)/2:#00000000[v5]; \
     [6:v][v5]overlay[v6]; \
 [v6]fade=t=in:start_time=61.3297419729:duration=1:color=#efe8e8,fade=t=out:start_time=64.6544529109:duration=1:color=#ebebeb[v6f2];
 \
 [o4][v6f2]overlay=x=0:y=0:enable='between(t,61.3297419729,65.6544529109)'[o5];
 \
 [7:v]scale=1920:1080:force_original_aspect_ratio=decrease,format=yuva420p,pad=1920:1080
 :(ow-iw)/2:(oh-ih)/2:#00000000[v7]; \
     [8:v][v7]overlay[v8]; \
 [v8]fade=t=in:start_time=253.269526335:duration=1:color=#f7f2f2,fade=t=out:start_time=256.362866879:duration=1:color=#f2eeee[v8f2];
 \
 [o5][v8f2]overlay=x=0:y=0:enable='between(t,253.269526335,257.362866879)'[o7];
 \
     [1:a][a2][a3][a4]amix=inputs=4,volume=4[amix]" \
  -b:v 4150k -maxrate 5187k -bufsize 5187k -pix_fmt yuv420p -preset fast
 -x264opts no-scenecut -acodec libfdk_aac -ac 2 -ab 256k  -ar 44100
 -profile:a aac_low \
  -hls_segment_filename '%04d.ts' playlist.m3u8
 }}}
 Full command line is in the log. Probably any HLS inputs and similar
 filter will work.

 What was tested:
 * master with `--enable-gnutls`
 * master with `--enable-openssl`
 * different HLS demuxer options: `http_persistent`, `http_multiple`,
 `http_seekable` and probably more
 * high `thread_queue_size` seems to resolve the issue, but only because in
 some cases whole streams are cached in the memory and no http requests are
 made after initialization
 * different providers (local files are working correctly)

 With openssl error is slightly different:
 {{{
 [tls @ 0x2806ac0] error:00000000:lib(0):func(0):reason(0)
 [https @ 0x2830340] Opening 'https://some.playlist/0002.ts' for reading
 [tls @ 0x2806ac0] error:00000000:lib(0):func(0):reason(0)
 [hls @ 0x27add80] keepalive request failed for
 'https://some.playlist/0002.ts' with error: 'Input/output error' when
 opening url, retrying with new connection
 [hls @ 0x27add80] Opening 'https://some.playlist/0002.ts' for reading
 [mpegts @ 0x2835700] Packet corrupt (stream = 0, dts = 390000).
 [mpegts @ 0x2835700] PES packet size mismatch
 [mpegts @ 0x2835700] Packet corrupt (stream = 1, dts = 384865).
 [aac @ 0x2896280] Number of bands (56) exceeds limit (44).
 Error while decoding stream #3:1: Invalid data found when processing input
 [aac @ 0x2896280] channel element 3.3 is not allocated
 Error while decoding stream #3:1: Invalid data found when processing input
 [tls @ 0x290c9c0] error:00000000:lib(0):func(0):reason(0)
 [https @ 0x2909e00] Opening 'https://some.playlist/0003.ts' for reading
 [tls @ 0x290c9c0] error:00000000:lib(0):func(0):reason(0)
 [hls @ 0x27add80] keepalive request failed for
 'https://some.playlist/0003.ts' with error: 'Input/output error' when
 opening url, retrying with new connection
 [hls @ 0x27add80] Opening 'https://some.playlist/0003.ts' for reading
 [mpegts @ 0x2835700] Packet corrupt (stream = 0, dts = 660000).
 [h264 @ 0x2b10a40] concealing 7521 DC, 7521 AC, 7521 MV errors in B frame
 https://some.playlist/playlist.m3u8: corrupt decoded frame in stream 0
 }}}
 but the result is the same - output with broken audio and video.

--
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/9371#comment:2>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker


More information about the FFmpeg-trac mailing list