[FFmpeg-trac] #2976(undetermined:new): m3u8 playlist download fails if the lines in the manifests are longer than 1024 characters

[FFmpeg]

#2976: m3u8 playlist download fails if the lines in the manifests are longer than
1024 characters
-------------------------------------+-------------------------------------
             Reporter:  jaimeMF      |                     Type:  defect
               Status:  new          |                 Priority:  normal
            Component:               |                  Version:  git-
  undetermined                       |  master
             Keywords:               |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 I'm using ffmpeg for watching Youtube's m3u8 streams and I've found that
 it fails if the lines in the manifest contains more than 1024 characters

 How to reproduce:
 using the version from git and commit
 8ad2465987d31bb818235303c0b02b897cd02133, I use this script (requires
 youtube-dl):
 {{{#!sh
 URL=$(youtube-dl 'http://www.youtube.com/watch?v=rdcvn4W-6xM' -g)
 echo $URL
 ./ffmpeg -y -i $URL -f mp4 hls.mp4
 }}}
 (You can manually find the manifest url in the video page)

 The output I get is:
 {{{
 http://www.youtube.com/api/manifest/hls_playlist/id/add72f9f85beeb13/itag/96/source/youtube/ratebypass/yes/pfa/1/pbr/yes/hls_chunk_host/www.youtube.com/gir/yes/clen/5547517760/lmt/1347749241948586/dur/8870.400/pmbypass/yes/playlist_type/DVR/cp/U0hWTldTVl9NU0NONl9PTVpGOl9QaC1zZ1NuSnEx/maudio/1/sver/3/fexp/903802,927703,932204,928023,924606,929117,929121,929906,929907,929922,929923,929127,929129,929131,929930,936403,925724,925726,936310,925720,925722,925718,925714,929917,906945,929933,929935,939604,906842,927704,913428,919811,932309,913563,919373,930803,908536,938701,931924,934005,940501,936308,909549,901608,900816,912711,934507,912714,907231,936312,906001/upn/5htNr8XQdyc/ip/{my_ip}/ipbits/8/expire/1379788589/sparams/ip,ipbits,expire,id,itag,source,ratebypass,pfa,pbr,hls_chunk_host,gir,clen,lmt,dur,pmbypass,playlist_type,cp,maudio/signature/6EC01EDEEB7D161229D81AD728A68C9C1154ECAD.4D9721C03E7CC061A5093D83355957C08AACA696/key/dg_yt0/file/index.m3u8
 ffmpeg version N-56538-g8ad2465 Copyright (c) 2000-2013 the FFmpeg
 developers
   built on Sep 21 2013 14:54:00 with llvm-gcc 4.2.1 (LLVM build
 2336.11.00)
   configuration: --disable-ffplay --disable-ffprobe --disable-ffserver
   libavutil      52. 45.100 / 52. 45.100
   libavcodec     55. 33.100 / 55. 33.100
   libavformat    55. 18.102 / 55. 18.102
   libavdevice    55.  3.100 / 55.  3.100
   libavfilter     3. 86.102 /  3. 86.102
   libswscale      2.  5.100 /  2.  5.100
   libswresample   0. 17.103 /  0. 17.103
 [http @ 0x7ff25a040600] HTTP error 403 Forbidden
 Unable to open key file
 http://www.youtube.com/api/drm/hlskey?id=add72f9f85beeb13&itag=96&source=youtube&ratebypass=yes&pfa=1&pbr=yes&hls_chunk_host=www.youtube.com&gir=yes&clen=5547517760&lmt=1347749241948586&dur=8870.400&pmbypass=yes&playlist_type=DVR&cp=U0hWTldTVl9NU0NONl9PTVpGOl9QaC1zZ1NuSnEx&maudio=1&sver=3&fexp=903802,927703,932204,928023,924606,929117,929121,929906,929907,929922,929923,929127,929129,929131,929930,936403,925724,925726,936310,925720,925722,925718,925714,929917,906945,929933,929935,939604,906842,927704,913428,919811,932309,913563,919373,930803,908536,938701,931924,934005,940501,936308,909549,901608,900816,912711,934507,912714,907231,936312,906001&upn=5htNr8XQdyc&ip={my_ip}&ipbits=8&expire=1379788589&sparams=ip,ipbits,expire,id,itag,source,ratebypass,pfa,pbr,hls_chunk_host,gir,clen,lmt,dur,pmbypass,playlist_type,cp,maudio&signature=6EC01EDEEB7D161229D81AD728A68C9C1154ECAD.4D9721C03E7CC061A5093D83355957C08AACA696&key=dg_yt0&ek=00d1d2078d07e911274516317c3229cce139c648113fc4167a
 [hls,applehttp @ 0x7ff25a03d800] Error when loading first segment
 'http://www.youtube.com/videoplayback/id/add72f9f85beeb13/itag/96/source/youtube/range/0-1606463/file/seg.ts?ratebypass=yes&pfa=1&pbr=yes&hls_chunk_host=www.youtube.com&gir=yes&clen=5547517760&lmt=1347749241948586&dur=8870.400&pmbypass=yes&playlist_type=DVR&cp=U0hWTldTVl9NU0NONl9PTVpGOl9QaC1zZ1NuSnEx&maudio=1&sver=3&fexp=903802,927703,932204,928023,924606,929117,929121,929906,929907,929922,929923,929127,929129,929131,929930,936403,925724,925726,936310,925720,925722,925718,925714,929917,906945,929933,929935,939604,906842,927704,913428,919811,932309,913563,919373,930803,908536,938701,931924,934005,940501,936308,909549,901608,900816,912711,934507,912714,907231,936312,906001&upn=5htNr8XQdyc&ip={my_ip}&ipbits=8&expire=1379788589&sparams=ip,ipbits,expire,id,itag,source,ratebypass,pfa,pbr,hls_chunk_host,gir,clen,lmt,dur,pmbypass,playlist_type,cp,maudio&signature=6EC01EDEEB7D161229D81AD728A68C9C1154ECAD.4D9721C03E7CC061A5093D83355957C08AACA696&key=dg_yt0'
 http://www.youtube.com/api/manifest/hls_playlist/id/add72f9f85beeb13/itag/96/source/youtube/ratebypass/yes/pfa/1/pbr/yes/hls_chunk_host/www.youtube.com/gir/yes/clen/5547517760/lmt/1347749241948586/dur/8870.400/pmbypass/yes/playlist_type/DVR/cp/U0hWTldTVl9NU0NONl9PTVpGOl9QaC1zZ1NuSnEx/maudio/1/sver/3/fexp/903802,927703,932204,928023,924606,929117,929121,929906,929907,929922,929923,929127,929129,929131,929930,936403,925724,925726,936310,925720,925722,925718,925714,929917,906945,929933,929935,939604,906842,927704,913428,919811,932309,913563,919373,930803,908536,938701,931924,934005,940501,936308,909549,901608,900816,912711,934507,912714,907231,936312,906001/upn/5htNr8XQdyc/ip/{my_ip}/ipbits/8/expire/1379788589/sparams/ip,ipbits,expire,id,itag,source,ratebypass,pfa,pbr,hls_chunk_host,gir,clen,lmt,dur,pmbypass,playlist_type,cp,maudio/signature/6EC01EDEEB7D161229D81AD728A68C9C1154ECAD.4D9721C03E7CC061A5093D83355957C08AACA696/key/dg_yt0/file/index.m3u8:
 Invalid data found when processing input
 }}}

 If I manually download the manifest and search for the key, it's longer:
 {{{
 http://www.youtube.com/api/drm/hlskey?id=add72f9f85beeb13&itag=96&source=youtube&ratebypass=yes&pfa=1&pbr=yes&hls_chunk_host=www.youtube.com&gir=yes&clen=5547517760&lmt=1347749241948586&dur=8870.400&pmbypass=yes&playlist_type=DVR&cp=U0hWTldTVl9NU0NONl9PTVpGOl9QaC1zZ1NuSnEx&maudio=1&sver=3&fexp=903802,927703,932204,928023,924606,929117,929121,929906,929907,929922,929923,929127,929129,929131,929930,936403,925724,925726,936310,925720,925722,925718,925714,929917,906945,929933,929935,939604,906842,927704,913428,919811,932309,913563,919373,930803,908536,938701,931924,934005,940501,936308,909549,901608,900816,912711,934507,912714,907231,936312,906001&upn=5htNr8XQdyc&ip={my_ip}&ipbits=8&expire=1379788589&sparams=ip,ipbits,expire,id,itag,source,ratebypass,pfa,pbr,hls_chunk_host,gir,clen,lmt,dur,pmbypass,playlist_type,cp,maudio&signature=6EC01EDEEB7D161229D81AD728A68C9C1154ECAD.4D9721C03E7CC061A5093D83355957C08AACA696&key=dg_yt0&ek=00d1d2078d07e911274516317c3229cce139c648113fc4167a59e8a64d87b0a8f11877de7e0197d2f1a49d1639eed28a5d1785639ac94664b328641b824f55413a88c43da5b88525b0
 }}}

 After looking into libavformat/hls.c I think I've found the problem, in
 the line 215
 {{{#!c
     char line[1024];
 }}}

 This causes lines to be trunctated, and therefore the key url is wrong. If
 I change the value to something higher, like 1500, the key url is
 correctly extracted and everythin works.

-- 
Ticket URL: <https://ffmpeg.org/trac/ffmpeg/ticket/2976>
FFmpeg <http://ffmpeg.org>
FFmpeg issue tracker