[FFmpeg-trac] #7282(undetermined:new): ffmpeg generates improper HE-AACv2 fMP4 files

FFmpeg trac at avcodec.org
Wed Jun 27 16:56:54 EEST 2018


#7282: ffmpeg generates improper HE-AACv2 fMP4 files
-------------------------------------+-------------------------------------
             Reporter:  ronak2121    |                    Owner:
                 Type:  defect       |                   Status:  new
             Priority:  normal       |                Component:
              Version:  git-master   |  undetermined
             Keywords:               |               Resolution:
             Blocking:               |               Blocked By:
Analyzed by developer:  0            |  Reproduced by developer:  0
-------------------------------------+-------------------------------------

Comment (by ronak2121):

 Here's what I did:

 First, I created an HE-AACv2 MP4 from a Wav file:

 ffmpeg -i short_44khz_16bit_2ch.wav -r:a 44100 -b:a 32k -codec libfdk_aac
 -profile:a aac_he_v2 test.aac

 with this console output:

 a45e60e46a79:input ronakp$ ffmpeg -i short_44khz_16bit_2ch.wav -r:a 44100
 -b:a 32k -codec libfdk_aac -profile:a aac_he_v2 test.aac
 ffmpeg version git-2018-06-23-b86c575 Copyright (c) 2000-2018 the FFmpeg
 developers
   built with Apple LLVM version 9.1.0 (clang-902.0.39.2)
   configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD-b86c575 --enable-
 shared --enable-pthreads --enable-version3 --enable-hardcoded-tables
 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-gpl
 --enable-libfdk-aac --enable-libmp3lame --enable-libopus --enable-
 librubberband --enable-libvorbis --enable-libvpx --enable-libx264
 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma
 --enable-nonfree
   libavutil      56. 18.102 / 56. 18.102
   libavcodec     58. 20.103 / 58. 20.103
   libavformat    58. 17.101 / 58. 17.101
   libavdevice    58.  4.101 / 58.  4.101
   libavfilter     7. 25.100 /  7. 25.100
   libavresample   4.  0.  0 /  4.  0.  0
   libswscale      5.  2.100 /  5.  2.100
   libswresample   3.  2.100 /  3.  2.100
   libpostproc    55.  2.100 / 55.  2.100
 Guessed Channel Layout for Input Stream #0.0 : stereo
 Input #0, wav, from 'short_44khz_16bit_2ch.wav':
   Duration: 00:09:44.61, bitrate: 1411 kb/s
     Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 44100 Hz,
 stereo, s16, 1411 kb/s
 File 'test.aac' already exists. Overwrite ? [y/N] y
 Stream mapping:
   Stream #0:0 -> #0:0 (pcm_s16le (native) -> aac (libfdk_aac))
 Press [q] to stop, [?] for help
 Output #0, adts, to 'test.aac':
   Metadata:
     encoder         : Lavf58.17.101
     Stream #0:0: Audio: aac (libfdk_aac) (HE-AACv2), 44100 Hz, stereo,
 s16, 32 kb/s
     Metadata:
       encoder         : Lavc58.20.103 libfdk_aac
 size=    2284kB time=00:09:44.65 bitrate=  32.0kbits/s speed=67.1x
 video:0kB audio:2284kB subtitle:0kB other streams:0kB global headers:0kB
 muxing overhead: 0.000000%

 Once I fragment this file with this command: ffmpeg -i "test.aac" -codec
 copy -bsf:a aac_adtstoasc -hls_time 9.75238095238095 -hls_segment_type
 fmp4 -hls_flags single_file+append_list+split_by_time -hls_playlist_type
 vod "test.m3u8"
 ffmpeg version git-2018-06-23-b86c575 Copyright (c) 2000-2018 the FFmpeg
 developers
   built with Apple LLVM version 9.1.0 (clang-902.0.39.2)
   configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD-b86c575 --enable-
 shared --enable-pthreads --enable-version3 --enable-hardcoded-tables
 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-gpl
 --enable-libfdk-aac --enable-libmp3lame --enable-libopus --enable-
 librubberband --enable-libvorbis --enable-libvpx --enable-libx264
 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma
 --enable-nonfree
   libavutil      56. 18.102 / 56. 18.102
   libavcodec     58. 20.103 / 58. 20.103
   libavformat    58. 17.101 / 58. 17.101
   libavdevice    58.  4.101 / 58.  4.101
   libavfilter     7. 25.100 /  7. 25.100
   libavresample   4.  0.  0 /  4.  0.  0
   libswscale      5.  2.100 /  5.  2.100
   libswresample   3.  2.100 /  3.  2.100
   libpostproc    55.  2.100 / 55.  2.100
 [aac @ 0x7ff9cd001200] Estimating duration from bitrate, this may be
 inaccurate
 Input #0, aac, from 'test.aac':
   Duration: 00:10:13.76, bitrate: 30 kb/s
     Stream #0:0: Audio: aac (HE-AACv2), 44100 Hz, stereo, fltp, 30 kb/s
 [hls @ 0x7ff9cd001e00] Opening 'test.m4s' for writing
 Output #0, hls, to 'test.m3u8':
   Metadata:
     encoder         : Lavf58.17.101
     Stream #0:0: Audio: aac (HE-AACv2), 44100 Hz, stereo, fltp, 30 kb/s
 Stream mapping:
   Stream #0:0 -> #0:0 (copy)
 Press [q] to stop, [?] for help
 [hls @ 0x7ff9cd001e00] Opening 'test.m3u8.tmp' for writing
     Last message repeated 58 times
 [hls @ 0x7ff9cd001e00] Opening 'test.m3u8.tmp' for writing
 size=N/A time=00:09:44.77 bitrate=N/A speed=6.29e+03x
 video:0kB audio:2198kB subtitle:0kB other streams:0kB global headers:0kB
 muxing overhead: unknown

 Open the M4S file, and look at the moof -> traf -> tfhd atom. The
 defaultSampleDuration is set to 2048.

 If you use Apple's mediafilesegmenter tool, the defaultSampleDuration is
 set to 1024 instead.

 mediafilesegmenter -a -s -r -B test_apple -i test_apple.m3u8 -t 10
 test.aac

 a45e60e46a79:input ronakp$ mediafilesegmenter -a -s -r -B test_apple -i
 test_apple.m3u8 -t 10 test.aac
 Jun 27 2018 09:38:41.908: ISO fragmented mode, forcing segments to start
 with I-Frame
 Jun 27 2018 09:38:41.908: Processing file
 /Users/ronakp/Downloads/input/test.aac
 Jun 27 2018 09:38:41.934: Wrote 0 to 621 to test_apple.mp4
 Jun 27 2018 09:38:41.937: Wrote 621 to 40406 to test_apple.mp4
 Jun 27 2018 09:38:41.937: segment bitrate 31.88 kbits/sec is new max
 Jun 27 2018 09:38:41.941: Wrote 40406 to 79905 to test_apple.mp4
 Jun 27 2018 09:38:41.945: Wrote 79905 to 119449 to test_apple.mp4
 Jun 27 2018 09:38:41.948: Wrote 119449 to 158987 to test_apple.mp4
 Jun 27 2018 09:38:41.952: Wrote 158987 to 198420 to test_apple.mp4
 Jun 27 2018 09:38:41.955: Wrote 198420 to 238025 to test_apple.mp4
 Jun 27 2018 09:38:41.959: Wrote 238025 to 277563 to test_apple.mp4
 Jun 27 2018 09:38:41.963: Wrote 277563 to 317093 to test_apple.mp4
 Jun 27 2018 09:38:41.966: Wrote 317093 to 356553 to test_apple.mp4
 Jun 27 2018 09:38:41.970: Wrote 356553 to 396035 to test_apple.mp4
 Jun 27 2018 09:38:41.974: Wrote 396035 to 435570 to test_apple.mp4
 Jun 27 2018 09:38:41.978: Wrote 435570 to 475084 to test_apple.mp4
 Jun 27 2018 09:38:41.982: Wrote 475084 to 514572 to test_apple.mp4
 Jun 27 2018 09:38:41.985: Wrote 514572 to 554179 to test_apple.mp4
 Jun 27 2018 09:38:41.988: Wrote 554179 to 593620 to test_apple.mp4
 Jun 27 2018 09:38:41.992: Wrote 593620 to 633215 to test_apple.mp4
 Jun 27 2018 09:38:41.994: Wrote 633215 to 672645 to test_apple.mp4
 Jun 27 2018 09:38:41.998: Wrote 672645 to 712211 to test_apple.mp4
 Jun 27 2018 09:38:42.002: Wrote 712211 to 751692 to test_apple.mp4
 Jun 27 2018 09:38:42.006: Wrote 751692 to 791271 to test_apple.mp4
 Jun 27 2018 09:38:42.008: Wrote 791271 to 830777 to test_apple.mp4
 Jun 27 2018 09:38:42.012: Wrote 830777 to 870272 to test_apple.mp4
 Jun 27 2018 09:38:42.015: Wrote 870272 to 909738 to test_apple.mp4
 Jun 27 2018 09:38:42.018: Wrote 909738 to 949296 to test_apple.mp4
 Jun 27 2018 09:38:42.022: Wrote 949296 to 988788 to test_apple.mp4
 Jun 27 2018 09:38:42.026: Wrote 988788 to 1028358 to test_apple.mp4
 Jun 27 2018 09:38:42.030: Wrote 1028358 to 1067856 to test_apple.mp4
 Jun 27 2018 09:38:42.034: Wrote 1067856 to 1107335 to test_apple.mp4
 Jun 27 2018 09:38:42.037: Wrote 1107335 to 1146795 to test_apple.mp4
 Jun 27 2018 09:38:42.041: Wrote 1146795 to 1186399 to test_apple.mp4
 Jun 27 2018 09:38:42.044: Wrote 1186399 to 1225815 to test_apple.mp4
 Jun 27 2018 09:38:42.047: Wrote 1225815 to 1265449 to test_apple.mp4
 Jun 27 2018 09:38:42.051: Wrote 1265449 to 1304925 to test_apple.mp4
 Jun 27 2018 09:38:42.054: Wrote 1304925 to 1344382 to test_apple.mp4
 Jun 27 2018 09:38:42.058: Wrote 1344382 to 1383935 to test_apple.mp4
 Jun 27 2018 09:38:42.062: Wrote 1383935 to 1423498 to test_apple.mp4
 Jun 27 2018 09:38:42.066: Wrote 1423498 to 1462984 to test_apple.mp4
 Jun 27 2018 09:38:42.069: Wrote 1462984 to 1502553 to test_apple.mp4
 Jun 27 2018 09:38:42.073: Wrote 1502553 to 1542116 to test_apple.mp4
 Jun 27 2018 09:38:42.076: Wrote 1542116 to 1581536 to test_apple.mp4
 Jun 27 2018 09:38:42.081: Wrote 1581536 to 1621122 to test_apple.mp4
 Jun 27 2018 09:38:42.084: Wrote 1621122 to 1660673 to test_apple.mp4
 Jun 27 2018 09:38:42.089: Wrote 1660673 to 1700069 to test_apple.mp4
 Jun 27 2018 09:38:42.093: Wrote 1700069 to 1739661 to test_apple.mp4
 Jun 27 2018 09:38:42.096: Wrote 1739661 to 1779163 to test_apple.mp4
 Jun 27 2018 09:38:42.100: Wrote 1779163 to 1818692 to test_apple.mp4
 Jun 27 2018 09:38:42.103: Wrote 1818692 to 1858169 to test_apple.mp4
 Jun 27 2018 09:38:42.106: Wrote 1858169 to 1897711 to test_apple.mp4
 Jun 27 2018 09:38:42.109: Wrote 1897711 to 1937200 to test_apple.mp4
 Jun 27 2018 09:38:42.113: Wrote 1937200 to 1976758 to test_apple.mp4
 Jun 27 2018 09:38:42.117: Wrote 1976758 to 2016276 to test_apple.mp4
 Jun 27 2018 09:38:42.120: Wrote 2016276 to 2055769 to test_apple.mp4
 Jun 27 2018 09:38:42.122: Wrote 2055769 to 2095311 to test_apple.mp4
 Jun 27 2018 09:38:42.124: Wrote 2095311 to 2134807 to test_apple.mp4
 Jun 27 2018 09:38:42.127: Wrote 2134807 to 2174290 to test_apple.mp4
 Jun 27 2018 09:38:42.129: Wrote 2174290 to 2213898 to test_apple.mp4
 Jun 27 2018 09:38:42.134: Wrote 2213898 to 2253400 to test_apple.mp4
 Jun 27 2018 09:38:42.138: Wrote 2253400 to 2292947 to test_apple.mp4
 Jun 27 2018 09:38:42.140: Wrote 2292947 to 2303705 to test_apple.mp4
 Jun 27 2018 09:38:42.140: Unknown video dynamic range value 0
 Jun 27 2018 09:38:42.140: average bit rate is 31.68 kbits/sec - max file
 bit rate is 31.88 kbits/sec

 Looking at this file, the defaultSampleDuration is set to 1024, which is
 correct at a 44100Hz sampling rate.

 Another issue is if you compare the codec outputs using Apple's tooling
 for both files:

 M3U8 created by FFMPEG
 mediastreamvalidator test.m3u8

 the output is:

 Track ID: 1
 Audio Codec: AAC-LC
 Audio sample rate: 22050 Hz
 Audio channels: 1
 Audio channel layout: Mono

 If you look at the file generated by Apple's tool:

 M3U8 created by Apple
 mediastreamvalidator test_apple.m3u8

 the output is:

 Track ID: 1
 Audio Codec: AAC-HEv2
 Audio sample rate: 44100 Hz
 Audio channels: 2
 Audio channel layout: Stereo (L R)

 Running ffprobe -i on the Apple Generated fragmented MP4:

 ffprobe -i test_apple.mp4
 ffprobe version git-2018-06-23-b86c575 Copyright (c) 2007-2018 the FFmpeg
 developers
   built with Apple LLVM version 9.1.0 (clang-902.0.39.2)
   configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD-b86c575 --enable-
 shared --enable-pthreads --enable-version3 --enable-hardcoded-tables
 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-gpl
 --enable-libfdk-aac --enable-libmp3lame --enable-libopus --enable-
 librubberband --enable-libvorbis --enable-libvpx --enable-libx264
 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma
 --enable-nonfree
   libavutil      56. 18.102 / 56. 18.102
   libavcodec     58. 20.103 / 58. 20.103
   libavformat    58. 17.101 / 58. 17.101
   libavdevice    58.  4.101 / 58.  4.101
   libavfilter     7. 25.100 /  7. 25.100
   libavresample   4.  0.  0 /  4.  0.  0
   libswscale      5.  2.100 /  5.  2.100
   libswresample   3.  2.100 /  3.  2.100
   libpostproc    55.  2.100 / 55.  2.100
 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test_apple.mp4':
   Metadata:
     major_brand     : mp42
     minor_version   : 1
     compatible_brands: mp41mp42isomhlsf
     creation_time   : 2018-06-27T13:38:41.000000Z
   Duration: 00:09:50.50, start: 9.904218, bitrate: 31 kb/s
     Stream #0:0(und): Audio: aac (HE-AACv2) (mp4a / 0x6134706D), 44100 Hz,
 stereo, fltp, 30 kb/s (default)
     Metadata:
       creation_time   : 2018-06-27T13:38:41.000000Z
       handler_name    : Core Media Audio

 So, in conclusion, here are the problems:

 1. It looks like ffmpeg's MP4 or fMP4 generation process ends up stripping
 the HE-AACv2 profile in a way that Apple's tooling does not recognize the
 profile anymore. This leads to players not always recognizing the profile
 properly and playing at degraded quality.
 2. The different defaultSampleDuration ends up causing issues when doing
 Adaptive Streaming between HE-AAC and LC-AAC audio streams in players like
 Exoplayer. I've filed an issue against Exoplayer and is how we discovered
 this problem: https://github.com/google/ExoPlayer/issues/3971

--
Ticket URL: <https://trac.ffmpeg.org/ticket/7282#comment:3>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker


More information about the FFmpeg-trac mailing list