[Libav-user] Missing Frame rate in onMetaData and huge frames

[Adam Fenn]

I'm looking for someone with FFmpeg development expertise who can help solve two problems.  

My company has created an RMTP encoder/streamer application using FFmpeg.  The application muxes audio and video from multiple sources, and produces output that it then streams via RTMP, using FFmpeg.  The application runs on macOS, currently on Sierra (10.12.6).   

The problems are as follows:

Issue #1. Our FFmpeg based RTMP encoder produces VERY large frames after changing from a static image input. 

The problem evolves as follows:

a. The encoder encodes and streams a still image for 30 or more seconds.  During this time, output frames are of a normal size, and the output bitrate is at or under the specified target bit rate.
b. The content changes to a different, and moving picture..  At this time, the encoder’s output frames are VERY large, and the output bitrate is far above the specified target bit rate.
c. After approximately 30 seconds, the frames sizes return to normal, and the output bitrate of the motion video is roughly equal to the specified target bit rate.

There is a proportional relationship between the duration the static image is encoded in step (a) and the frame size and duration of impact.  

Our encoder settings are set as follows:

Video:
- Codec: H.264
- AVC Level: 4.1
- Data Rate: 1000kbps
- Resolution: 1280x720p
- Keyframe Interval: 2 seconds
- Frames per Second: 30
- Profile: High

Audio:
- Codec: AAC
- Sample Rate: 48,000
- Channels: 2 (Stereo)
- Data Rate 96kbps

FFmpeg command line does not exhibit the same problematic behavior.  FFmpeg produces consistent frame sizes and the output bitrate remains consistent for the duration of the encode/stream.  The following command line is used to in our testing:

ffmpeg -re -i TestPatternVideo.mp4 -b:v 1000k -vcodec h264_videotoolbox -ar 48000 -ab 96k -acodec aac -profile:v high -level:v 4.1 -f flv rtmp://hostname/appname/live/ffmpeg-test1

TestPatternVideo.mp4, which was used in our testing is available here: https://www.dropbox.com/sh/2g0oihshdg1z9g5/AAD7Aui_I7zWrCW6N-GrvLtGa?dl=0

We have also examined the recordings of both the FFmpeg output and of our encoder’s output with FFprobe.  We have converted the FFprobe output to CSV format, so it is easier to read.  Our encoder's recorded video output (MyEncoderOutput.mp4), and the FFprobe output (MyEncoderOutput.csv) are available here: https://www.dropbox.com/sh/2g0oihshdg1z9g5/AAD7Aui_I7zWrCW6N-GrvLtGa?dl=0

You will notice that our command line uses h264_videotoolbox rather than libx264.  Our encoder users h264_videotoolbox also.  We have also tested our encoder when using libx264 and it shows exactly the same problem.

We would like to resolve this issue with our encoder, so that it will not inflate the frame size, forcing the bit rate over the target bit rate.  Ideally our encoder should perform as well as FFmpeg command line does.

The most relevant source files (FFMpegStreamer.m and FFMpegStreamer.h) are available here: https://www.dropbox.com/sh/2g0oihshdg1z9g5/AAD7Aui_I7zWrCW6N-GrvLtGa?dl=0

Issue #2. The RTMP @setDataFrame’s onMetaData payload for our implementation differs from what FFmpeg produces from the command line. The primary concern is that the framerate is not specified.  We would like to make sure the framerate is included.  Secondarily, We would like to understand why our implementation specifies <duration> and <filesize>.  This is a live stream and those values seem invalid for a live stream.

Our implementation sends the following values in the onMetaData message:

<metadata>
<duration>0.0</duration>
<width>1280.0</width>
<height>720.0</height>
<videodatarate>1000.0</videodatarate>
<videocodecid>7.0</videocodecid>
<audiodatarate>96.0</audiodatarate>
<audiosamplerate>48000.0</audiosamplerate>
<audiosamplesize>16.0</audiosamplesize>
<stereo>true</stereo>
<audiocodecid>10.0</audiocodecid>
<encoder>Lavf57.66.102</encoder>
<filesize>0.0</filesize>
</metadata>

Our FFmpeg command line is set as follows, to match the values configured in my implementation as closely as possible. 

ffmpeg -re -i TestPatternVideo.mp4 -b:v 1000k -vcodec h264_videotoolbox -ar 48000 -ab 96k -acodec aac -profile:v high -level:v 4.1 -f flv rtmp://flocasts-os1.vorigin.scaleengine.net/flocasts-os/live/ffmpeg-test1

FFmpeg sends the following values in the onMetaData message.  

<metadata>
<width>1280.0</width>
<height>720.0</height>
<videodatarate>976.5625</videodatarate>
<framerate>30.0</framerate>
<videocodecid>7.0</videocodecid>
<audiodatarate>93.75</audiodatarate>
<audiosamplerate>48000.0</audiosamplerate>
<audiosamplesize>16.0</audiosamplesize>
<stereo>true</stereo>
<audiocodecid>10.0</audiocodecid>
<major_brand>qt</major_brand>
<minor_version>0</minor_version>
<compatible_brands>qt</compatible_brands>
<encoder>Lavf57.56.101</encoder>

We notice both the audio and video data rates are not reported exactly as they are specified in the command line.  We don’t believe this is a problem, but we’d like to understand why.  We’d also notice the Libavformat, as specified by <encoder>, is not exactly the same as in our implementation.  We would also like to know why this is.



Adam Fenn
adamfenn28 at gmail.com



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20170922/bd8f0a39/attachment.html>