[FFmpeg-trac] #4644(avdevice:new): Corrupt H264 when streaming H264 from v4l2 device
FFmpeg
trac at avcodec.org
Sat Jun 20 01:44:13 CEST 2015
#4644: Corrupt H264 when streaming H264 from v4l2 device
-----------------------------------+--------------------------------------
Reporter: noah | Type: defect
Status: new | Priority: normal
Component: avdevice | Version: git-master
Keywords: v4l2 h264 | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-----------------------------------+--------------------------------------
I would like to stream H264 without transcoding on a Raspberry Pi with a
Raspberry Pi Camera module. Currently it seems FFmpeg gets confused
somewhere after data has been captured from the V4L2 device; the H264
demuxer(?) starts spitting out a bunch of errors which seem to indicate it
is being fed junk. Depending on the ffmpeg version, I either get
corrupted output or no output at all.
I'm seeing the problem in git master and also verified it to be present in
tag n1.2.12 and n2.6.1 in git.
The problem seems to be very similar to what was reported in 2013, and
subsequently fixed, here:
https://trac.ffmpeg.org/ticket/2882 (ffmpeg can't save h264 stream from
v4l2 webcam into file without recoding)
Changing AVSTREAM_PARSE_HEADERS to AVSTREAM_PARSE_FULL in
libavdevice/v4l2.c:v4l2_read_header() fixes the problem for me, though I'm
not sure it's the correct solution.
Steps to reproduce:
1. Get a Raspberry Pi with an attached Raspberry Pi Camera module, load
bcm2835_v4l kernel module
2. Build ffmpeg from source
3. Run: ./ffmpeg -report -f v4l2 -input_format h264 -t 2 -i /dev/video0
-vcodec copy -loglevel debug -y foo.mkv
{{{
$ ./ffmpeg -report -f v4l2 -input_format h264 -t 2 -i /dev/video0 -vcodec
copy -loglevel debug -y foo.mkv
...
Opening an input file: /dev/video0.
[video4linux2,v4l2 @ 0x1aec9e0] fd:4 capabilities:85200005
[video4linux2,v4l2 @ 0x1aec9e0] Current input_channel: 0, input_name:
Camera 0, input_std: 0
[video4linux2,v4l2 @ 0x1aec9e0] Querying the device for the current frame
size
[video4linux2,v4l2 @ 0x1aec9e0] Setting frame size to 1024x768
[h264 @ 0x1aed230] missing picture in access unit with size 27
[h264 @ 0x1aed230] no frame!
[h264 @ 0x1aed230] missing picture in access unit with size 27
[video4linux2,v4l2 @ 0x1aec9e0] Non-increasing DTS in stream 0: packet 11
with DTS 7463370264, packet 12 with DTS 7462329653
[h264 @ 0x1aed230] missing picture in access unit with size 27
[video4linux2,v4l2 @ 0x1aec9e0] Non-increasing DTS in stream 0: packet 22
with DTS 7463770172, packet 23 with DTS 7462729442
[video4linux2,v4l2 @ 0x1aec9e0] Non-increasing DTS in stream 0: packet 26
with DTS 7462849378, packet 27 with DTS 0
[video4linux2,v4l2 @ 0x1aec9e0] Non-increasing DTS in stream 0: packet 27
with DTS 0, packet 28 with DTS 0
[video4linux2,v4l2 @ 0x1aec9e0] Non-increasing DTS in stream 0: packet 28
with DTS 0, packet 29 with DTS 0
[video4linux2,v4l2 @ 0x1aec9e0] Non-increasing DTS in stream 0: packet 29
with DTS 0, packet 30 with DTS 0
[video4linux2,v4l2 @ 0x1aec9e0] Non-increasing DTS in stream 0: packet 30
with DTS 0, packet 31 with DTS 0
[video4linux2,v4l2 @ 0x1aec9e0] Non-increasing DTS in stream 0: packet 31
with DTS 0, packet 32 with DTS 0
[h264 @ 0x1aed230] missing picture in access unit with size 27
... more messages similar to the three above ...
Input #0, video4linux2,v4l2, from '/dev/video0':
Duration: N/A, start: -140455148.518520, bitrate: N/A
Stream #0:0, 125, 1/1000000: Video: h264 (High), 1 reference frame,
yuv420p(left), 1024x768, 1/2000000, -4 kb/s, 25 fps, 25 tbr, 10
00k tbn, 2000k tbc
Successfully opened the file.
Parsing a group of options: output file foo.mkv.
Applying option vcodec (force video codec ('copy' to copy stream)) with
argument copy.
Successfully parsed a group of options.
Opening an output file: foo.mkv.
Successfully opened the file.
[matroska @ 0x1b68230] Codec for stream 0 does not use global headers but
container format requires global headers
Output #0, matroska, to 'foo.mkv':
Metadata:
encoder : Lavf56.37.100
Stream #0:0, 0, 1/1000: Video: h264, 1 reference frame (H264 /
0x34363248), yuv420p(left), 1024x768 (0x0), 1/1000000, q=2-31, -4 kb
/s, 25 fps, 25 tbr, 1k tbn, 1000k tbc
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
}}}
A full log is attached as ffmpeg-v4l2-h264-copy-fails-20150620-010456.log.
Ffprobe gives similar errors:
{{{
$ ./ffprobe -report -f v4l2 -input_format h264 /dev/video0
...
[h264 @ 0x2fcee10] missing picture in access unit with size 27
[h264 @ 0x2fcee10] no frame!
[h264 @ 0x2fcee10] missing picture in access unit with size 27
Last message repeated 10 times
Input #0, video4linux2,v4l2, from '/dev/video0':
Duration: N/A, start: -140455000.691313, bitrate: N/A
Stream #0:0: Video: h264 (High), yuv420p, 1024x768, -4 kb/s, 25 fps,
25 tbr, 1000k tbn, 2000k tbc
$ echo $?
130
}}}
A full log is attached as ffprobe-v4l2-h264-20150620-005024.log.
What does work, however, is to let ffmpeg stream raw video from the camera
(e.g. remove '-input_format h264'). Example:
{{{
Opening an input file: /dev/video0.
[video4linux2,v4l2 @ 0x1dbc9e0] fd:4 capabilities:85200005
[video4linux2,v4l2 @ 0x1dbc9e0] Current input_channel: 0, input_name:
Camera 0, input_std: 0
[video4linux2,v4l2 @ 0x1dbc9e0] Querying the device for the current frame
size
[video4linux2,v4l2 @ 0x1dbc9e0] Setting frame size to 1024x768
[video4linux2,v4l2 @ 0x1dbc9e0] All info found
Input #0, video4linux2,v4l2, from '/dev/video0':
Duration: N/A, start: 9375.983690, bitrate: 235929 kb/s
Stream #0:0, 1, 1/1000000: Video: rawvideo, 1 reference frame (I420 /
0x30323449), yuv420p, 1024x768, 1/1000000, 235929 kb/s, 25 fps, 25 tbr,
1000k tbn, 1000k tbc
Successfully opened the file.
Parsing a group of options: output file out.mkv.
Applying option vcodec (force video codec ('copy' to copy stream)) with
argument copy.
Successfully parsed a group of options.
Opening an output file: out.mkv.
Successfully opened the file.
[matroska @ 0x1dbdc90] Codec for stream 0 does not use global headers but
container format requires global headers
Output #0, matroska, to 'out.mkv':
Metadata:
encoder : Lavf56.37.100
Stream #0:0, 0, 1/1000: Video: rawvideo, 1 reference frame (I420 /
0x30323449), yuv420p, 1024x768 (0x0), 1/1000000, q=2-31, 235929 kb/s, 25
fps, 25 tbr, 1k tbn, 1000k tbc
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
[matroska @ 0x1dbdc90] Writing block at offset 536, size 1179648, pts 0,
dts 0, duration 40, keyframe 1
[matroska @ 0x1dbdc90] Starting new cluster at offset 1180192 bytes, pts
182dts 182
[matroska @ 0x1dbdc90] Writing block at offset 1180207, size 1179648, pts
182, dts 182, duration 40, keyframe 1
[matroska @ 0x1dbdc90] Starting new cluster at offset 2359863 bytes, pts
365dts 365
[matroska @ 0x1dbdc90] Writing block at offset 2359879, size 1179648, pts
365, dts 365, duration 40, keyframe 1
[matroska @ 0x1dbdc90] Starting new cluster at offset 3539535 bytes, pts
547dts 547
[matroska @ 0x1dbdc90] Writing block at offset 3539551, size 1179648, pts
547, dts 547, duration 40, keyframe 1
frame= 4 fps=0.0 q=-1.0 size= 4609kB time=00:00:00.54
bitrate=69019.5kbits/s
[matroska @ 0x1dbdc90] Starting new cluster at offset 4719207 bytes, pts
729dts 729
[matroska @ 0x1dbdc90] Writing block at offset 4719223, size 1179648, pts
729, dts 729, duration 40, keyframe 1
[matroska @ 0x1dbdc90] Starting new cluster at offset 5898879 bytes, pts
912dts 912
[matroska @ 0x1dbdc90] Writing block at offset 5898895, size 1179648, pts
912, dts 912, duration 40, keyframe 1
...
[matroska @ 0x1dbdc90] Writing block at offset 25953319, size 1179648, pts
4010, dts 4010, duration 40, keyframe 1
[matroska @ 0x1dbdc90] end duration = 4050
frame= 23 fps=5.0 q=-1.0 Lsize= 26497kB time=00:00:04.01
bitrate=54131.5kbits/s
video:26496kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
muxing overhead: 0.005621%
Input file #0 (/dev/video0):
Input stream #0:0 (video): 23 packets read (27131904 bytes);
Total: 23 packets (27131904 bytes) demuxed
Output file #0 (out.mkv):
Output stream #0:0 (video): 23 packets muxed (27131904 bytes);
Total: 23 packets (27131904 bytes) muxed
0 frames successfully decoded, 0 decoding errors
[AVIOContext @ 0x1dc74c0] Statistics: 156 seeks, 957 writeouts
}}}
A full log is attached as ffmpeg-v4l2-raw-streaming-
works-20150620-013650.log.
Since ticket #2882 was marked as fixed, v4l2_read_header() in
libavdevice/v4l2.c contains:
st->need_parsing = AVSTREAM_PARSE_HEADERS;
If I change this to AVSTREAM_PARSE_FULL, as was suggested in an initial
patch in attached to the ticket, ffmpeg appears to do better and produces
playable output data. However, while it no longer complains about H264, it
floods the screen with warnings about 'Non-monotonous DTS in output
stream'.
{{{
...
Opening an input file: /dev/video0.
[video4linux2,v4l2 @ 0x13cd9e0] fd:4 capabilities:85200005
[video4linux2,v4l2 @ 0x13cd9e0] Current input_channel: 0, input_name:
Camera 0, input_std: 0
[video4linux2,v4l2 @ 0x13cd9e0] Querying the device for the current frame
size
[video4linux2,v4l2 @ 0x13cd9e0] Setting frame size to 1024x768
[video4linux2,v4l2 @ 0x13cd9e0] v4l2_read_header: turning on
AVSTREAM_PARSE_FULL
[video4linux2,v4l2 @ 0x13cd9e0] All info found
Input #0, video4linux2,v4l2, from '/dev/video0':
Duration: N/A, start: 8047.143224, bitrate: N/A
Stream #0:0, 8, 1/1000000: Video: h264 (High), 1 reference frame,
yuv420p(left), 1024x768, 1/2000000, -4 kb/s, 25 fps, 25 tbr, 1000k tbn,
2000k tbc
Successfully opened the file.
Parsing a group of options: output file foo.mkv.
Applying option vcodec (force video codec ('copy' to copy stream)) with
argument copy.
Successfully parsed a group of options.
Opening an output file: foo.mkv.
Successfully opened the file.
[matroska @ 0x144acd0] Codec for stream 0 does not use global headers but
container format requires global headers
Output #0, matroska, to 'foo.mkv':
Metadata:
encoder : Lavf56.37.100
Stream #0:0, 0, 1/1000: Video: h264, 1 reference frame (H264 /
0x34363248), yuv420p(left), 1024x768 (0x0), 1/1000000, q=2-31, -4 kb/s, 25
fps, 25 tbr, 1k tbn, 1000k tbc
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
[matroska @ 0x144acd0] Writing block at offset 562, size 1898, pts 0, dts
0, duration 0, keyframe 1
[matroska @ 0x144acd0] Writing block at offset 2467, size 2579, pts 0, dts
0, duration 0, keyframe 0
[matroska @ 0x144acd0] Writing block at offset 5053, size 2148, pts 40,
dts 40, duration 0, keyframe 0
[matroska @ 0x144acd0] Writing block at offset 7208, size 3204, pts 80,
dts 80, duration 0, keyframe 0
[matroska @ 0x144acd0] Writing block at offset 10419, size 2867, pts 120,
dts 120, duration 0, keyframe 0
[matroska @ 0x144acd0] Writing block at offset 13293, size 2737, pts 160,
dts 160, duration 0, keyframe 0
[matroska @ 0x144acd0] Writing block at offset 16037, size 3747, pts 200,
dts 200, duration 0, keyframe 0
[matroska @ 0x144acd0] Writing block at offset 19791, size 5135, pts 240,
dts 240, duration 0, keyframe 0
[matroska @ 0x144acd0] Writing block at offset 24933, size 7273, pts 280,
dts 280, duration 0, keyframe 0
[matroska @ 0x144acd0] Writing block at offset 32213, size 12305, pts 320,
dts 320, duration 0, keyframe 0
[matroska @ 0x144acd0] Starting new cluster at offset 44525 bytes, pts
1401dts 1401
[matroska @ 0x144acd0] Writing block at offset 44541, size 9172, pts 1401,
dts 1401, duration 0, keyframe 1
[matroska @ 0x144acd0] Non-monotonous DTS in output stream 0:0; previous:
1401, current: 400; changing to 1401. This may result in incorrect
timestamps in the output file.
[matroska @ 0x144acd0] Writing block at offset 53720, size 7825, pts 1401,
dts 1401, duration 0, keyframe 0
[matroska @ 0x144acd0] Non-monotonous DTS in output stream 0:0; previous:
1401, current: 440; changing to 1401. This may result in incorrect
timestamps in the output file.
...
[matroska @ 0x144acd0] end duration = 4998
frame= 102 fps= 28 q=-1.0 Lsize= 1657kB time=00:00:04.99
bitrate=2716.7kbits/s
video:1656kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
muxing overhead: 0.101796%
Input file #0 (/dev/video0):
Input stream #0:0 (video): 102 packets read (1695552 bytes);
Total: 102 packets (1695552 bytes) demuxed
Output file #0 (foo.mkv):
Output stream #0:0 (video): 102 packets muxed (1695552 bytes);
Total: 102 packets (1695552 bytes) muxed
0 frames successfully decoded, 0 decoding errors
}}}
A full log is attached as ffmpeg-v4l2-h264-copy-AVSTREAM_PARSE_FULL-
ok-20150620-011441.log.
FWIW, I see similar problem if I use .ts instead of .mkv for the output
file.
My environment is Raspberry Pi (all models) with attached a Raspberry Pi
Camera module. Running Raspbian 2015-05-05 (based on Debian 7.8) with the
bcm2835-v4l2 kernel module providing a V4L2 interface to the camera.
{{{
$ uname -a
Linux raspberrypi 3.18.11-v7+ #781 SMP PREEMPT Tue Apr 21 18:07:59 BST
2015 armv7l GNU/Linux
$ lsmod |egrep 'v4l2|video'
bcm2835_v4l2 37225 0
videobuf2_vmalloc 3009 1 bcm2835_v4l2
videobuf2_memops 1552 1 videobuf2_vmalloc
videobuf2_core 33578 1 bcm2835_v4l2
v4l2_common 5077 2 bcm2835_v4l2,videobuf2_core
videodev 122487 3 bcm2835_v4l2,v4l2_common,videobuf2_core
media 11577 1 videodev
}}}
Output from v4l2-ctl with --get-fmt-video, --list-formats-ext and --list-
ctrls-menus:
{{{
$ v4l2-ctl --get-fmt-video
Format Video Capture:
Width/Height : 1024/768
Pixel Format : 'H264'
Field : None
Bytes per Line: 1024
Size Image : 786432
Colorspace : Broadcast NTSC/PAL (SMPTE170M/ITU601)
Custom Info : feedcafe
$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: 'YU12'
Name : 4:2:0, packed YUV
Size: Stepwise 16x16 - 2592x1944 with step 2/2
Index : 1
Type : Video Capture
Pixel Format: 'YUYV'
Name : 4:2:2, packed, YUYV
Size: Stepwise 16x16 - 2592x1944 with step 2/2
Index : 2
Type : Video Capture
Pixel Format: 'RGB3'
Name : RGB24 (LE)
Size: Stepwise 16x16 - 2592x1944 with step 2/2
Index : 3
Type : Video Capture
Pixel Format: 'JPEG' (compressed)
Name : JPEG
Size: Stepwise 16x16 - 2592x1944 with step 2/2
Index : 4
Type : Video Capture
Pixel Format: 'H264' (compressed)
Name : H264
Size: Stepwise 16x16 - 2592x1944 with step 2/2
Index : 5
Type : Video Capture
Pixel Format: 'MJPG' (compressed)
Name : MJPEG
Size: Stepwise 16x16 - 2592x1944 with step 2/2
Index : 6
Type : Video Capture
Pixel Format: 'YVYU'
Name : 4:2:2, packed, YVYU
Size: Stepwise 16x16 - 2592x1944 with step 2/2
Index : 7
Type : Video Capture
Pixel Format: 'VYUY'
Name : 4:2:2, packed, VYUY
Size: Stepwise 16x16 - 2592x1944 with step 2/2
Index : 8
Type : Video Capture
Pixel Format: 'UYVY'
Name : 4:2:2, packed, UYVY
Size: Stepwise 16x16 - 2592x1944 with step 2/2
Index : 9
Type : Video Capture
Pixel Format: 'NV12'
Name : 4:2:0, packed, NV12
Size: Stepwise 16x16 - 2592x1944 with step 2/2
Index : 10
Type : Video Capture
Pixel Format: 'BGR3'
Name : RGB24 (BE)
Size: Stepwise 16x16 - 2592x1944 with step 2/2
Index : 11
Type : Video Capture
Pixel Format: 'YV12'
Name : 4:2:0, packed YVU
Size: Stepwise 16x16 - 2592x1944 with step 2/2
Index : 12
Type : Video Capture
Pixel Format: 'NV21'
Name : 4:2:0, packed, NV21
Size: Stepwise 16x16 - 2592x1944 with step 2/2
Index : 13
Type : Video Capture
Pixel Format: 'BGR4'
Name : RGB32 (BE)
Size: Stepwise 16x16 - 2592x1944 with step 2/2
$ v4l2-ctl --list-ctrls-menus
User Controls
brightness (int) : min=0 max=100 step=1 default=50
value=50 flags=slider
contrast (int) : min=-100 max=100 step=1
default=0 value=0 flags=slider
saturation (int) : min=-100 max=100 step=1
default=0 value=0 flags=slider
red_balance (int) : min=1 max=7999 step=1
default=1000 value=1000 flags=slider
blue_balance (int) : min=1 max=7999 step=1
default=1000 value=1000 flags=slider
horizontal_flip (bool) : default=0 value=1
vertical_flip (bool) : default=0 value=1
power_line_frequency (menu) : min=0 max=3 default=1 value=1
0: Disabled
1: 50 Hz
2: 60 Hz
3: Auto
sharpness (int) : min=-100 max=100 step=1
default=0 value=0 flags=slider
color_effects (menu) : min=0 max=15 default=0 value=0
0: None
1: Black & White
2: Sepia
3: Negative
4: Emboss
5: Sketch
6: Sky Blue
7: Grass Green
8: Skin Whiten
9: Vivid
10: Aqua
11: Art Freeze
12: Silhouette
13: Solarization
14: Antique
15: Set Cb/Cr
rotate (int) : min=0 max=360 step=90 default=0
value=0
color_effects_cbcr (int) : min=0 max=65535 step=1
default=32896 value=32896
Codec Controls
video_bitrate_mode (menu) : min=0 max=1 default=0 value=0
flags=update
0: Variable Bitrate
1: Constant Bitrate
video_bitrate (int) : min=25000 max=25000000
step=25000 default=10000000 value=6000000
repeat_sequence_header (bool) : default=0 value=1
h264_i_frame_period (int) : min=0 max=2147483647 step=1
default=60 value=10
h264_level (menu) : min=0 max=11 default=11
value=11
0: 1
1: 1b
2: 1.1
3: 1.2
4: 1.3
5: 2
6: 2.1
7: 2.2
8: 3
9: 3.1
10: 3.2
11: 4
h264_profile (menu) : min=0 max=4 default=4 value=4
0: Baseline
1: Constrained Baseline
2: Main
4: High
Camera Controls
auto_exposure (menu) : min=0 max=3 default=0 value=0
0: Auto Mode
1: Manual Mode
exposure_time_absolute (int) : min=1 max=10000 step=1
default=1000 value=1000
exposure_dynamic_framerate (bool) : default=0 value=0
auto_exposure_bias (intmenu): min=0 max=24 default=12
value=12
0: -4000 (0xfffffffffffff060)
1: -3667 (0xfffffffffffff1ad)
2: -3333 (0xfffffffffffff2fb)
3: -3000 (0xfffffffffffff448)
4: -2667 (0xfffffffffffff595)
5: -2333 (0xfffffffffffff6e3)
6: -2000 (0xfffffffffffff830)
7: -1667 (0xfffffffffffff97d)
8: -1333 (0xfffffffffffffacb)
9: -1000 (0xfffffffffffffc18)
10: -667 (0xfffffffffffffd65)
11: -333 (0xfffffffffffffeb3)
12: 0 (0x0)
13: 333 (0x14d)
14: 667 (0x29b)
15: 1000 (0x3e8)
16: 1333 (0x535)
17: 1667 (0x683)
18: 2000 (0x7d0)
19: 2333 (0x91d)
20: 2667 (0xa6b)
21: 3000 (0xbb8)
22: 3333 (0xd05)
23: 3667 (0xe53)
24: 4000 (0xfa0)
white_balance_auto_preset (menu) : min=0 max=9 default=1 value=1
0: Manual
1: Auto
2: Incandescent
3: Fluorescent
4: Fluorescent H
5: Horizon
6: Daylight
7: Flash
8: Cloudy
9: Shade
image_stabilization (bool) : default=0 value=0
iso_sensitivity (intmenu): min=0 max=4 default=0 value=0
0: 0 (0x0)
1: 100 (0x64)
2: 200 (0xc8)
3: 400 (0x190)
4: 800 (0x320)
exposure_metering_mode (menu) : min=0 max=2 default=0 value=0
0: Average
1: Center Weighted
2: Spot
scene_mode (menu) : min=0 max=13 default=0 value=0
0: None
8: Night
11: Sports
JPEG Compression Controls
compression_quality (int) : min=1 max=100 step=1 default=30
value=30
}}}
--
Ticket URL: <https://trac.ffmpeg.org/ticket/4644>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
More information about the FFmpeg-trac
mailing list