Problem with combining a video and an audio stream from USB device
Dear all, I hope you could provide a hint to my problem: I have two USB devices attached to an RPi, both show up as usual as /dev/video0. Here's some additional info coming from two command line inputs: Device 1, video only (attached to an RPi4): "ffmpeg -f v4l2 -list_formats all -i /dev/video0" reports [video4linux2,v4l2 @ 0xe5e1c0] Compressed: mjpeg : Motion-JPEG : 1280x720 640x480 320x240 "v4l2-ctl --list-formats-ext" reports ioctl: VIDIOC_ENUM_FMT Type: Video Capture [0]: 'MJPG' (Motion-JPEG, compressed) Size: Discrete 1280x720 Interval: Stepwise 0.033s - 0.033s with step 0.000s (30.000-30.000 fps) Size: Discrete 640x480 Interval: Stepwise 0.033s - 0.033s with step 0.000s (30.000-30.000 fps) Size: Discrete 320x240 Interval: Stepwise 0.033s - 0.033s with step 0.000s (30.000-30.000 fps) Does work: ffmpeg -f v4l2 -i /dev/video0 -vcodec h264_omx -preset ultrafast -tune zerolatency -g 300 -b:v 1M -mpegts_service_type advanced_codec_digital_hdtv -f mpegts udp://OtherMachine:Port?pkt_size=1316 Device 2, video and audio (attached to an RPi3, but does not work either on the RPi4): "ffmpeg -f v4l2 -list_formats all -i /dev/video0" reports [video4linux2,v4l2 @ 0x2c41210] Compressed: mjpeg : Motion-JPEG : 1920x1080 1280x720 "v4l2-ctl --list-formats-ext" reports ioctl: VIDIOC_ENUM_FMT Index : 0 Type : Video Capture Pixel Format: 'MJPG' (compressed) Name : Motion-JPEG Size: Discrete 1920x1080 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.067s (15.000 fps) Size: Discrete 1280x720 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.067s (15.000 fps) After quite some tedious work and way too many hours I got this running: Video only: ffmpeg -f v4l2 -input_format mjpeg -i /dev/video0 -c:v copy -preset ultrafast -tune zerolatency -g 300 -f matroska udp://OtherMachine:Port?pkt_size=1316 Does not work at all: ffmpeg -f v4l2 -input_format mjpeg -i /dev/video0 -c:v copy -preset ultrafast -tune zerolatency -g 300 -f mpegts udp://OtherMachine:Port?pkt_size=1316, on "OtherMachine" I do see that there is an incoming data stream via VLC, but it could not be digested properly. Audio only: ffmpeg -f alsa -thread_queue_size 1024 -i plughw:1 -c:a mp2 -ac 2 -ar 44100 -preset ultrafast -tune zerolatency -b:a 128K -f mpegts udp://OtherMachine:Port?pkt_size=1316 But this does not work either: ffmpeg -f v4l2 -input_format mjpeg -i /dev/video0 -f alsa -thread_queue_size 1024 -i plughw:1 -c:v copy -c:a mp2 -ac 2 -ar 44100 -preset ultrafast -tune zerolatency -g 300 -b:a 128K -f mpegts udp://OtherMachine:Port?pkt_size=1316 Could you please provide a hint on how to get these two streams for device 2 working together? Both of them come from the same hardware/device, my guess is that the MJPG video stream is somehow not fully compliant with the mpegts standard (like it is for device 1) since it works with matroska, but not with mpegts. Could that be? What needs to be done in that case? Thanks in advance and kind regards FHoevi
Am 01.02.2020 um 17:18 schrieb FHoevi <fhoevi@fullbeamahead.com>:
since it works with matroska, but not with mpegts. Could that be? What needs to be done in that case?
You cannot put mjpeg into mpegts, this is not a limitation of FFmpeg. For future question: Please understand that we require all users to test current FFmpeg git head and provide the command tested together with the complete, uncut console output because it makes understanding issues like yours infinitely easier. Carl Eugen
Dear Carl Eugen, thanks for your immediate reply. I got Raspbian Buster freshly installed with ffmpeg included. Building all this from Git head code base would potentially be one of the next steps but only if anything else won't work. All streams below do show no errors at all, therefore I have decided to not disclose them, mea culpa. And, sorry, I'm certainly not a video expert like you obviously are and thus haven't been aware of the fact that mjpeg and mpegts may not work with each other by design. I simply would like to use what I have in the most efficient way and just tried to get potential aid by ffmpeg experts. Perhaps let me ask this question: Given that I have an incoming mjpeg video stream as well as an incoming mp2 audio stream which kind of container format (ok, it's obviously not mpegts) is the most appropriate one for combined streaming across my LAN or even into a pipe for further processing? Believe me, I tried my very best over a couple of hours to find out how to proceed but with no success. At least to my humble knowledge there is nothing such like a table providing answers to questions of that kind. Perhaps another hint, with the same kind of hardware setup I can do this cvlc -vvv v4l2:///dev/video0 --input-slave=alsa://plughw:1,0 --sout='#transcode{acodec=mpga,ab=128}:std{access=http,mux=asf,dst=:Port}' So, here my understanding is that video gets passed on unchanged (mjpeg) and audio gets transcoded via vlc's mpga which presumably corresponds to mp2 for ffmpeg. The container format is asf, but I was not able to get that running with ffmpeg for no obvious reason. Anyway, picking up this vlc broadcast stream via http://StreamingMachine:Port on any other machine in my network is working well. But how to achieve that with ffmpeg directly and potentially not as http:// but udp:// or pipe stream? Thx in advance FHoevi
participants (2)
-
Carl Eugen Hoyos -
FHoevi