kmsgrab: Oh man, this could be so great. But it's just... not.
I've been experimenting with the kmsgrab device in ffmpeg. For recording screens, and especially games, this could be huge for Linux nerds. And ditching xcbgrab would be the icing on the cake. But, as it stands now, it's meh. Here's why: 1.) There is a 2 year old bug that causes frames to be encoded out of order: https://trac.ffmpeg.org/ticket/8377. I've tested it myself (yes, with git head), on two separate AMD cards running on Debian with both the radeon and amdgpu drivers, and this bug is definitely there when recording alsa audio in the same process (the workaround is to capture streams separately with two ffmpeg processes, but that's a sucky way to do it). I've determined the issue has something to do with packet buffering, as changing the thread_queue_size to 1 for video appears to eliminate the problem, but then causes audio to be out of sync. So, not a solution, really. The problem happens whether or not you use vaapi for encoding or software encoding using libx264. So it happens well before the encoding stage. 2.) As far as I can tell, kmsgrab (like x11grab or other screen devices) wants a constant frame rate value (-r). That's not unusual, but the issue is this won't work well with FreeSync rendering. More and more gamers are using FreeSync (or Gsync) monitors that clock the refresh rate of the monitor to the rendering calls. The correct way to capture this would be to encode the output from kmsgrab using the system timer as the PTS or whatever, so you end up with a variable frame rate mp4 that has each and every frame discretely represented with the correct time value. Now, whether or not you want to conform the entire thing later to 59.94p for YouTube is another issue, but I think there needs to be an option (if there isn't already one) to just write each frame as it is presented to the file along with the system timestamp. This is exactly how camera input is recorded on mobile devices, as they're all variable frame rate, and it works fine. kmsgrab should do the same. So those are my gripes for now. Hopefully this can get fixed and we can all brag about our kill counts with dignity.
2.) As far as I can tell, kmsgrab (like x11grab or other screen devices) wants a constant frame rate value (-r). [...] This is exactly how camera input is recorded on mobile devices, as they're all variable frame rate, and it works fine. kmsgrab should do the same.
Wouldn't this work the same as camera devices that change frame rate with exposure rate? I'm using a Raspberry Pi camera that slows down the frame rate as it gets darker, and using -use_wallclock_as_timestamps seems to work for that. I also combine it with -vsync 1 to duplicate the frames so I get a constant FPS in the output video but I think -vf fps would probably be better. Maybe you need to mess with the setpts or setts filters to get accurate timing but I'm pretty sure it's all doable already. At least I can produce a fixed-framerate video that runs at the correct speed when taken from a variable-framerate V4L2 camera device, so it should be similar, assuming of course kmsgrab only provides a frame upon pageflipping or similar. If it's just sampling the screen contents at regular intervals then none of this will work and you'll probably get bad tearing in the recorded video, so hopefully it's not done that way. Cheers, Adam.
participants (2)
-
Adam Nielsen -
Thomas Worth