[FFmpeg-devel] [PATCH] VFW capture support

Michael Niedermayer michaelni
Fri Mar 7 23:06:17 CET 2008


On Fri, Mar 07, 2008 at 02:52:18PM -0300, Ramiro Polla wrote:
> Hello Michael,
>
> I've rewritten vfwcap to use a different capture strategy. It's more 
> complex but much more accurate now.
>
> [...]
>>>> Doesnt vfw provide some timestamp for each frame?
>>> The timestamps returned are incredibly unreliable (if returned at all, 
>>> most times I just get 0x00000000).
>> Could you give me a hint of how they are returned, so i can check the docs
>> from M$ ?
>> Use of "get current time" is very problematic as frames will have very
>> inaccuarte timestamps.
>> Thus if that is the only kind of time you can get from VFW then i fear
>> you will need to clean them up somehow so that they are at least stable
>> to +- 1ms of the nominal frame rate of the video source.
>
> Using the grab_frame strategy no timestamps were returned. With this new 
> implementation the timestamps returned are accurate. There's no more need 
> to use the system time.
>
> [...]
>
>>>>>     if( s->flags & AVFMT_FLAG_NONBLOCK ) {
>>>>>         av_log( s, AV_LOG_ERROR, "Non blocking capture not yet 
>>>>> implemented.\n" );
>>>>>         return AVERROR_PATCHWELCOME;
>>>>>     }
>>>> where is the problem with that?
>>> I thought we didn't want FFmpeg to block unless the user specifies 
>>> AVFMT_FLAG_NONBLOCK. Isn't that the purpose of the flag?
>> Yes, what i was asking, is where the problem is with implementing support
>> for AVFMT_FLAG_NONBLOCK?
>
> The new implementation deals better with AVFMT_FLAG_NONBLOCK. It does 
> pretty much the same thing v4l2 does.
>
> [...]
>
>> Also can this patch do full framerate(30/25fps 50/60fields/sec) / full
>> resolution realtime capture and encoding in realtime with audio? If not
>> its definitly not well implemented.
>> Also does VFW buffer frames? If no then a seperate thread is pretty much
>> required for video capture. OTOH if it does buffer frames then the way
>> you generate timestamps is just wrong.
>
> VFW does buffer frames. The timestamps were indeed very inaccurate using 
> the system time.
>
> Mans suggested using a ring buffer, a semaphore and dropping frames instead 
> of using the mutexes and events and piling up every single frame.
>
> If it is preferred to drop frames, is there an existing option that could 
> be used to set the maximum amount of memory/time buffered?

Droping frames might be more user friendly. But simply droping when the buffer
is full is not optimal. Reason being that with a nearly full buffer and
some short drop in available resources (cronjob or whatever) several frames
could be droped in a row.
What should be done is that as the buffer becomes more full more frames
should be droped, so for example maybe with the buffer 0-50% full no
frame drops would happen, with 62% every 4th frame would be droped, with
75% every 2nd frame would be droped, 87% 3 out of 4 frames would be droped
...
The trick here is to keep the droped frames evenly distributed.
There are many ways this can be implemnted, the simplest i could think of is

const static uint8_t score[4]={62,87,75,100};
if(score[frame_num%4] < buffer_fullness)
    drop;

Another one would be to keep track of how many have been droped/not droped
in a row ...



[...]
> static enum PixelFormat vfw_pixfmt(DWORD biCompression)
> {
>     switch(biCompression) {
>     case MKTAG('Y', 'U', 'Y', '2'):
>         return PIX_FMT_YUYV422;

>     case BI_RGB:
>         return PIX_FMT_BGR24;

This isnt correct or is it? BI_RGB just says RGB but not which one or am i
remembering this wrong?


[...]
> static LRESULT CALLBACK videostream_cb(HWND hwnd, LPVIDEOHDR vdhdr)
> {
>     struct vfw_ctx *ctx;
>     AVPacketList *pktl, *pktl_next;
> 
>     ctx = (struct vfw_ctx *) GetWindowLongPtr(hwnd, GWLP_USERDATA);
> 
> #ifdef VFW_DEBUG
>     dump_videohdr(ctx->s, vdhdr);
> #endif
> 
>     WaitForSingleObject(ctx->mutex, INFINITE);
> 
>     pktl_next = av_mallocz(sizeof(AVPacketList));
>     if(!pktl_next)
>         return FALSE;
> 
>     if(av_new_packet(&pktl_next->pkt, vdhdr->dwBytesUsed) < 0) {
>         av_free(pktl_next);
>         return FALSE;
>     }

What will happen with the mutex if you just return here?


> 
>     pktl_next->pkt.pts = vdhdr->dwTimeCaptured;
>     memcpy(pktl_next->pkt.data, vdhdr->lpData, vdhdr->dwBytesUsed);
> 

>     for(pktl = ctx->pktl ; pktl && pktl->next ; pktl = pktl->next);
>     if(!pktl)
>         ctx->pktl = pktl_next;
>     else
>         pktl->next = pktl_next;

following is simpler:

for(plast_pktl= &ctx->pktl ; *plast_pktl ; plast_pktl= &(*plast_pktl)->next);
*plast_pktl= pktl_next


> 
>     SetEvent(ctx->event);
>     ReleaseMutex(ctx->mutex);
> 
>     return TRUE;
> }
> 
> static int vfw_read_header(AVFormatContext *s, AVFormatParameters *ap)
> {
>     struct vfw_ctx *ctx = s->priv_data;
>     AVCodecContext *codec;
>     AVStream *st;
>     int devnum;
>     int bisize;
>     BITMAPINFO *bi;
>     CAPTUREPARMS cparms;
>     DWORD biCompression;
>     int width;
>     int height;
>     int ret;
> 
>     if(!ap->time_base.den) {
>         av_log(s, AV_LOG_ERROR, "A time base must be specified.\n");
>         return AVERROR_IO;
>     }
> 
> #ifdef VFW_DEBUG
>     ctx->s = s;
> #endif
> 
>     ctx->hwnd = capCreateCaptureWindow(NULL, 0, 0, 0, 0, 0, HWND_MESSAGE, 0);
>     if(!ctx->hwnd) {
>         av_log(s, AV_LOG_ERROR, "Could not create capture window.\n");
>         return AVERROR_IO;
>     }
> 
>     /* If atoi fails, devnum==0 and the default device is used */
>     devnum = atoi(s->filename);
> 
>     ret = SendMessage(ctx->hwnd, WM_CAP_DRIVER_CONNECT, devnum, 0);
>     if(!ret) {
>         av_log(s, AV_LOG_ERROR, "Could not connect to device.\n");
>         return AVERROR(ENODEV);
>     }
> 
>     SendMessage(ctx->hwnd, WM_CAP_SET_OVERLAY, 0, 0);
>     SendMessage(ctx->hwnd, WM_CAP_SET_PREVIEW, 0, 0);
> 
>     ret = SendMessage(ctx->hwnd, WM_CAP_SET_CALLBACK_VIDEOSTREAM, 0,
>                       (LPARAM) videostream_cb);
>     if(!ret)
>         return AVERROR_IO;
> 
>     SetWindowLongPtr(ctx->hwnd, GWLP_USERDATA, (LONG_PTR) ctx);
> 
>     st = av_new_stream(s, 0);
>     if(!st)
>         return AVERROR_NOMEM;
> 
>     /* Set video format */
>     bisize = SendMessage(ctx->hwnd, WM_CAP_GET_VIDEOFORMAT, 0, 0);
>     if(!bisize)
>         return AVERROR_IO;
>     bi = av_malloc(bisize);
>     if(!bi)
>         return AVERROR_NOMEM;
>     ret = SendMessage(ctx->hwnd, WM_CAP_GET_VIDEOFORMAT, bisize, (LPARAM) bi);
>     if(!ret) {
>         av_free(bi);
>         return AVERROR_IO;
>     }
> 
>     dump_bih(s, &bi->bmiHeader);
> 
>     width  = ap->width  ? ap->width  : bi->bmiHeader.biWidth ;
>     height = ap->height ? ap->height : bi->bmiHeader.biHeight;
>     bi->bmiHeader.biWidth  = width ;
>     bi->bmiHeader.biHeight = height;
> 
>     ret = SendMessage(ctx->hwnd, WM_CAP_SET_VIDEOFORMAT, bisize, (LPARAM) bi);
>     if(!ret) {
>         av_log(s, AV_LOG_ERROR, "Could not set Video Format.\n");
>         av_free(bi);
>         return AVERROR_IO;
>     }
> 
>     biCompression = bi->bmiHeader.biCompression;
> 
>     av_free(bi);
> 
>     /* Set sequence setup */
>     ret = SendMessage(ctx->hwnd, WM_CAP_GET_SEQUENCE_SETUP, sizeof(cparms),
>                       (LPARAM) &cparms);
>     if(!ret)
>         return AVERROR_IO;
> 
>     dump_captureparms(s, &cparms);
> 
>     cparms.fYield = 1; // Spawn a background thread
>     cparms.dwRequestMicroSecPerFrame =
>                                (ap->time_base.num*1000000) / ap->time_base.den;
>     cparms.fAbortLeftMouse = 0;
>     cparms.fAbortRightMouse = 0;
>     cparms.fCaptureAudio = 0;
>     cparms.vKeyAbort = 0;
> 
>     ret = SendMessage(ctx->hwnd, WM_CAP_SET_SEQUENCE_SETUP, sizeof(cparms),
>                       (LPARAM) &cparms);
>     if(!ret)
>         return AVERROR_IO;
> 
>     codec = st->codec;
>     codec->time_base = ap->time_base;
>     codec->codec_type = CODEC_TYPE_VIDEO;
>     codec->width = width;
>     codec->height = height;
>     codec->codec_id = CODEC_ID_RAWVIDEO;
>     codec->pix_fmt = vfw_pixfmt(biCompression);
> 
>     av_set_pts_info(st, 32, 1, 1000);
> 
>     if(codec->pix_fmt == -1) {
>         av_log(s, AV_LOG_ERROR, "Unknown compression type."
>                          "Please report verbose (-v 99) debug information.\n");
>         return AVERROR_PATCHWELCOME;
>     }
> 
>     ctx->mutex = CreateMutex(NULL, 0, NULL);
>     ctx->event = CreateEvent(NULL, 1, 0, NULL);
> 
>     ret = SendMessage(ctx->hwnd, WM_CAP_SEQUENCE_NOFILE, 0, 0);
>     if(!ret)
>         return AVERROR_IO;

I suspect that quite a few of the returns cause various leaks, that is
the stuff from vfw_read_close() is missing.


[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

If you really think that XML is the answer, then you definitly missunderstood
the question -- Attila Kinali
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20080307/545c2963/attachment.pgp>



More information about the ffmpeg-devel mailing list