[FFmpeg-devel] [PATCH] avdevice/decklink_dec: Extract 1080i and NTSC VANC

Ray Tiley raytiley at gmail.com
Tue Jan 23 05:47:24 EET 2018


On Mon, Jan 22, 2018 at 10:20 PM Devin Heitmueller <
dheitmueller at ltnglobal.com> wrote:

> Hi Ray,
>
> Thanks for your patch.  A few questions:
>
> Could you confirm whether you are capturing via SDI or analog (i.e.
> composite/component)?  Also what is the capturing device and SDK version
> you are using?  I’ve found various bugs in the numbering of VANC lines in
> some cards, particularly with interlaced formats, and it would be good to
> understand if perhaps this is a card-specific issue.
>
> I'm capturing SDI, the only decklink card I have access to is the Decklink
mini recorder, and I was using the latest sdk ( I can check exact version
tomorrow during work.) It's definitely possible I expanded the search lines
a bit too far, I was just hacking until I found the data.


>
> > On Jan 20, 2018, at 12:33 PM, Ray Tiley <raytiley at gmail.com> wrote:
> >
> > This changes the vertical blanking lines extracted for NTSC and 1080i
> > resolutions that in personal testing were required to extract closed
> > caption data from the decklink video frames.
> >
> > Additionally NTSC resolutions have the vanc data interleved between the
> uyvy
> > and not just the luma as in high definition resolutions.
> >
> > In my testing this allows a decklink card encoding valid NTSC and 1080i
> > closed captions to pass the caption data to the x264 encoder.
> >
> > Signed-off-by: Ray Tiley <raytiley at gmail.com>
> > ---
> > libavdevice/decklink_dec.cpp | 37 ++++++++++++++++++++++++++++++++-----
> > 1 file changed, 32 insertions(+), 5 deletions(-)
> >
> > diff --git a/libavdevice/decklink_dec.cpp b/libavdevice/decklink_dec.cpp
> > index 94dae26..bceced5 100644
> > --- a/libavdevice/decklink_dec.cpp
> > +++ b/libavdevice/decklink_dec.cpp
> > @@ -67,8 +67,7 @@ typedef struct VANCLineNumber {
> >  * another source during switching*/
> > static VANCLineNumber vanc_line_numbers[] = {
> >     /* SD Modes */
> > -
> > -    {bmdModeNTSC, 11, 19, 274, 282},
> > +    {bmdModeNTSC, 4, 21, 24, 284},
>
> I hadn’t previously reviewed this table, but now that I look at it, I’m
> not confident your proposed patch is correct.  VANC data generally cannot
> appear until after the switching line (through the first line of active
> video), which is on line 10 for NTSC 480i video.  Could you elaborate on
> what equipment you have which is putting VBI data out on lines 4-10?
> Agreed though that the upper limit for field one is incorrect - it should
> be 21 as your patch proposed.
>
> Also, it’s highly unlikely that you really want to search line 24-284 for
> field 2 VBI data.  Perhaps you meant 274-284?
>
> FYI:  SMPTE RP 168-2009 is the normative reference for the location of
> switching lines across various video resolutions/framerates.
>
> For NTSC my source hardware was outputting on line 12, so I'll resubmit
the patch with it adjusted after I dig up SMPTE RP 168-2009 and check the
values.. The one I definitely had to modify in order for the search to find
the data was bmdModeHD1080i5994, when I get to the office I'll hook up the
signal to my scope and check the exact line and also run a test in 720p


>
> >     {bmdModeNTSC2398, 11, 19, 274, 282},
> >     {bmdModePAL, 7, 22, 320, 335},
> >     {bmdModeNTSCp, 11, -1, -1, 39},
> > @@ -82,7 +81,7 @@ static VANCLineNumber vanc_line_numbers[] = {
> >     {bmdModeHD1080p2997, 8, -1, -1, 42},
> >     {bmdModeHD1080p30, 8, -1, -1, 42},
> >     {bmdModeHD1080i50, 8, 20, 570, 585},
> > -    {bmdModeHD1080i5994, 8, 20, 570, 585},
> > +    {bmdModeHD1080i5994, 6, 30, 568, 595},
>
> For 1080i the switching line is line 7, hence VBI should appear on line
> 8+.  It’s possible for data to appear on line 7 if the transmitting
> equipment is misconfigured, and only some cards can actually capture that
> data (for example, the Decklink Duo cannot but the Decklink Duo2 can).
>
> Again, what is the signal source?
>

This makes sense, VBI was on line 12 for 1080i. The source was a Matrox DSX
LE4.


>
> >     {bmdModeHD1080i6000, 8, 20, 570, 585},
> >     {bmdModeHD1080p50, 8, -1, -1, 42},
> >     {bmdModeHD1080p5994, 8, -1, -1, 42},
> > @@ -92,7 +91,7 @@ static VANCLineNumber vanc_line_numbers[] = {
> >
> >     {bmdModeHD720p50, 8, -1, -1, 26},
> >     {bmdModeHD720p5994, 8, -1, -1, 26},
> > -    {bmdModeHD720p60, 8, -1, -1, 26},
> > +    {bmdModeHD720p60, 7, -1, -1, 26},
>
> Same questions as with the 1080i change above.  Also, switching line
> location is the same for 720p/5994 and 720p/60, so presumably we would want
> to adjust both if it is actually needed.
>
> >
> >     /* For all other modes, for which we don't support VANC */
> >     {bmdModeUnknown, 0, -1, -1, -1}
> > @@ -149,6 +148,30 @@ static void extract_luma_from_v210(uint16_t *dst,
> const uint8_t *src, int width)
> >     }
> > }
> >
> > +static void unpack_v210(uint16_t *dst, const uint8_t *src, int width)
> > +{
> > +    int i;
> > +    for (i = 0; i < width / 6; i++) {
> > +        *dst++ =  src[0]       + ((src[1] & 3)  << 8);
> > +        *dst++ = (src[1] >> 2) + ((src[2] & 15) << 6);
> > +        *dst++ = (src[2] >> 4) + ((src[3] & 63) << 4);
> > +
> > +        *dst++ =  src[4]       + ((src[5] & 3)  << 8);
> > +        *dst++ = (src[5] >> 2) + ((src[6] & 15) << 6);
> > +        *dst++ = (src[6] >> 4) + ((src[7] & 63) << 4);
> > +
> > +        *dst++ =  src[8]       + ((src[9] & 3)  << 8);
> > +        *dst++ = (src[9] >> 2) + ((src[10] & 15) << 6);
> > +        *dst++ = (src[10] >> 4) + ((src[11] & 63) << 4);
> > +
> > +        *dst++ =  src[12]       + ((src[13] & 3)  << 8);
> > +        *dst++ = (src[13] >> 2) + ((src[14] & 15) << 6);
> > +        *dst++ = (src[14] >> 4) + ((src[15] & 63) << 4);
> > +
> > +        src += 16;
> > +    }
> > +}
> > +
> > static uint8_t calc_parity_and_line_offset(int line)
> > {
> >     uint8_t ret = (line < 313) << 5;
> > @@ -741,7 +764,11 @@ HRESULT
> decklink_input_callback::VideoInputFrameArrived(
> >                         uint8_t *buf;
> >                         if (vanc->GetBufferForVerticalBlankingLine(i,
> (void**)&buf) == S_OK) {
> >                             uint16_t luma_vanc[MAX_WIDTH_VANC];
> > -                            extract_luma_from_v210(luma_vanc, buf,
> videoFrame->GetWidth());
> > +                            if (ctx->bmd_mode == bmdModeNTSC) {
> > +                              unpack_v210(luma_vanc, buf,
> videoFrame->GetWidth());
> > +                            } else {
> > +                              extract_luma_from_v210(luma_vanc, buf,
> videoFrame->GetWidth());
> > +                            }
>
> So I have to admit I find this bit odd.  For composite video, the VBI data
> would always be in the luma region (in fact, most VBI decoders will strip
> out the chroma channel prior to processing).  For SDI, it’s possible that
> VANC could appear in the luma or chroma channel, but it wouldn’t be be
> interleaved across both.
>
> I would be really interested to know what the source equipment is here,
> since I’m seeing a number of things that I believe would violate the
> various standards.  While I’m a big believer in Postel’s Law, I would
> really like to know whether this is just some really bad output
> implementation, and if it’s open source then bugs should be raised against
> it rather than putting a bunch of hacks into ffmpeg.  I guess it’s also
> possible the video went through a scaler which scaled the video but
> preserved the original VBI format (which would be bad but not terribly
> surprising).
>

The source equipment was again a Matrox LE4. The source file was 1080i
while I was testing the NTSC output, so it was perhaps a scaling issue.
However the spec says that VANC data will be on the luma channel for high
definition sources, so that is why I tried extracting it from the
interleaved and was able to successfully parse the VANC packets.

I chatted with someone in the video developers slack community and they
suggested that for NTSC the data is indeed interleaved.

I have access to some SD captioned files that I can play out my test
hardware to eliminate scaling.

-ray


>
> Cheers,
>
> Devin


More information about the ffmpeg-devel mailing list