[FFmpeg-devel] [PATCH] lavfi/buffersrc: push the frame deeper if requested.

[Nicolas George]

Le primidi 21 messidor, an CCXXV, Marton Balint a écrit :
> I am not saying they aren't, I'm just showing a case where your revert is
> breaking another case which was working before. Nicolas also stated there
> can be cases where the old code (before the revert) worked better. I just
> hope that I found a simple enough case, so somebody can take a look and find
> the root cause.

Thanks for stepping in. I will try to dot the ‘ı’s in this issue.

From the start, libavfilter has had subtle issues in corner cases. These
issue reside primarily with filters with several inputs and the handling
of EOF. We all tried to get rid of them, but soon realized they are like
lamination bubbles: once you think you are rid of one, another pops up a
few centimeters away, the only solution is to redo almost the whole
thing.

I have been doing just that for libavfilter. And since doing it all at
once is not an option, I have been doing it step by step. The
consequence is that there is a large stitch between the new and the old
code, that may be as annoying as the bubbles, and even more so when the
stitch approaches areas where there are bubbles.

Dropping the metaphor: I have been reworking the scheduling of
libavfilter to get filter with multiple inputs and EOF working, but
look, I don't mean to be rude, but this is not as easy as it looks.

Since changing all the filters is not an option, I had to include a
compatibility wrapper to have the old implementations work in the new
design. This compatibility wrapper is obviously not perfect. In fact, it
probably cannot be perfect, because if it could, we would not need it in
the first place.

It took time, but the new design is there, and as far as I know, it
works. Filters with several inputs are activated when needed, EOF is
propagated with its timestamp (but ffmpeg does not inject it because the
patch series implementing it was blocked by bikeshedding). The only
thing that does not work is the existing filters. That makes quite a big
“only”, I grant you.

Now, I realize how frustrating to have a feature in a filter that used
to work and no longer does, especially if it is a feature that you
implemented yourself lovingly, and I sympathize.

My goal, obviously, is to have everything working. But it takes time,
especially when alone.

Also, there have been unexpected stuff in my personal life that have
left me with less time than before to work on ffmpeg.

Now, if a feature has been broken and you want it back, you can have two
options: you can revert, or you can help move forward.

If you revert, you will possibly fix the feature, but you will also
bring back all the issues that were fixed by these commits. Plus
possibly other issues due to code changes that expect the new behaviour.
And thus, you will have to revert and revert and revert.

Also, you make more work for me, delaying by that much the time where
the work is done.

If you want to help move forward, on the other hand... IMO, it is the
only sane option.

Now, I also realize that helping move forward is easier said than done,
even more so since I have not documented the new design. Documentation
takes time, documenting in details something that does not work yet is a
waste of time. Still, there is already quite a lot in the doxygen
comments, and if somebody wants to help, I will give any information
needed.

And I will do so for the issue at hand.

The problem is that with the "shortest" option, filters that use the
"dualinput" system do not report EOF correctly.

I have not yet been able to determine if the problem is in the dualinput
system, not returning an error when it should, or in the wrapper code,
not taking the return code into account properly.

Still, I am not sure understanding that and fixing it would be a good
use of the time, because it may not be possible, and because the filters
with several inputs need to be rewritten with the new framework in mind:
only then can they reap the benefits, especially getting rid of the
"buffer queue overflow" problems.

So, how to write a filter like that with the new framework?

There is a single callback, called "activate". It must, in succession:

- Examine the input links (using:
  http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20170615/4490ac33/attachment.patch
  ):

  - If a link has no queued frames but a status, use
    ff_inlink_acknowledge_status().

  - If the queued frames (and if relevant status) allow to produce a
    frame on output, do it and return.

  - If the input status change allow to deduce that one output or
    several are finished, report it with
    ff_avfilter_link_set_in_status() and return.

- Examine the output links (using a wrapper, to plan for future
  evolutions such as mutexes)

  - If frame_wanted_out is set on an output, find out which inputs would
    require a frame to allow to produce one on that output and request
    them using ff_inlink_request_frame().

- If nothing could be done, just return FFERROR_NOT_READY (it will need
  to be moved into filters.h).

I think doing that work for the dualinput / framesync filters is not a
very complex work, but I am afraid it will require similar changes to
all filters using it.


(A side note about lamination: if what you want to laminate is
water-resistant, adding a layer of soapy water will help: it will not
damage the adhesive but slow it down, and the bubbles, filled with water
instead of air, are much easier to lead to the edges. But do not do that
to apply a screen protector on a capacitive touchscreen, they do not
like to get wet!)

Regards,

-- 
  Nicolas George
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20170711/7bfd203a/attachment.sig>