[FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Kieran Kunhya]

On Wed, Jan 22, 2025 at 4:53 PM Romain Beauxis <romain.beauxis at gmail.com> wrote:
>
> Le mer. 22 janv. 2025 à 10:29, Kieran Kunhya via ffmpeg-devel
> <ffmpeg-devel at ffmpeg.org> a écrit :
> >
> > On Wed, Jan 22, 2025 at 3:33 PM Michael Niedermayer
> > <michael at niedermayer.cc> wrote:
> > >
> > > On Mon, Jan 20, 2025 at 06:46:46AM +0000, Kieran Kunhya via ffmpeg-devel wrote:
> > > > >
> > > > > > The data arrives on multiple sockets, leading to all sorts
> > > > > > of opportunities for timing behavior and reordering issues.
> > > > >
> > > > > how does this matter?
> > > > >
> > > >
> > > > There are routers that put traffic on a different port down a different ISP
> > > > so you have to compensate for latency delays between the two links. You
> > > > can't just "buffer N packets".
> > > >
> > > > Another edge case.
> > >
> > > I dont know why you assume "buffer N packets".
> > >
> > > Packets enter this buffer when they are received
> > > Packets should exit that (buffer + FEC) when they are needed for decompression
> > > (for presentation to the user) or slightly prior
> > >
> > > Removing them from the buffer earlier has a higher propability of failure
> > > so it is strictly worse.
> >
> > Wrong, the packets leave the buffer at a fixed duration corresponding
> > to (an estimate of) 2 times the matrix duration (the spec was written
> > for CBR so this is a constant) so that latency is fixed. There is
> > packet loss (by definition) so a buffer of N packets may be much
> > larger than 2 times the matrix duration.
> >
> > "when they are needed for decompression" -> this is real-time RTP, not a file.
> > The issue you have with FEC going down a different path is the FEC
> > packets can arrive a lot earlier than the latency window or a lot
> > later and you have to compensate for this with heuristics.
> >
> > All packets can be reordered or even duplicated which as far as I know
> > this code doesn't cover either.
>
> Packet reordering is covered.
>
> Packets are stored in a binary tree indexed by their sequence ID and
> pulled out based on that ID as well. That takes care of reordering.
>
> Duplicated packets are not properly covered yet but the change is
> trivial, this will be in the next version of the patch, thanks for
> pointing this out.
>
> I have posted detailed feedback with testing in real conditions, do
> you have any feedback on that?

Do you plan to support a fixed duration? Buffering N packets has a
variable duration when there is packet loss.
Do you plan to support calculating the matrix size from the RTP FEC packets?
Do you support block and non-block aligned permutations?

Also saying you don't support an encoder restart is basically saying
you want to write something that only deals with steady state
behaviour.

Kieran