[PATCH] avfilter: merge loudnorm filter functionality into f_ebur128.c
Attached.
Attached.
Hi, On Wed, Nov 15, 2023 at 12:39 PM Paul B Mahol <onemda@gmail.com> wrote:
Attached.
Only had a few minutes to look at this. Seems like more than just merging two filters, I see a bunch of new filter options for example. Can you explain? Thanks, Kyle
On Fri, Nov 17, 2023 at 7:38 AM Kyle Swanson <k@ylo.ph> wrote:
Hi,
On Wed, Nov 15, 2023 at 12:39 PM Paul B Mahol <onemda@gmail.com> wrote:
Attached.
Only had a few minutes to look at this. Seems like more than just merging two filters, I see a bunch of new filter options for example. Can you explain?
The linear mode and scanning, both input to filter and filter output itself should give similar results. The dynamic mode now actually can be configured how aggressively it will expand / compress audio. Because current state of filter have numerous issues: - using unmaintained old libebur128 module, when same functionality is already available in existing filter. - code duplication and functionality duplication due the above - buggy limiter - causing clipped samples randomly - buggy first and final frame filtering - over-complicated flow path for dynamic code in filter - excessive compressing of audio dynamic range, causing extreme smaller LRU from output audio - and probably more that I forgot Some options from this patch can be probably removed, like attack/release options, and just use defaults as currently in patch.
Thanks, Kyle _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
On Sun, 19 Nov 2023, Paul B Mahol wrote:
On Fri, Nov 17, 2023 at 7:38 AM Kyle Swanson <k@ylo.ph> wrote:
Hi,
On Wed, Nov 15, 2023 at 12:39 PM Paul B Mahol <onemda@gmail.com> wrote:
Attached.
Only had a few minutes to look at this. Seems like more than just merging two filters, I see a bunch of new filter options for example. Can you explain?
The linear mode and scanning, both input to filter and filter output itself should give similar results. The dynamic mode now actually can be configured how aggressively it will expand / compress audio. Because current state of filter have numerous issues:
- using unmaintained old libebur128 module, when same functionality is already available in existing filter. - code duplication and functionality duplication due the above - buggy limiter - causing clipped samples randomly - buggy first and final frame filtering - over-complicated flow path for dynamic code in filter - excessive compressing of audio dynamic range, causing extreme smaller LRU from output audio - and probably more that I forgot
Some options from this patch can be probably removed, like attack/release options, and just use defaults as currently in patch.
Previously ebur128 functionality was decoupled from specific filters, so there was a chance that multiple filters can use it. Unfortunately f_ebur128.c was never migrated to use internal ebur128 lib, as far as I remember the maintaner rejected the idea for some reason back then. IMHO having some generic ebur128 functionality would be preferable. I have an old patch for example which adds EBUR128 mode to af_dynaudnorm, see attached for reference. Looks much cleaner than af_loudnorm, which was always a bit overcomplicated and slightly buggy, as you mentioned. So please consider two things: - Can you keep some generic ebur128 functionality which can easily reused by multiple filters? I don't mind if it is the old code from ebur128 lib or current code from f_ebur128, but it should be reusable internal ff_ functions. - Does it make sense to maintain a separate loudnorm filter for EBUR128 loudness, or it can be integrated into af_dynaudnorm? Because I kind of think that having this as a option of af_dynaudnorm would be cleaner, at least for the dynamic normalization functionality. For the linear mode, well, we already have compressor filters, so I am not sure if that mode is worth keeping. But maybe it is easier for the end user, I don't know. Thanks, Marton
On Sun, Nov 19, 2023 at 10:55 PM Marton Balint <cus@passwd.hu> wrote:
On Sun, 19 Nov 2023, Paul B Mahol wrote:
On Fri, Nov 17, 2023 at 7:38 AM Kyle Swanson <k@ylo.ph> wrote:
Hi,
On Wed, Nov 15, 2023 at 12:39 PM Paul B Mahol <onemda@gmail.com> wrote:
Attached.
Only had a few minutes to look at this. Seems like more than just merging two filters, I see a bunch of new filter options for example. Can you explain?
The linear mode and scanning, both input to filter and filter output itself should give similar results. The dynamic mode now actually can be configured how aggressively it will expand / compress audio. Because current state of filter have numerous issues:
- using unmaintained old libebur128 module, when same functionality is already available in existing filter. - code duplication and functionality duplication due the above - buggy limiter - causing clipped samples randomly - buggy first and final frame filtering - over-complicated flow path for dynamic code in filter - excessive compressing of audio dynamic range, causing extreme smaller LRU from output audio - and probably more that I forgot
Some options from this patch can be probably removed, like attack/release options, and just use defaults as currently in patch.
Previously ebur128 functionality was decoupled from specific filters, so there was a chance that multiple filters can use it. Unfortunately f_ebur128.c was never migrated to use internal ebur128 lib, as far as I remember the maintaner rejected the idea for some reason back then.
Because its pointless, both use histograms, and this is just duplicated code. Also f_ebur128.c code/filter come first, even though its scanner only and have visuals.
IMHO having some generic ebur128 functionality would be preferable. I have an old patch for example which adds EBUR128 mode to af_dynaudnorm, see attached for reference. Looks much cleaner than af_loudnorm, which was always a bit overcomplicated and slightly buggy, as you mentioned.
So please consider two things:
- Can you keep some generic ebur128 functionality which can easily reused by multiple filters? I don't mind if it is the old code from ebur128 lib or current code from f_ebur128, but it should be reusable internal ff_ functions.
That can be done, I already done something similar in this patch, but it can be extended and improved.
- Does it make sense to maintain a separate loudnorm filter for EBUR128 loudness, or it can be integrated into af_dynaudnorm? Because I kind of think that having this as a option of af_dynaudnorm would be cleaner, at least for the dynamic normalization functionality. For the linear mode, well, we already have compressor filters, so I am not sure if that mode is worth keeping. But maybe it is easier for the end user, I don't know.
dynaudnorm filter is using different approach completely. It can have delay less than second and more that 30 seconds depending on parameters. ebur128 is using 3 second window and also history of previous measurements, and minimal delay is 1/10 of second. Also ebur128 applies special biquad before calculating gains... So the approach of loudnorm vs dynaudnorm is completely different and does not make sense to put it into dynaudnorm. (one could add optional side-chain stream to dynaudnorm but that is max where i would go) Linear mode of loudnorm filter is just volume filter. It just set single gain factor by interpreting parameters from input. (The only drawback is that you rescan audio again, twice, input of filter and output of filter, so there are 4 scans in 2-pass mode, when there is only 1 scan pass needed for linear mode....) Also linear mode is used heavily by other users/project as can be found on net. Dynamic mode (for real-time streams) is also used, even in its current state, thats why I wanted to cleanup it.
Thanks, Marton_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Hi, On Sun, Nov 19, 2023 at 3:48 AM Paul B Mahol <onemda@gmail.com> wrote:
On Fri, Nov 17, 2023 at 7:38 AM Kyle Swanson <k@ylo.ph> wrote:
Hi,
On Wed, Nov 15, 2023 at 12:39 PM Paul B Mahol <onemda@gmail.com> wrote:
Attached.
Only had a few minutes to look at this. Seems like more than just merging two filters, I see a bunch of new filter options for example. Can you explain?
The linear mode and scanning, both input to filter and filter output itself should give similar results. The dynamic mode now actually can be configured how aggressively it will expand / compress audio. Because current state of filter have numerous issues:
- using unmaintained old libebur128 module, when same functionality is already available in existing filter. - code duplication and functionality duplication due the above - buggy limiter - causing clipped samples randomly - buggy first and final frame filtering - over-complicated flow path for dynamic code in filter - excessive compressing of audio dynamic range, causing extreme smaller LRU from output audio - and probably more that I forgot
Some options from this patch can be probably removed, like attack/release options, and just use defaults as currently in patch.
Thanks, Kyle
OK. Give me some time to review this. Unfortunately, I can't be quick since it is a Holiday week here and the diff is pretty hard to read. I'm hoping this still gives exact/equivalent loudnorm output, since this is a pretty widely used filter. My first reaction is combining the filters like this might make things more complicated, but I will keep an open mind while reviewing. Thanks, Kyle
Major change: loudnorm no longer returns oversampled audio at 192000 Hz when doing dynamic processing. Oversampled audio is only used for true peak finding now. This was trivial improvement as possible with ebur128 code. Minor changes: numerous stability fixes
Quoting Paul B Mahol (2023-11-28 17:51:14)
Major change: loudnorm no longer returns oversampled audio at 192000 Hz when doing dynamic processing. Oversampled audio is only used for true peak finding now. This was trivial improvement as possible with ebur128 code. Minor changes: numerous stability fixes
This patch in unreviewable and should be split. -- Anton Khirnov
On Thu, Nov 30, 2023 at 12:43 PM Anton Khirnov <anton@khirnov.net> wrote:
Quoting Paul B Mahol (2023-11-28 17:51:14)
Major change: loudnorm no longer returns oversampled audio at 192000 Hz when doing dynamic processing. Oversampled audio is only used for true peak finding now. This was trivial improvement as possible with ebur128 code. Minor changes: numerous stability fixes
This patch in unreviewable and should be split.
It cant be split, its unreviewable only for those lacking advanced C and git skills.
-- Anton Khirnov _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Quoting Paul B Mahol (2023-11-30 13:48:16)
On Thu, Nov 30, 2023 at 12:43 PM Anton Khirnov <anton@khirnov.net> wrote:
Quoting Paul B Mahol (2023-11-28 17:51:14)
Major change: loudnorm no longer returns oversampled audio at 192000 Hz when doing dynamic processing. Oversampled audio is only used for true peak finding now. This was trivial improvement as possible with ebur128 code. Minor changes: numerous stability fixes
This patch in unreviewable and should be split.
It cant be split,
Why not? -- Anton Khirnov
On Thu, Nov 30, 2023 at 2:47 PM Anton Khirnov <anton@khirnov.net> wrote:
Quoting Paul B Mahol (2023-11-30 13:48:16)
On Thu, Nov 30, 2023 at 12:43 PM Anton Khirnov <anton@khirnov.net> wrote:
Quoting Paul B Mahol (2023-11-28 17:51:14)
Major change: loudnorm no longer returns oversampled audio at 192000 Hz when doing dynamic processing. Oversampled audio is only used for true peak finding now. This was trivial improvement as possible with ebur128 code. Minor changes: numerous stability fixes
This patch in unreviewable and should be split.
It cant be split,
Why not?
Because it is new code and refactoring, and I know it enough that I'm confident to claim so.
-- Anton Khirnov _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Quoting Paul B Mahol (2023-11-30 15:01:23)
On Thu, Nov 30, 2023 at 2:47 PM Anton Khirnov <anton@khirnov.net> wrote:
Quoting Paul B Mahol (2023-11-30 13:48:16)
On Thu, Nov 30, 2023 at 12:43 PM Anton Khirnov <anton@khirnov.net> wrote:
Quoting Paul B Mahol (2023-11-28 17:51:14)
Major change: loudnorm no longer returns oversampled audio at 192000 Hz when doing dynamic processing. Oversampled audio is only used for true peak finding now. This was trivial improvement as possible with ebur128 code. Minor changes: numerous stability fixes
This patch in unreviewable and should be split.
It cant be split,
Why not?
Because it is new code and refactoring,
Then it should be first refactoring, then new code.
and I know it enough that I'm confident to claim so.
Hubris usually implies more bugs. -- Anton Khirnov
On Thu, Nov 30, 2023 at 2:57 PM Anton Khirnov <anton@khirnov.net> wrote:
Quoting Paul B Mahol (2023-11-30 15:01:23)
On Thu, Nov 30, 2023 at 2:47 PM Anton Khirnov <anton@khirnov.net> wrote:
Quoting Paul B Mahol (2023-11-30 13:48:16)
On Thu, Nov 30, 2023 at 12:43 PM Anton Khirnov <anton@khirnov.net> wrote:
Quoting Paul B Mahol (2023-11-28 17:51:14)
Major change: loudnorm no longer returns oversampled audio at 192000 Hz when doing dynamic processing. Oversampled audio is only used for true peak finding now. This was trivial improvement as possible with ebur128 code. Minor changes: numerous stability fixes
This patch in unreviewable and should be split.
It cant be split,
Why not?
Because it is new code and refactoring,
Then it should be first refactoring, then new code.
I wrote it wrongly its more than refactoring as code shares no similar code between this two filters.
and I know it enough that I'm confident to claim so.
Hubris usually implies more bugs.
I tested this a lot, and its great move forward. Make more useful testing and review next time, I'm sure you are quite capable person.
-- Anton Khirnov _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Hi, On Thu, Nov 30, 2023 at 6:11 AM Paul B Mahol <onemda@gmail.com> wrote:
I tested this a lot, and its great move forward.
Make more useful testing and review next time, I'm sure you are quite capable person.
Paul, I agree with Anton. Refactoring the code (i.e. changing filter behavior) and combining it with f_ebur128.c all at once makes the diff very hard to review. I suggest an incremental approach, let's address the issues you've identified with the loudnorm filter one by one, and then a second stage where we see about combining with f_ebur128.c. Thanks, Kyle
On Thu, Nov 30, 2023 at 8:39 PM Kyle Swanson <k@ylo.ph> wrote:
Hi,
On Thu, Nov 30, 2023 at 6:11 AM Paul B Mahol <onemda@gmail.com> wrote:
I tested this a lot, and its great move forward.
Make more useful testing and review next time, I'm sure you are quite capable person.
Paul, I agree with Anton. Refactoring the code (i.e. changing filter behavior) and combining it with f_ebur128.c all at once makes the diff very hard to review. I suggest an incremental approach, let's address the issues you've identified with the loudnorm filter one by one, and then a second stage where we see about combining with f_ebur128.c.
Loudnorm filter is big pile of hacks, I wanted to move forward and I improved it. I received nothing in return just some politics talks. I will apply this soon unless technical comments arise. Why would I spend on this more my precious time? For no real gain as I will again receive nothing in return except some useless comment and no single technical aspect.
Thanks, Kyle _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Hi, On Thu, Nov 30, 2023 at 1:36 PM Paul B Mahol <onemda@gmail.com> wrote:
Loudnorm filter is big pile of hacks, I wanted to move forward and I improved it.
I received nothing in return just some politics talks.
I will apply this soon unless technical comments arise.
Why would I spend on this more my precious time? For no real gain as I will again receive nothing in return except some useless comment and no single technical aspect.
Please don't merge this as-is. I'm sure there are good changes/fixes here that we should merge, but you need to help your reviewers understand what is going on. One big commit that combines both refactoring across files with introduction of new filter behavior is very hard to review. That's why I'm suggesting we do this in two steps, I think Anton's suggestion is the same. Thanks, Kyle
On Thu, Nov 30, 2023 at 11:20 PM Kyle Swanson <k@ylo.ph> wrote:
Hi,
On Thu, Nov 30, 2023 at 1:36 PM Paul B Mahol <onemda@gmail.com> wrote:
Loudnorm filter is big pile of hacks, I wanted to move forward and I improved it.
I received nothing in return just some politics talks.
I will apply this soon unless technical comments arise.
Why would I spend on this more my precious time? For no real gain as I
will
again receive nothing in return except some useless comment and no single technical aspect.
Please don't merge this as-is. I'm sure there are good changes/fixes here that we should merge, but you need to help your reviewers understand what is going on. One big commit that combines both refactoring across files with introduction of new filter behavior is very hard to review. That's why I'm suggesting we do this in two steps, I think Anton's suggestion is the same.
But how you could refactor code if one filter shares nothing with another filter code? Its not possible. You all seem to not understand problem at all.
Thanks, Kyle _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Hi, On Thu, Nov 30, 2023 at 2:43 PM Paul B Mahol <onemda@gmail.com> wrote:
But how you could refactor code if one filter shares nothing with another filter code?
Its not possible. You all seem to not understand problem at all.
I get that this patch swaps the libebur128 loudness computation with the f_ebur128 code. But it also changes the behavior of the AGC and limiter, right? It looks like a whole new algorithm. That's what I mean by new filter behavior. Thanks, Kyle
On Fri, Dec 1, 2023 at 12:29 AM Kyle Swanson <k@ylo.ph> wrote:
Hi,
On Thu, Nov 30, 2023 at 2:43 PM Paul B Mahol <onemda@gmail.com> wrote:
But how you could refactor code if one filter shares nothing with another filter code?
Its not possible. You all seem to not understand problem at all.
I get that this patch swaps the libebur128 loudness computation with the f_ebur128 code. But it also changes the behavior of the AGC and limiter, right? It looks like a whole new algorithm. That's what I mean by new filter behavior.
Keeping old code will not make patch any prettier.
Thanks, Kyle _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Hi, On Fri, Dec 1, 2023 at 2:37 AM Paul B Mahol <onemda@gmail.com> wrote:
Keeping old code will not make patch any prettier.
It's not really about pretty, it's about splitting up the patch so we can do things one at a time. I think the change to use f_ebur128.c loudness computation might have some advantages, but I can't test them in isolation. The changes to the loudnorm algorithm I have more questions about, but I can't test those in isolation either. Thanks, Kyle
On Fri, Dec 1, 2023 at 10:13 PM Kyle Swanson <k@ylo.ph> wrote:
Hi,
On Fri, Dec 1, 2023 at 2:37 AM Paul B Mahol <onemda@gmail.com> wrote:
Keeping old code will not make patch any prettier.
It's not really about pretty, it's about splitting up the patch so we can do things one at a time. I think the change to use f_ebur128.c loudness computation might have some advantages, but I can't test them in isolation. The changes to the loudnorm algorithm I have more questions about, but I can't test those in isolation either.
Define what isolation means for you? You all are just allowing poor code state and quality stay.
Thanks, Kyle _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
participants (4)
-
Anton Khirnov -
Kyle Swanson -
Marton Balint -
Paul B Mahol