[FFmpeg-devel] Audio conversion and floating-point codecs

[Måns Rullgård]

Peter Ross <pross at xvid.org> writes:

> On Tue, Jul 06, 2010 at 03:13:26PM +0100, M?ns Rullg?rd wrote:
>> Peter Ross <pross at xvid.org> writes:
>> 
>> > On Sat, May 15, 2010 at 08:17:51PM +0100, M?ns Rullg?rd wrote:
>> >> There is a long-standing desire from some to make the floating-point
>> >> decoders output float samples instead of converting to int16
>> >> internally, and I agree with the reasons for this.  However, making
>> >> this change hastily will make decoding orders of magnitude slower on
>> >> many CPUs.  The reason is that when a decoder outputs float samples,
>> >> the fast asm code for float-to-int conversion is not used.
>> >> 
>> >> In order to change the output format of these decoders without
>> >> impacting performance, we must first make a few improvements to the
>> >> avcodec API and to the generic audio format conversion code.
>> > [...]
>> >
>> >> - The decoders should output planar audio instead of interleaved for
>> >>   multichannel streams.  This probably means introducing
>> >>   avcodec_decode_audio4() with an AVFrame output.
>> >
>> > Q: does it make sense to expand the existing AVFrame structure, or
>> > define a new struct specific to audio?
>> >
>> > #define FF_MAX_CHANNELS  8
>> > struct AVAudioFrame {
>> >     uint16_t *data[FF_MAX_CHANNELS];
>> > };
>> 
>> I've posed the same question myself, without finding a good answer.
>> Some codecs support a huge number of channels.  I can say for sure,
>
> Second contenious point:
> At present, the user allocates the samples buffer that is handed
> of to avcodec_decode_audioN().
>
> IMHO this is sloppy. Just look at how ffmpeg.c guesses the buffer size.
> The alternative is to have the decoder do it, e.g. by calling
> avctx->get_buffer() with the number the samples/channels to be output.
> Thoughts?

Makes sense, although get_buffer() is presently rather image-centric.

>> however, that uint16_t is the wrong data type to use here.
>
> Oops. I intended int16_t.

Also not good.  We want to allow any sample format.

-- 
M?ns Rullg?rd
mans at mansr.com