[FFmpeg-devel] [RFC] AAC Encoder
Kostya
kostya.shishkov
Wed Aug 13 15:42:56 CEST 2008
On Wed, Aug 13, 2008 at 02:57:50PM +0200, Michael Niedermayer wrote:
> On Wed, Aug 13, 2008 at 09:16:48AM +0300, Kostya wrote:
> > On Tue, Aug 12, 2008 at 07:48:59PM +0200, Michael Niedermayer wrote:
> > > On Tue, Aug 12, 2008 at 08:09:36PM +0300, Kostya wrote:
> > > > On Tue, Aug 12, 2008 at 02:14:20PM +0200, Michael Niedermayer wrote:
> > [...]
> > > > > We have a problem here, because this isnt optimal
> > > > > It seems we agree that each bit counts the same no matter what psy says.
> > > > > Maybe a example will best show the problem
> > > > > lets assume we have a coeff of 11.5, the psy model decides that a change
> > > > > to 10 would be ok for the given audio quality/bitrate and thus outputs 10
> > > > > let us assume that storing a coefficient of 10 and one of 11 both take
> > > > > 7 bit, the decission to store 10 clearly was bad. OTOH it could have
> > > > > been that storing 11 requires twice as many bits in which case the
> > > > > decission would have been good. One simply cannot quantize values optimally
> > > > > without considering the number of bits they need. This is even more true
> > > > > for vector quantization based codecs than it is for scalar quantization.
> > > > > it may very well be that psy thinks that both {-1,1} and {-2,0} are an
> > > > > equally good representation of the exact {-1.5,0.5} but its not until
> > > > > the encoding that it becomes known which of the two need fewer bits.
> > > > >
> > > > > Id say the psy model should return an array of perceptual weights W[i]
> > > > > and the bitstream encode should choose the (global) minimum of
> > > > > bits[i] + distortion(W[i], coeff[i]-stored[i])
> > > > > where distortion is a appropriate function whos output matches how audible
> > > > > a change is, this may be a simple W[i]*(coeff[i]-stored[i])^2 but iam no
> > > > > psychoacoustic expert so there may be better choices.
> > > > >
> > > > > And of course the suggested system above needs to be compared to what you
> > > > > have currenty so that we can be sure it really does sound better.
> > > >
> > > > I understand what you mean but I suspect that is of complexity O("shaving piglets").
> > > >
> > > > I followed 3GPP TS26.403 which relies on perceptual entropy which more
> > > > or less corresponds to the number of bits needed to code it since it's easier.
> > > > Anyway, it would be easy to implement psy model that will consider
> > > > real coding cost vs. distortion.
> > >
> > > if you do not want to implement this then i will have to investigate if it
> > > is doable or not and why, could you provide me with some more elaborate
> > > explanation of where the problem is?
> >
> > Current scheme (just to clarify things a bit):
>
> > 1. encoder calls psy model functions to preprocess data
>
> This should eventually be done in a filter prior to the encoder, but that
> can wait until after its in svn and libavfilter is there and capable to
> filter audio
of course, since it's totally codec-independent thing
> > 2. then encoder calls psy model to determine frame and window type
>
> This is almost ok
> What the psy model should return is long_window, short_windows, dont_know
> and in the dont_know case both should be encoded and the one with better
> rate distortion choosen (distortion would be calculated by the psy model
> using whatever (posibly non trivial) method it sees fit.
> how often the dont_know case is returned could be determined by some
> speed/quality tradeoff option from the command line
it's possible and relatively easy to implement
> > 3. based on psy model suggestions, encoder performs windowing and MDCT
>
> ok
>
>
> > 4. encoder feeds coefficients to psy model
> > 5. psy model by some magic determines scalefactors and use them to convert
> > coefficients into integer form
> > 6. encoder encodes obtained scalefactors and integer coefficients
> >
> > There are 11 codebooks for AAC, each designed to code either pairs or quads
> > of values with sign coded separately or incorporated into value,
> > each has a maximum value limit.
> > While it's feasible to find the best encoding (like take raw coeff, quantize
> > it and round up or down, then see which vector takes less bits), I feel
> > it would be too slow.
>
> thats fine, you already have the fast variant implemented i do not suggest
> that to be removed, what we need is a high quality variant. The encoder should
> be better than other encoders ...
> Also as the max value you mentioned is another example of where your code
> fails fatally, a single +3 that would sound nearly as good when encoded as +2
> could force a less efficient code book to be choosen. Also the +3 could be
> encoded as a pulse, i dont remember if your code optimally choose between
> pulse and normal codebook encodings?
not optimally, unfortunately, but it can search for pulses and encode them
in any case, here's a new encoder version
> [...]
> --
> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> Dictatorship naturally arises out of democracy, and the most aggravated
> form of tyranny and slavery out of the most extreme liberty. -- Plato
-------------- next part --------------
/*
* AAC encoder
* Copyright (C) 2008 Konstantin Shishkov
*
* This file is part of FFmpeg.
*
* FFmpeg is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* FFmpeg is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with FFmpeg; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
/**
* @file aacenc.c
* AAC encoder
*/
/***********************************
* TODOs:
* psy model selection with some option
* change greedy codebook search into something more optimal, like Viterbi algorithm
* determine run lengths along with codebook
***********************************/
#include "avcodec.h"
#include "bitstream.h"
#include "dsputil.h"
#include "mpeg4audio.h"
#include "aacpsy.h"
#include "aac.h"
#include "aactab.h"
static const uint8_t swb_size_1024_96[] = {
4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 8, 8, 8, 8, 8,
12, 12, 12, 12, 12, 16, 16, 24, 28, 36, 44,
64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64
};
static const uint8_t swb_size_1024_64[] = {
4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 8, 8, 8, 8,
12, 12, 12, 16, 16, 16, 20, 24, 24, 28, 36,
40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40
};
static const uint8_t swb_size_1024_48[] = {
4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 8, 8, 8, 8, 8, 8, 8,
12, 12, 12, 12, 16, 16, 20, 20, 24, 24, 28, 28,
32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32,
96
};
static const uint8_t swb_size_1024_32[] = {
4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 8, 8, 8, 8, 8, 8, 8,
12, 12, 12, 12, 16, 16, 20, 20, 24, 24, 28, 28,
32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32
};
static const uint8_t swb_size_1024_24[] = {
4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
12, 12, 12, 12, 16, 16, 16, 20, 20, 24, 24, 28, 28,
32, 36, 36, 40, 44, 48, 52, 52, 64, 64, 64, 64, 64
};
static const uint8_t swb_size_1024_16[] = {
8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
12, 12, 12, 12, 12, 12, 12, 12, 12, 16, 16, 16, 16, 20, 20, 20, 24, 24, 28, 28,
32, 36, 40, 40, 44, 48, 52, 56, 60, 64, 64, 64
};
static const uint8_t swb_size_1024_8[] = {
12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12,
16, 16, 16, 16, 16, 16, 16, 20, 20, 20, 20, 24, 24, 24, 28, 28,
32, 36, 36, 40, 44, 48, 52, 56, 60, 64, 80
};
static const uint8_t *swb_size_1024[] = {
swb_size_1024_96, swb_size_1024_96, swb_size_1024_64,
swb_size_1024_48, swb_size_1024_48, swb_size_1024_32,
swb_size_1024_24, swb_size_1024_24, swb_size_1024_16,
swb_size_1024_16, swb_size_1024_16, swb_size_1024_8
};
static const uint8_t swb_size_128_96[] = {
4, 4, 4, 4, 4, 4, 8, 8, 8, 16, 28, 36
};
static const uint8_t swb_size_128_48[] = {
4, 4, 4, 4, 4, 8, 8, 8, 12, 12, 12, 16, 16, 16
};
static const uint8_t swb_size_128_24[] = {
4, 4, 4, 4, 4, 4, 4, 8, 8, 8, 12, 12, 16, 16, 20
};
static const uint8_t swb_size_128_16[] = {
4, 4, 4, 4, 4, 4, 4, 4, 8, 8, 12, 12, 16, 20, 20
};
static const uint8_t swb_size_128_8[] = {
4, 4, 4, 4, 4, 4, 4, 8, 8, 8, 8, 12, 16, 20, 20
};
static const uint8_t *swb_size_128[] = {
/* the last entry on the following row is swb_size_128_64 but is a
duplicate of swb_size_128_96 */
swb_size_128_96, swb_size_128_96, swb_size_128_96,
swb_size_128_48, swb_size_128_48, swb_size_128_48,
swb_size_128_24, swb_size_128_24, swb_size_128_16,
swb_size_128_16, swb_size_128_16, swb_size_128_8
};
#define CB_UNSIGNED 0x01 ///< coefficients are coded as absolute values
#define CB_PAIRS 0x02 ///< coefficients are grouped into pairs before coding (quads by default)
#define CB_ESCAPE 0x04 ///< codebook allows escapes
/** spectral coefficients codebook information */
static const struct {
int16_t maxval; ///< maximum possible value
int8_t cb_num; ///< codebook number
uint8_t flags; ///< codebook features
} aac_cb_info[] = {
{ 0, -1, CB_UNSIGNED }, // zero codebook
{ 1, 0, 0 },
{ 1, 1, 0 },
{ 2, 2, CB_UNSIGNED },
{ 2, 3, CB_UNSIGNED },
{ 4, 4, CB_PAIRS },
{ 4, 5, CB_PAIRS },
{ 7, 6, CB_PAIRS | CB_UNSIGNED },
{ 7, 7, CB_PAIRS | CB_UNSIGNED },
{ 12, 8, CB_PAIRS | CB_UNSIGNED },
{ 12, 9, CB_PAIRS | CB_UNSIGNED },
{ 8191, 10, CB_PAIRS | CB_UNSIGNED | CB_ESCAPE },
{ -1, -1, 0 }, // reserved
{ -1, -1, 0 }, // perceptual noise substitution
{ -1, -1, 0 }, // intensity out-of-phase
{ -1, -1, 0 }, // intensity in-phase
};
/** default channel configurations */
static const uint8_t aac_chan_configs[6][5] = {
{1, ID_SCE}, // 1 channel - single channel element
{1, ID_CPE}, // 2 channels - channel pair
{2, ID_SCE, ID_CPE}, // 3 channels - center + stereo
{3, ID_SCE, ID_CPE, ID_SCE}, // 4 channels - front center + stereo + back center
{3, ID_SCE, ID_CPE, ID_CPE}, // 5 channels - front center + stereo + back stereo
{4, ID_SCE, ID_CPE, ID_CPE, ID_LFE}, // 6 channels - front center + stereo + back stereo + LFE
};
/**
* AAC encoder context
*/
typedef struct {
PutBitContext pb;
MDCTContext mdct1024; ///< long (1024 samples) frame transform context
MDCTContext mdct128; ///< short (128 samples) frame transform context
DSPContext dsp;
DECLARE_ALIGNED_16(FFTSample, output[2048]); ///< temporary buffer for MDCT input coefficients
DECLARE_ALIGNED_16(FFTSample, tmp[1024]); ///< temporary buffer used by MDCT
int16_t* samples; ///< saved preprocessed input
int samplerate_index; ///< MPEG-4 samplerate index
const uint8_t *swb_sizes1024; ///< scalefactor band sizes for long frame
int swb_num1024; ///< number of scalefactor bands for long frame
const uint8_t *swb_sizes128; ///< scalefactor band sizes for short frame
int swb_num128; ///< number of scalefactor bands for short frame
ChannelElement *cpe; ///< channel elements
AACPsyContext psy; ///< psychoacoustic model context
int last_frame;
} AACEncContext;
/**
* Make AAC audio config object.
* @see 1.6.2.1 "Syntax - AudioSpecificConfig"
*/
static void put_audio_specific_config(AVCodecContext *avctx)
{
PutBitContext pb;
AACEncContext *s = avctx->priv_data;
init_put_bits(&pb, avctx->extradata, avctx->extradata_size*8);
put_bits(&pb, 5, 2); //object type - AAC-LC
put_bits(&pb, 4, s->samplerate_index); //sample rate index
put_bits(&pb, 4, avctx->channels);
//GASpecificConfig
put_bits(&pb, 1, 0); //frame length - 1024 samples
put_bits(&pb, 1, 0); //does not depend on core coder
put_bits(&pb, 1, 0); //is not extension
flush_put_bits(&pb);
}
static av_cold int aac_encode_init(AVCodecContext *avctx)
{
AACEncContext *s = avctx->priv_data;
int i;
avctx->frame_size = 1024;
for(i = 0; i < 16; i++)
if(avctx->sample_rate == ff_mpeg4audio_sample_rates[i])
break;
if(i == 16){
av_log(avctx, AV_LOG_ERROR, "Unsupported sample rate %d\n", avctx->sample_rate);
return -1;
}
if(avctx->channels > 6){
av_log(avctx, AV_LOG_ERROR, "Unsupported number of channels: %d\n", avctx->channels);
return -1;
}
s->samplerate_index = i;
s->swb_sizes1024 = swb_size_1024[i];
s->swb_num1024 = ff_aac_num_swb_1024[i];
s->swb_sizes128 = swb_size_128[i];
s->swb_num128 = ff_aac_num_swb_128[i];
dsputil_init(&s->dsp, avctx);
ff_mdct_init(&s->mdct1024, 11, 0);
ff_mdct_init(&s->mdct128, 8, 0);
// window init
ff_kbd_window_init(ff_aac_kbd_long_1024, 4.0, 1024);
ff_kbd_window_init(ff_aac_kbd_short_128, 6.0, 128);
ff_sine_window_init(ff_aac_sine_long_1024, 1024);
ff_sine_window_init(ff_aac_sine_short_128, 128);
s->samples = av_malloc(2 * 1024 * avctx->channels * sizeof(s->samples[0]));
s->cpe = av_mallocz(sizeof(ChannelElement) * aac_chan_configs[avctx->channels-1][0]);
if(ff_aac_psy_init(&s->psy, avctx, AAC_PSY_3GPP, aac_chan_configs[avctx->channels-1][0], 0, s->swb_sizes1024, s->swb_num1024, s->swb_sizes128, s->swb_num128) < 0){
av_log(avctx, AV_LOG_ERROR, "Cannot initialize selected model.\n");
return -1;
}
avctx->extradata = av_malloc(2);
avctx->extradata_size = 2;
put_audio_specific_config(avctx);
return 0;
}
static void apply_window_and_mdct(AVCodecContext *avctx, AACEncContext *s, ChannelElement *cpe, short *audio, int channel)
{
int i, j, k;
const float * lwindow = cpe->ch[channel].ics.use_kb_window[0] ? ff_aac_kbd_long_1024 : ff_aac_sine_long_1024;
const float * swindow = cpe->ch[channel].ics.use_kb_window[0] ? ff_aac_kbd_short_128 : ff_aac_sine_short_128;
const float * pwindow = cpe->ch[channel].ics.use_kb_window[1] ? ff_aac_kbd_short_128 : ff_aac_sine_short_128;
if (cpe->ch[channel].ics.window_sequence[0] != EIGHT_SHORT_SEQUENCE) {
memcpy(s->output, cpe->ch[channel].saved, sizeof(float)*1024);
if(cpe->ch[channel].ics.window_sequence[0] == LONG_STOP_SEQUENCE){
memset(s->output, 0, sizeof(s->output[0]) * 448);
for(i = 448; i < 576; i++)
s->output[i] = cpe->ch[channel].saved[i] * pwindow[i - 448];
for(i = 576; i < 704; i++)
s->output[i] = cpe->ch[channel].saved[i];
}
if(cpe->ch[channel].ics.window_sequence[0] != LONG_START_SEQUENCE){
j = channel;
for (i = 0; i < 1024; i++, j += avctx->channels){
s->output[i+1024] = audio[j] * lwindow[1024 - i - 1];
cpe->ch[channel].saved[i] = audio[j] * lwindow[i];
}
}else{
j = channel;
for(i = 0; i < 448; i++, j += avctx->channels)
s->output[i+1024] = audio[j];
for(i = 448; i < 576; i++, j += avctx->channels)
s->output[i+1024] = audio[j] * swindow[576 - i - 1];
memset(s->output+1024+576, 0, sizeof(s->output[0]) * 448);
j = channel;
for(i = 0; i < 1024; i++, j += avctx->channels)
cpe->ch[channel].saved[i] = audio[j];
}
ff_mdct_calc(&s->mdct1024, cpe->ch[channel].coeffs, s->output, s->tmp);
}else{
j = channel;
for (k = 0; k < 1024; k += 128) {
for(i = 448 + k; i < 448 + k + 256; i++)
s->output[i - 448 - k] = (i < 1024) ? cpe->ch[channel].saved[i] : audio[channel + (i-1024)*avctx->channels] / 512.0;
s->dsp.vector_fmul (s->output, k ? swindow : pwindow, 128);
s->dsp.vector_fmul_reverse(s->output+128, s->output+128, swindow, 128);
ff_mdct_calc(&s->mdct128, cpe->ch[channel].coeffs + k, s->output, s->tmp);
}
j = channel;
for(i = 0; i < 1024; i++, j += avctx->channels)
cpe->ch[channel].saved[i] = audio[j];
}
}
/**
* Encode ics_info element.
* @see Table 4.6 (syntax of ics_info)
*/
static void put_ics_info(AVCodecContext *avctx, IndividualChannelStream *info)
{
AACEncContext *s = avctx->priv_data;
int i;
put_bits(&s->pb, 1, 0); // ics_reserved bit
put_bits(&s->pb, 2, info->window_sequence[0]);
put_bits(&s->pb, 1, info->use_kb_window[0]);
if(info->window_sequence[0] != EIGHT_SHORT_SEQUENCE){
put_bits(&s->pb, 6, info->max_sfb);
put_bits(&s->pb, 1, 0); // no prediction
}else{
put_bits(&s->pb, 4, info->max_sfb);
for(i = 1; i < info->num_windows; i++)
put_bits(&s->pb, 1, info->group_len[i]);
}
}
/**
* Encode MS data.
* @see 4.6.8.1 "Joint Coding - M/S Stereo"
*/
static void encode_ms_info(PutBitContext *pb, ChannelElement *cpe)
{
int i, w;
put_bits(pb, 2, cpe->ms.present);
if(cpe->ms.present == 1)
for(w = 0; w < cpe->ch[0].ics.num_windows; w++){
if(cpe->ch[0].ics.group_len[w]) continue;
for(i = 0; i < cpe->ch[0].ics.max_sfb; i++)
put_bits(pb, 1, cpe->ms.mask[w][i]);
}
}
/**
* Scan scalefactor band and determine optimal codebook for it.
*
* @param s encoder context
* @param cpe channel element
* @param channel channel number inside channel pair
* @param win window group start number
* @param band scalefactor band to analyze
* @param start scalefactor band position in spectral coefficients
* @param size scalefactor band size
*/
static int determine_section_info(AACEncContext *s, ChannelElement *cpe, int channel, int win, int band, int start, int size)
{
int i, j, w;
int maxval, sign;
int score, best, cb, bestcb, dim, idx, start2;
maxval = 0;
sign = 0;
w = win;
start2 = start;
do{
for(i = start2; i < start2 + size; i++){
maxval = FFMAX(maxval, FFABS(cpe->ch[channel].icoefs[i]));
if(cpe->ch[channel].icoefs[i] < 0) sign = 1;
}
w++;
start2 += 128;
}while(w < cpe->ch[channel].ics.num_windows && cpe->ch[channel].ics.group_len[w]);
if(maxval > 12) return 11;
if(!maxval) return 0;
for(cb = 0; cb < 12; cb++)
if(aac_cb_info[cb].maxval >= maxval)
break;
best = INT_MAX;
bestcb = 11;
for(; cb < 12; cb++){
score = 0;
dim = (aac_cb_info[cb].flags & CB_PAIRS) ? 2 : 4;
if(!band || cpe->ch[channel].band_type[win][band - 1] != cb)
score += 9; //that's for new codebook entry
w = win;
start2 = start;
if(aac_cb_info[cb].flags & CB_UNSIGNED){
do{
for(i = start2; i < start2 + size; i += dim){
idx = 0;
for(j = 0; j < dim; j++)
idx = idx * aac_cb_info[cb].maxval + FFABS(cpe->ch[channel].icoefs[i+j]);
score += ff_aac_spectral_bits[aac_cb_info[cb].cb_num][idx];
for(j = 0; j < dim; j++)
if(cpe->ch[channel].icoefs[i+j])
score++;
}
w++;
start2 += 128;
}while(w < cpe->ch[channel].ics.num_windows && cpe->ch[channel].ics.group_len[w]);
}else{
do{
for(i = start2; i < start2 + size; i += dim){
idx = 0;
for(j = 0; j < dim; j++)
idx = idx * (aac_cb_info[cb].maxval*2 + 1) + cpe->ch[channel].icoefs[i+j] + aac_cb_info[cb].maxval;
score += ff_aac_spectral_bits[aac_cb_info[cb].cb_num][idx];
}
w++;
start2 += 128;
}while(w < cpe->ch[channel].ics.num_windows && cpe->ch[channel].ics.group_len[w]);
}
if(score < best){
best = score;
bestcb = cb;
}
}
return bestcb;
}
/**
* Encode one scalefactor band with selected codebook.
*/
static void encode_band_coeffs(AACEncContext *s, ChannelElement *cpe, int channel, int start, int size, int cb)
{
const uint8_t *bits = ff_aac_spectral_bits [aac_cb_info[cb].cb_num];
const uint16_t *codes = ff_aac_spectral_codes[aac_cb_info[cb].cb_num];
const int dim = (aac_cb_info[cb].flags & CB_PAIRS) ? 2 : 4;
int i, j, idx;
if(!bits) return;
if(aac_cb_info[cb].flags & CB_ESCAPE){
for(i = start; i < start + size; i += dim){
idx = 0;
for(j = 0; j < dim; j++)
idx = idx*17 + FFMIN(FFABS(cpe->ch[channel].icoefs[i+j]), 16);
put_bits(&s->pb, bits[idx], codes[idx]);
//output signs
for(j = 0; j < dim; j++)
if(cpe->ch[channel].icoefs[i+j])
put_bits(&s->pb, 1, cpe->ch[channel].icoefs[i+j] < 0);
//output escape values
for(j = 0; j < dim; j++)
if(FFABS(cpe->ch[channel].icoefs[i+j]) > 15){
int l = av_log2(FFABS(cpe->ch[channel].icoefs[i+j]));
put_bits(&s->pb, l - 4 + 1, (1 << (l - 4 + 1)) - 2);
put_bits(&s->pb, l, FFABS(cpe->ch[channel].icoefs[i+j]) & ((1 << l) - 1));
}
}
}else if(aac_cb_info[cb].flags & CB_UNSIGNED){
for(i = start; i < start + size; i += dim){
idx = 0;
for(j = 0; j < dim; j++)
idx = idx * (aac_cb_info[cb].maxval + 1) + FFABS(cpe->ch[channel].icoefs[i+j]);
put_bits(&s->pb, bits[idx], codes[idx]);
//output signs
for(j = 0; j < dim; j++)
if(cpe->ch[channel].icoefs[i+j])
put_bits(&s->pb, 1, cpe->ch[channel].icoefs[i+j] < 0);
}
}else{
for(i = start; i < start + size; i += dim){
idx = 0;
for(j = 0; j < dim; j++)
idx = idx * (aac_cb_info[cb].maxval*2 + 1) + cpe->ch[channel].icoefs[i+j] + aac_cb_info[cb].maxval;
put_bits(&s->pb, bits[idx], codes[idx]);
}
}
}
/**
* Encode scalefactor band coding type.
*/
static void encode_band_info(AVCodecContext *avctx, AACEncContext *s, ChannelElement *cpe, int channel)
{
int i, w;
int bits = cpe->ch[channel].ics.num_windows == 1 ? 5 : 3;
int esc = (1 << bits) - 1;
int count;
for(w = 0; w < cpe->ch[channel].ics.num_windows; w++){
if(cpe->ch[channel].ics.group_len[w]) continue;
count = 0;
for(i = 0; i < cpe->ch[channel].ics.max_sfb; i++){
if(!i || cpe->ch[channel].band_type[w][i] != cpe->ch[channel].band_type[w][i-1]){
if(count){
while(count >= esc){
put_bits(&s->pb, bits, esc);
count -= esc;
}
put_bits(&s->pb, bits, count);
}
put_bits(&s->pb, 4, cpe->ch[channel].band_type[w][i]);
count = 1;
}else
count++;
}
if(count){
while(count >= esc){
put_bits(&s->pb, bits, esc);
count -= esc;
}
put_bits(&s->pb, bits, count);
}
}
}
/**
* Encode scalefactors.
*/
static void encode_scale_factors(AVCodecContext *avctx, AACEncContext *s, ChannelElement *cpe, int channel)
{
int off = cpe->ch[channel].mixing_gain, diff;
int i, w;
for(w = 0; w < cpe->ch[channel].ics.num_windows; w++){
if(cpe->ch[channel].ics.group_len[w]) continue;
for(i = 0; i < cpe->ch[channel].ics.max_sfb; i++){
if(!cpe->ch[channel].zeroes[w][i]){
diff = cpe->ch[channel].sf_idx[w][i] - off + SCALE_DIFF_ZERO;
if(diff < 0 || diff > 120) av_log(avctx, AV_LOG_ERROR, "Scalefactor difference is too big to be coded\n");
off = cpe->ch[channel].sf_idx[w][i];
put_bits(&s->pb, ff_aac_scalefactor_bits[diff], ff_aac_scalefactor_code[diff]);
}
}
}
}
/**
* Encode pulse data.
*/
static void encode_pulses(AVCodecContext *avctx, AACEncContext *s, ChannelElement *cpe, int channel)
{
int i;
put_bits(&s->pb, 1, !!cpe->ch[channel].pulse.num_pulse);
if(!cpe->ch[channel].pulse.num_pulse) return;
put_bits(&s->pb, 2, cpe->ch[channel].pulse.num_pulse - 1);
put_bits(&s->pb, 6, cpe->ch[channel].pulse.start);
for(i = 0; i < cpe->ch[channel].pulse.num_pulse; i++){
put_bits(&s->pb, 5, cpe->ch[channel].pulse.offset[i]);
put_bits(&s->pb, 4, cpe->ch[channel].pulse.amp[i]);
}
}
/**
* Encode temporal noise shaping data.
*/
static void encode_tns_data(AVCodecContext *avctx, AACEncContext *s, ChannelElement *cpe, int channel)
{
int i, w;
put_bits(&s->pb, 1, cpe->ch[channel].tns.present);
if(!cpe->ch[channel].tns.present) return;
if(cpe->ch[channel].ics.window_sequence[0] == EIGHT_SHORT_SEQUENCE){
for(w = 0; w < cpe->ch[channel].ics.num_windows; w++){
put_bits(&s->pb, 1, cpe->ch[channel].tns.n_filt[w]);
if(!cpe->ch[channel].tns.n_filt[w]) continue;
put_bits(&s->pb, 1, cpe->ch[channel].tns.coef_res[w] - 3);
put_bits(&s->pb, 4, cpe->ch[channel].tns.length[w][0]);
put_bits(&s->pb, 3, cpe->ch[channel].tns.order[w][0]);
if(cpe->ch[channel].tns.order[w][0]){
put_bits(&s->pb, 1, cpe->ch[channel].tns.direction[w][0]);
put_bits(&s->pb, 1, cpe->ch[channel].tns.coef_compress[w][0]);
for(i = 0; i < cpe->ch[channel].tns.order[w][0]; i++)
put_bits(&s->pb, cpe->ch[channel].tns.coef_len[w][0], cpe->ch[channel].tns.coef[w][0][i]);
}
}
}else{
put_bits(&s->pb, 1, cpe->ch[channel].tns.n_filt[0]);
if(!cpe->ch[channel].tns.n_filt[0]) return;
put_bits(&s->pb, 1, cpe->ch[channel].tns.coef_res[0] - 3);
for(w = 0; w < cpe->ch[channel].tns.n_filt[0]; w++){
put_bits(&s->pb, 6, cpe->ch[channel].tns.length[0][w]);
put_bits(&s->pb, 5, cpe->ch[channel].tns.order[0][w]);
if(cpe->ch[channel].tns.order[0][w]){
put_bits(&s->pb, 1, cpe->ch[channel].tns.direction[0][w]);
put_bits(&s->pb, 1, cpe->ch[channel].tns.coef_compress[0][w]);
for(i = 0; i < cpe->ch[channel].tns.order[0][w]; i++)
put_bits(&s->pb, cpe->ch[channel].tns.coef_len[0][w], cpe->ch[channel].tns.coef[0][w][i]);
}
}
}
}
/**
* Encode spectral coefficients processed by psychoacoustic model.
*/
static void encode_spectral_coeffs(AVCodecContext *avctx, AACEncContext *s, ChannelElement *cpe, int channel)
{
int start, i, w, w2;
for(w = 0; w < cpe->ch[channel].ics.num_windows; w++){
if(cpe->ch[channel].ics.group_len[w]) continue;
start = 0;
for(i = 0; i < cpe->ch[channel].ics.max_sfb; i++){
if(cpe->ch[channel].zeroes[w][i]){
start += cpe->ch[channel].ics.swb_sizes[i];
continue;
}
w2 = w;
do{
encode_band_coeffs(s, cpe, channel, start + w2*128, cpe->ch[channel].ics.swb_sizes[i], cpe->ch[channel].band_type[w][i]);
w2++;
}while(w2 < cpe->ch[channel].ics.num_windows && cpe->ch[channel].ics.group_len[w2]);
start += cpe->ch[channel].ics.swb_sizes[i];
}
}
}
/**
* Encode one channel of audio data.
*/
static int encode_individual_channel(AVCodecContext *avctx, ChannelElement *cpe, int channel)
{
AACEncContext *s = avctx->priv_data;
int i, g, w;
for(w = 0; w < cpe->ch[channel].ics.num_windows; w++){
i = w << 7;
if(cpe->ch[channel].ics.group_len[w]) continue;
for(g = 0; g < cpe->ch[channel].ics.max_sfb; g++){
if(!cpe->ch[channel].zeroes[w][g]){
cpe->ch[channel].band_type[w][g] = determine_section_info(s, cpe, channel, w, g, i, cpe->ch[channel].ics.swb_sizes[g]);
cpe->ch[channel].zeroes[w][g] = !cpe->ch[channel].band_type[w][g];
}else
cpe->ch[channel].band_type[w][g] = 0;
i += cpe->ch[channel].ics.swb_sizes[g];
}
}
put_bits(&s->pb, 8, cpe->ch[channel].mixing_gain); //global gain
if(!cpe->common_window) put_ics_info(avctx, &cpe->ch[channel].ics);
encode_band_info(avctx, s, cpe, channel);
encode_scale_factors(avctx, s, cpe, channel);
encode_pulses(avctx, s, cpe, channel);
encode_tns_data(avctx, s, cpe, channel);
put_bits(&s->pb, 1, 0); //ssr
encode_spectral_coeffs(avctx, s, cpe, channel);
return 0;
}
/**
* Write some auxiliary information about the created AAC file.
*/
static void put_bitstream_info(AVCodecContext *avctx, AACEncContext *s, const char *name)
{
int i, namelen, padbits;
namelen = strlen(name) + 2;
put_bits(&s->pb, 3, ID_FIL);
put_bits(&s->pb, 4, FFMIN(namelen, 15));
if(namelen >= 15)
put_bits(&s->pb, 8, namelen - 16);
put_bits(&s->pb, 4, 0); //extension type - filler
padbits = 8 - (put_bits_count(&s->pb) & 7);
align_put_bits(&s->pb);
for(i = 0; i < namelen - 2; i++)
put_bits(&s->pb, 8, name[i]);
put_bits(&s->pb, 12 - padbits, 0);
}
static int aac_encode_frame(AVCodecContext *avctx,
uint8_t *frame, int buf_size, void *data)
{
AACEncContext *s = avctx->priv_data;
int16_t *samples = s->samples, *samples2, *la;
ChannelElement *cpe;
int i, j, chans, tag, start_ch;
const uint8_t *chan_map = aac_chan_configs[avctx->channels-1];
int chan_el_counter[4];
if(s->last_frame)
return 0;
if(data){
if((s->psy.flags & PSY_MODEL_NO_PREPROC) == PSY_MODEL_NO_PREPROC){
memcpy(s->samples + 1024 * avctx->channels, data, 1024 * avctx->channels * sizeof(s->samples[0]));
}else{
start_ch = 0;
samples2 = s->samples + 1024 * avctx->channels;
for(i = 0; i < chan_map[0]; i++){
tag = chan_map[i+1];
chans = tag == ID_CPE ? 2 : 1;
ff_aac_psy_preprocess(&s->psy, (uint16_t*)data + start_ch, samples2 + start_ch, i, tag);
start_ch += chans;
}
}
}
if(!avctx->frame_number){
memmove(s->samples, s->samples + 1024 * avctx->channels, 1024 * avctx->channels * sizeof(s->samples[0]));
return 0;
}
init_put_bits(&s->pb, frame, buf_size*8);
if(avctx->frame_number==1 && !(avctx->flags & CODEC_FLAG_BITEXACT)){
put_bitstream_info(avctx, s, LIBAVCODEC_IDENT);
}
start_ch = 0;
memset(chan_el_counter, 0, sizeof(chan_el_counter));
for(i = 0; i < chan_map[0]; i++){
tag = chan_map[i+1];
chans = tag == ID_CPE ? 2 : 1;
cpe = &s->cpe[i];
samples2 = samples + start_ch;
la = samples2 + 1024 * avctx->channels + start_ch;
if(!data) la = NULL;
ff_aac_psy_suggest_window(&s->psy, samples2, la, i, tag, cpe);
for(j = 0; j < chans; j++){
apply_window_and_mdct(avctx, s, cpe, samples2, j);
}
ff_aac_psy_analyze(&s->psy, i, tag, cpe);
put_bits(&s->pb, 3, tag);
put_bits(&s->pb, 4, chan_el_counter[tag]++);
if(chans == 2){
put_bits(&s->pb, 1, cpe->common_window);
if(cpe->common_window){
put_ics_info(avctx, &cpe->ch[0].ics);
encode_ms_info(&s->pb, cpe);
}
}
for(j = 0; j < chans; j++){
encode_individual_channel(avctx, cpe, j);
}
start_ch += chans;
}
put_bits(&s->pb, 3, ID_END);
flush_put_bits(&s->pb);
avctx->frame_bits = put_bits_count(&s->pb);
if(!data)
s->last_frame = 1;
memmove(s->samples, s->samples + 1024 * avctx->channels, 1024 * avctx->channels * sizeof(s->samples[0]));
return put_bits_count(&s->pb)>>3;
}
static av_cold int aac_encode_end(AVCodecContext *avctx)
{
AACEncContext *s = avctx->priv_data;
ff_mdct_end(&s->mdct1024);
ff_mdct_end(&s->mdct128);
ff_aac_psy_end(&s->psy);
av_freep(&s->samples);
av_freep(&s->cpe);
return 0;
}
AVCodec aac_encoder = {
"aac",
CODEC_TYPE_AUDIO,
CODEC_ID_AAC,
sizeof(AACEncContext),
aac_encode_init,
aac_encode_frame,
aac_encode_end,
.capabilities = CODEC_CAP_SMALL_LAST_FRAME | CODEC_CAP_DELAY,
.sample_fmts = (enum SampleFormat[]){SAMPLE_FMT_S16,SAMPLE_FMT_NONE},
.long_name = NULL_IF_CONFIG_SMALL("Advanced Audio Coding"),
};
-------------- next part --------------
/*
* AAC encoder psychoacoustic model
* Copyright (C) 2008 Konstantin Shishkov
*
* This file is part of FFmpeg.
*
* FFmpeg is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* FFmpeg is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with FFmpeg; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
#ifndef FFMPEG_AACPSY_H
#define FFMPEG_AACPSY_H
#include "avcodec.h"
#include "aac.h"
#include "lowpass.h"
enum AACPsyModelType{
AAC_PSY_NULL, ///< do nothing with frequencies
AAC_PSY_NULL8, ///< do nothing with frequencies but work with short windows
AAC_PSY_3GPP, ///< model following recommendations from 3GPP TS 26.403
AAC_NB_PSY_MODELS ///< total number of psychoacoustic models, since it's not a part of the ABI new models can be added freely
};
enum AACPsyModelMode{
PSY_MODE_CBR, ///< follow bitrate as closely as possible
PSY_MODE_ABR, ///< try to achieve bitrate but actual bitrate may differ significantly
PSY_MODE_QUALITY, ///< try to achieve set quality instead of bitrate
};
#define PSY_MODEL_MODE_MASK 0x0000000F ///< bit fields for storing mode (CBR, ABR, VBR)
#define PSY_MODEL_NO_PULSE 0x00000010 ///< disable pulse searching
#define PSY_MODEL_NO_SWITCH 0x00000020 ///< disable window switching
#define PSY_MODEL_NO_ST_ATT 0x00000040 ///< disable stereo attenuation
#define PSY_MODEL_NO_LOWPASS 0x00000080 ///< disable low-pass filtering
#define PSY_MODEL_NO_PREPROC (PSY_MODEL_NO_ST_ATT | PSY_MODEL_NO_LOWPASS)
#define PSY_MODEL_MODE(a) ((a) & PSY_MODEL_MODE_MASK)
/**
* context used by psychoacoustic model
*/
typedef struct AACPsyContext {
AVCodecContext *avctx; ///< encoder context
int flags; ///< model flags
const uint8_t *bands1024; ///< scalefactor band sizes for long (1024 samples) frame
int num_bands1024; ///< number of scalefactor bands for long frame
const uint8_t *bands128; ///< scalefactor band sizes for short (128 samples) frame
int num_bands128; ///< number of scalefactor bands for short frame
const struct AACPsyModel *model; ///< pointer to the psychoacoustic model implementation
void* model_priv_data; ///< psychoacoustic model implementation private data
float stereo_att; ///< stereo attenuation factor
LPFilterCoeffs lp_coeffs; ///< lowpass filter coefficients
LPFilterState *lp_state; ///< lowpass filter state
}AACPsyContext;
typedef struct AACPsyModel {
const char *name;
int (*init) (AACPsyContext *apc, int elements);
void (*window) (AACPsyContext *apc, int16_t *audio, int16_t *la, int tag, int type, ChannelElement *cpe);
void (*process)(AACPsyContext *apc, int tag, int type, ChannelElement *cpe);
void (*end) (AACPsyContext *apc);
}AACPsyModel;
/**
* Initialize psychoacoustic model.
*
* @param ctx model context
* @param avctx codec context
* @param model model implementation that will be used
* @param elements number of channel elements (single channel or channel pair) to handle by model
* @param flags model flags, may be ignored by model if unsupported
* @param bands1024 scalefactor band lengths for long (1024 samples) frame
* @param num_bands1024 number of scalefactor bands for long frame
* @param bands128 scalefactor band lengths for short (128 samples) frame
* @param num_bands128 number of scalefactor bands for short frame
*
* @return zero if successful, a negative value if not
*/
int ff_aac_psy_init(AACPsyContext *ctx, AVCodecContext *avctx,
enum AACPsyModelType model, int elements, int flags,
const uint8_t *bands1024, int num_bands1024,
const uint8_t *bands128, int num_bands128);
/**
* Preprocess audio frame in order to compress it better.
*
* @param ctx model context
* @param audio samples to preprocess
* @param dest place to put filtered samples
* @param tag number of channel element to analyze
* @param type channel element type (e.g. ID_SCE or ID_CPE)
*/
void ff_aac_psy_preprocess(AACPsyContext *ctx, int16_t *audio, int16_t *dest, int tag, int type);
/**
* Set window sequence and related parameters for channel element.
*
* @param ctx model context
* @param audio samples for the current frame
* @param la lookahead samples (NULL when unavailable)
* @param tag number of channel element to analyze
* @param type channel element type (e.g. ID_SCE or ID_CPE)
* @param cpe pointer to the current channel element
*/
void ff_aac_psy_suggest_window(AACPsyContext *ctx, int16_t *audio, int16_t *la, int tag, int type, ChannelElement *cpe);
/**
* Perform psychoacoustic analysis and output coefficients in integer form
* along with scalefactors, M/S flags, etc.
*
* @param ctx model context
* @param tag number of channel element to analyze
* @param type channel element type (e.g. ID_SCE or ID_CPE)
* @param cpe pointer to the current channel element
*/
void ff_aac_psy_analyze(AACPsyContext *ctx, int tag, int type, ChannelElement *cpe);
/**
* Cleanup model context at the end.
*
* @param ctx model context
*/
void ff_aac_psy_end(AACPsyContext *ctx);
#endif /* FFMPEG_AACPSY_H */
-------------- next part --------------
/*
* AAC encoder psychoacoustic model
* Copyright (C) 2008 Konstantin Shishkov
*
* This file is part of FFmpeg.
*
* FFmpeg is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* FFmpeg is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with FFmpeg; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
/**
* @file aacpsy.c
* AAC encoder psychoacoustic model
*/
#include "avcodec.h"
#include "aacpsy.h"
#include "aactab.h"
/***********************************
* TODOs:
* General:
* better audio preprocessing (add DC highpass filter?)
* more psy models
**********************************/
/**
* Convert coefficients to integers.
* @return sum of coefficients
* @see 3GPP TS26.403 5.6.2 "Scalefactor determination"
*/
static inline int convert_coeffs(float *in, int *out, int size, int scale_idx)
{
int i, sign, sum = 0;
for(i = 0; i < size; i++){
sign = in[i] > 0.0;
out[i] = (int)(pow(FFABS(in[i]) * ff_aac_pow2sf_tab[200 - scale_idx + SCALE_ONE_POS - SCALE_DIV_512], 0.75) + 0.4054);
out[i] = av_clip(out[i], 0, 8191);
sum += out[i];
if(sign) out[i] = -out[i];
}
return sum;
}
static inline float unquant(int q, int scale_idx){
return (FFABS(q) * cbrt(q*1.0)) * ff_aac_pow2sf_tab[200 + scale_idx - SCALE_ONE_POS];
}
static inline float calc_distortion(float *c, int size, int scale_idx)
{
int i;
int q;
float coef, unquant, sum = 0.0f;
for(i = 0; i < size; i++){
coef = FFABS(c[i]);
q = (int)(pow(FFABS(coef) * ff_aac_pow2sf_tab[200 - scale_idx + SCALE_ONE_POS - SCALE_DIV_512], 0.75) + 0.4054);
q = av_clip(q, 0, 8191);
unquant = (q * cbrt(q)) * ff_aac_pow2sf_tab[200 + scale_idx - SCALE_ONE_POS + SCALE_DIV_512];
sum += (coef - unquant) * (coef - unquant);
}
return sum;
}
/**
* Produce integer coefficients from scalefactors provided by the model.
*/
static void psy_create_output(AACPsyContext *apc, ChannelElement *cpe, int chans, int search_pulses)
{
int i, w, w2, g, ch;
int start, sum, maxsfb, cmaxsfb;
int pulses, poff[4], pamp[4];
for(ch = 0; ch < chans; ch++){
start = 0;
maxsfb = 0;
cpe->ch[ch].pulse.num_pulse = 0;
for(w = 0; w < cpe->ch[ch].ics.num_windows; w++){
for(g = 0; g < cpe->ch[ch].ics.num_swb; g++){
sum = 0;
//apply M/S
if(!ch && cpe->ms.mask[w][g]){
for(i = 0; i < cpe->ch[ch].ics.swb_sizes[g]; i++){
cpe->ch[0].coeffs[start+i] = (cpe->ch[0].coeffs[start+i] + cpe->ch[1].coeffs[start+i]) / 2.0;
cpe->ch[1].coeffs[start+i] = cpe->ch[0].coeffs[start+i] - cpe->ch[1].coeffs[start+i];
}
}
if(!cpe->ch[ch].zeroes[w][g])
sum = convert_coeffs(cpe->ch[ch].coeffs + start, cpe->ch[ch].icoefs + start, cpe->ch[ch].ics.swb_sizes[g], cpe->ch[ch].sf_idx[w][g]);
else
memset(cpe->ch[ch].icoefs + start, 0, cpe->ch[ch].ics.swb_sizes[g] * sizeof(cpe->ch[0].icoefs[0]));
cpe->ch[ch].zeroes[w][g] = !sum;
//try finding pulses
if(search_pulses && cpe->ch[ch].ics.num_windows == 1 && !cpe->ch[ch].pulse.num_pulse){
pulses = 0;
memset(poff,0,sizeof(poff));
memset(pamp,0,sizeof(pamp));
for(i = 0; i < cpe->ch[ch].ics.swb_sizes[g]; i++){
if(pulses > 4 || (pulses && i > cpe->ch[ch].pulse.offset[pulses-1] - 31)) break;
if(FFABS(cpe->ch[ch].icoefs[start+i]) > 4 && pulses < 4){
poff[pulses] = i;
pamp[pulses] = FFMIN(FFABS(cpe->ch[ch].icoefs[start+i]) - 1, 15);
pulses++;
}
}
if(pulses){
cpe->ch[ch].pulse.start = g;
cpe->ch[ch].pulse.num_pulse = pulses;
for(i = 0; i < pulses; i++){
cpe->ch[ch].pulse.amp[i] = pamp[i];
cpe->ch[ch].pulse.offset[i] = i ? poff[i] - poff[i-1] : poff[0];
if(cpe->ch[ch].icoefs[start+poff[i]] > 0)
cpe->ch[ch].icoefs[start+poff[i]] -= pamp[i];
else
cpe->ch[ch].icoefs[start+poff[i]] += pamp[i];
}
}
}
start += cpe->ch[ch].ics.swb_sizes[g];
}
for(cmaxsfb = cpe->ch[ch].ics.num_swb; cmaxsfb > 0 && cpe->ch[ch].zeroes[w][cmaxsfb-1]; cmaxsfb--);
maxsfb = FFMAX(maxsfb, cmaxsfb);
}
cpe->ch[ch].ics.max_sfb = maxsfb;
//adjust zero bands for window groups
for(w = 0; w < cpe->ch[ch].ics.num_windows; w++){
if(cpe->ch[ch].ics.group_len[w]) continue;
for(g = 0; g < cpe->ch[ch].ics.max_sfb; g++){
i = 1;
w2 = w;
do{
if(!cpe->ch[ch].zeroes[w2][g]){
i = 0;
break;
}
w2++;
}while(w2 < cpe->ch[ch].ics.num_windows && cpe->ch[ch].ics.group_len[w2]);
cpe->ch[ch].zeroes[w][g] = i;
}
}
}
if(chans > 1 && cpe->common_window){
int msc = 0;
cpe->ch[0].ics.max_sfb = FFMAX(cpe->ch[0].ics.max_sfb, cpe->ch[1].ics.max_sfb);
cpe->ch[1].ics.max_sfb = cpe->ch[0].ics.max_sfb;
for(w = 0; w < cpe->ch[0].ics.num_windows; w++)
for(i = 0; i < cpe->ch[0].ics.max_sfb; i++)
if(cpe->ms.mask[w][i]) msc++;
if(msc == 0 || cpe->ch[0].ics.max_sfb == 0) cpe->ms.present = 0;
else cpe->ms.present = msc < cpe->ch[0].ics.max_sfb ? 1 : 2;
}
}
static void psy_null_window(AACPsyContext *apc, int16_t *audio, int16_t *la, int tag, int type, ChannelElement *cpe)
{
int ch;
int chans = type == ID_CPE ? 2 : 1;
for(ch = 0; ch < chans; ch++){
cpe->ch[ch].ics.window_sequence[0] = ONLY_LONG_SEQUENCE;
cpe->ch[ch].ics.use_kb_window[0] = 1;
cpe->ch[ch].ics.num_windows = 1;
cpe->ch[ch].ics.swb_sizes = apc->bands1024;
cpe->ch[ch].ics.num_swb = apc->num_bands1024;
cpe->ch[ch].ics.group_len[0] = 0;
}
cpe->common_window = cpe->ch[0].ics.use_kb_window[0] == cpe->ch[1].ics.use_kb_window[0];
}
static void psy_null_process(AACPsyContext *apc, int tag, int type, ChannelElement *cpe)
{
int start;
int ch, g, i;
int minscale;
int chans = type == ID_CPE ? 2 : 1;
for(ch = 0; ch < chans; ch++){
start = 0;
for(g = 0; g < apc->num_bands1024; g++){
float energy = 0.0f, ffac = 0.0f, thr, dist;
for(i = 0; i < apc->bands1024[g]; i++){
energy += cpe->ch[ch].coeffs[start+i]*cpe->ch[ch].coeffs[start+i];
ffac += sqrt(FFABS(cpe->ch[ch].coeffs[start+i]));
}
thr = energy * 0.001258925f;
cpe->ch[ch].sf_idx[ch][g] = 136;
cpe->ch[ch].zeroes[ch][g] = (energy == 0.0);
if(cpe->ch[ch].zeroes[ch][g]) continue;
minscale = (int)(2.66667 * (log2(6.75*thr) - log2(ffac)));
cpe->ch[ch].sf_idx[ch][g] = SCALE_ONE_POS - minscale;
while(cpe->ch[ch].sf_idx[ch][g] > 3){
dist = calc_distortion(cpe->ch[ch].coeffs + start, apc->bands1024[g], cpe->ch[ch].sf_idx[ch][g]);
if(dist < thr) break;
cpe->ch[ch].sf_idx[ch][g] -= 3;
}
}
}
for(ch = 0; ch < chans; ch++){
minscale = 255;
for(g = 0; g < apc->num_bands1024; g++)
if(!cpe->ch[ch].zeroes[0][g])
minscale = FFMIN(minscale, cpe->ch[ch].sf_idx[0][g]);
cpe->ch[ch].mixing_gain = minscale;
for(g = 0; g < apc->num_bands1024; g++)
if(!cpe->ch[ch].zeroes[0][g])
cpe->ch[ch].sf_idx[0][g] = FFMIN(minscale + SCALE_MAX_DIFF, cpe->ch[ch].sf_idx[0][g]);
}
psy_create_output(apc, cpe, chans, 1);
}
static void psy_null8_window(AACPsyContext *apc, int16_t *audio, int16_t *la, int tag, int type, ChannelElement *cpe)
{
int ch, i;
int chans = type == ID_CPE ? 2 : 1;
for(ch = 0; ch < chans; ch++){
int prev_seq = cpe->ch[ch].ics.window_sequence[1];
cpe->ch[ch].ics.use_kb_window[1] = cpe->ch[ch].ics.use_kb_window[0];
cpe->ch[ch].ics.window_sequence[1] = cpe->ch[ch].ics.window_sequence[0];
switch(cpe->ch[ch].ics.window_sequence[0]){
case ONLY_LONG_SEQUENCE: if(prev_seq == ONLY_LONG_SEQUENCE)cpe->ch[ch].ics.window_sequence[0] = LONG_START_SEQUENCE; break;
case LONG_START_SEQUENCE: cpe->ch[ch].ics.window_sequence[0] = EIGHT_SHORT_SEQUENCE; break;
case EIGHT_SHORT_SEQUENCE: if(prev_seq == EIGHT_SHORT_SEQUENCE)cpe->ch[ch].ics.window_sequence[0] = LONG_STOP_SEQUENCE; break;
case LONG_STOP_SEQUENCE: cpe->ch[ch].ics.window_sequence[0] = ONLY_LONG_SEQUENCE; break;
}
if(cpe->ch[ch].ics.window_sequence[0] != EIGHT_SHORT_SEQUENCE){
cpe->ch[ch].ics.use_kb_window[0] = 1;
cpe->ch[ch].ics.num_windows = 1;
cpe->ch[ch].ics.swb_sizes = apc->bands1024;
cpe->ch[ch].ics.num_swb = apc->num_bands1024;
cpe->ch[ch].ics.group_len[0] = 0;
}else{
cpe->ch[ch].ics.use_kb_window[0] = 1;
cpe->ch[ch].ics.num_windows = 8;
cpe->ch[ch].ics.swb_sizes = apc->bands128;
cpe->ch[ch].ics.num_swb = apc->num_bands128;
for(i = 0; i < cpe->ch[ch].ics.num_windows; i++)
cpe->ch[ch].ics.group_len[i] = i & 1;
}
}
cpe->common_window = cpe->ch[0].ics.use_kb_window[0] == cpe->ch[1].ics.use_kb_window[0];
}
static void psy_null8_process(AACPsyContext *apc, int tag, int type, ChannelElement *cpe)
{
int start;
int w, ch, g, i;
int chans = type == ID_CPE ? 2 : 1;
//detect M/S
if(chans > 1 && cpe->common_window){
start = 0;
for(w = 0; w < cpe->ch[0].ics.num_windows; w++){
for(g = 0; g < cpe->ch[0].ics.num_swb; g++){
float diff = 0.0f;
for(i = 0; i < cpe->ch[0].ics.swb_sizes[g]; i++)
diff += fabs(cpe->ch[0].coeffs[start+i] - cpe->ch[1].coeffs[start+i]);
cpe->ms.mask[w][g] = diff == 0.0;
}
}
}
for(ch = 0; ch < chans; ch++){
cpe->ch[ch].mixing_gain = SCALE_ONE_POS;
for(w = 0; w < cpe->ch[ch].ics.num_windows; w++){
for(g = 0; g < cpe->ch[ch].ics.num_swb; g++){
cpe->ch[ch].sf_idx[w][g] = SCALE_ONE_POS;
cpe->ch[ch].zeroes[w][g] = 0;
}
}
}
psy_create_output(apc, cpe, chans, 0);
}
static const AACPsyModel psy_models[AAC_NB_PSY_MODELS] =
{
{
"Null model",
NULL,
psy_null_window,
psy_null_process,
NULL,
},
{
"Null model - short windows",
NULL,
psy_null8_window,
psy_null8_process,
NULL,
},
};
int av_cold ff_aac_psy_init(AACPsyContext *ctx, AVCodecContext *avctx,
enum AACPsyModelType model, int elements, int flags,
const uint8_t *bands1024, int num_bands1024,
const uint8_t *bands128, int num_bands128)
{
int i;
if(model >= AAC_NB_PSY_MODELS || !psy_models[model].window || !psy_models[model].process){
av_log(avctx, AV_LOG_ERROR, "Invalid psy model\n");
return -1;
}
#ifndef CONFIG_HARDCODED_TABLES
for (i = 0; i < 316; i++)
ff_aac_pow2sf_tab[i] = pow(2, (i - 200)/4.);
#endif /* CONFIG_HARDCODED_TABLES */
ctx->avctx = avctx;
ctx->flags = flags;
ctx->bands1024 = bands1024;
ctx->num_bands1024 = num_bands1024;
ctx->bands128 = bands128;
ctx->num_bands128 = num_bands128;
ctx->model = &psy_models[model];
if(ctx->flags & PSY_MODEL_NO_ST_ATT || PSY_MODEL_MODE(ctx->flags) == PSY_MODE_QUALITY){
ctx->flags |= PSY_MODEL_NO_ST_ATT;
ctx->stereo_att = 0.5f;
}else{
ctx->stereo_att = av_clipf(avctx->bit_rate / elements / 192000.0, 0.0f, 0.5f);
}
if(ctx->flags & PSY_MODEL_NO_LOWPASS || PSY_MODEL_MODE(ctx->flags) == PSY_MODE_QUALITY){
ctx->flags |= PSY_MODEL_NO_LOWPASS;
}else{
int cutoff;
cutoff = avctx->bit_rate / elements / 8;
if(ff_lowpass_filter_init_coeffs(&ctx->lp_coeffs, avctx->sample_rate/2, cutoff) < 0){
ctx->flags |= PSY_MODEL_NO_LOWPASS;
}else{
ctx->lp_state = av_mallocz(sizeof(LPFilterState) * elements * 2);
}
}
if(ctx->model->init)
return ctx->model->init(ctx, elements);
return 0;
}
void ff_aac_psy_suggest_window(AACPsyContext *ctx, int16_t *audio, int16_t *la, int tag, int type, ChannelElement *cpe)
{
ctx->model->window(ctx, audio, la, tag, type, cpe);
}
void ff_aac_psy_analyze(AACPsyContext *ctx, int tag, int type, ChannelElement *cpe)
{
ctx->model->process(ctx, tag, type, cpe);
}
void av_cold ff_aac_psy_end(AACPsyContext *ctx)
{
av_freep(&ctx->lp_state);
if(ctx->model->end)
return ctx->model->end(ctx);
}
void ff_aac_psy_preprocess(AACPsyContext *ctx, int16_t *audio, int16_t *dest, int tag, int type)
{
int chans = type == ID_CPE ? 2 : 1;
const int chstride = ctx->avctx->channels;
int i, ch;
float t[2];
if(chans == 1){
for(ch = 0; ch < chans; ch++){
for(i = 0; i < 1024; i++){
dest[i * chstride + ch] = audio[i * chstride + ch];
}
}
}else{
for(i = 0; i < 1024; i++){
if(ctx->flags & PSY_MODEL_NO_ST_ATT){
for(ch = 0; ch < 2; ch++)
t[ch] = audio[i * chstride + ch];
}else{
t[0] = audio[i * chstride + 0] * (0.5 + ctx->stereo_att) + audio[i * chstride + 1] * (0.5 - ctx->stereo_att);
t[1] = audio[i * chstride + 0] * (0.5 - ctx->stereo_att) + audio[i * chstride + 1] * (0.5 + ctx->stereo_att);
}
if(!(ctx->flags & PSY_MODEL_NO_LOWPASS)){
LPFilterState *is = (LPFilterState*)ctx->lp_state + tag*2;
for(ch = 0; ch < 2; ch++)
t[ch] = ff_lowpass_filter(&ctx->lp_coeffs, is + ch, t[ch]);
}
for(ch = 0; ch < 2; ch++)
dest[i * chstride + ch] = av_clip_int16(t[ch]);
}
}
}
More information about the ffmpeg-devel
mailing list