[FFmpeg-devel] [RFC v2 3/3] daaladec: Implement a native Daala decoder
Ronald S. Bultje
rsbultje at gmail.com
Tue Dec 29 19:46:56 CET 2015
On Tue, Dec 29, 2015 at 11:49 AM, Andreas Cadhalpun <
andreas.cadhalpun at googlemail.com> wrote:
> On 29.12.2015 17:32, Ronald S. Bultje wrote:
> > On Tue, Dec 29, 2015 at 11:29 AM, Andreas Cadhalpun <
> > andreas.cadhalpun at googlemail.com> wrote:
> >> On 29.12.2015 17:15, Ronald S. Bultje wrote:
> >>> In that case, you likely need 64bit integers for 15bit math precision
> >> (look at
> >>> vp9 code), or daala needs to reduce precision (as does hevc).
> >> Yes, either the intermediate calculation needs to happen with 64bit
> >> integers,
> >> or the input has to be sanitized in some way.
> > It depends what the purpose and source was. Was this real input, or
> > or what?
> These were fuzzed samples.
> > vp9 decoder can certainly overflow with garbage input and that is
> > specifically defined so in libvpx. "Only input generated from a real
> > is considered sane and has a defined outcome.
> Do you have a sample causing overflows in the vp9 decoder?
Nope, but I'm going to assume they're not hard to create, just use a few
high same-sign quantized coefficients (e.g. SHORT_MAX) and a high quantizer
(qidx=255, dq_ac=1828). Both dequant into int16_t (1828*SHORT_MAX doesn't
fit in 16 bits) as well as the idct results themselves (because you're
adding and subtracting from an already near-edge value which should fit in
16bits) will overflow.
> (Overflows in dsp code are typically not a security concern.)
> Well, the overflows in the imdct calculation of the aac_fixed decoder
> caused crashes.
I would prefer if for video codecs, unless specifically proven otherwise
for that codec with specific content, we assume by default that overflows
in performance-sensitive dsp code (specifically idct) are OK. ubsan may not
like us on fuzzed content, but so be it.
More information about the ffmpeg-devel