[FFmpeg-devel] [RFC] avcodec/avcodec.h: Add encryption info side data

wm4 nfxjfg at googlemail.com
Tue Dec 19 01:17:11 EET 2017


On Tue, 19 Dec 2017 00:00:15 +0100
Michael Niedermayer <michael at niedermayer.cc> wrote:

> On Mon, Dec 18, 2017 at 10:28:14PM +0100, wm4 wrote:
> > On Mon, 18 Dec 2017 22:17:14 +0100
> > Michael Niedermayer <michael at niedermayer.cc> wrote:
> >   
> > > On Mon, Dec 18, 2017 at 10:02:52PM +0100, wm4 wrote:  
> > > > On Mon, 18 Dec 2017 21:38:24 +0100
> > > > Michael Niedermayer <michael at niedermayer.cc> wrote:
> > > >     
> > > > > On Mon, Dec 18, 2017 at 04:56:08PM -0300, James Almer wrote:    
> > > > > > On 12/18/2017 4:52 PM, wm4 wrote:      
> > > > > > > On Fri, 15 Dec 2017 14:24:17 -0800
> > > > > > > Jacob Trimble <modmaker-at-google.com at ffmpeg.org> wrote:
> > > > > > >       
> > > > > > >> From a1b2cbcb7da4da69685f8f1299b70b672ce448e3 Mon Sep 17 00:00:00 2001
> > > > > > >> From: Jacob Trimble <modmaker at google.com>
> > > > > > >> Date: Tue, 5 Dec 2017 14:52:22 -0800
> > > > > > >> Subject: [PATCH] avcodec/avcodec.h: Add encryption info side data.
> > > > > > >>
> > > > > > >> This new side-data will contain info on how a packet is encrypted.
> > > > > > >> This allows the app to handle packet decryption.  To allow for a
> > > > > > >> variable number of subsamples, the buffer for the side-data will be
> > > > > > >> allocated to hold both the structure and the array of subsamples.  So
> > > > > > >> the |subsamples| member will point to right after the struct.
> > > > > > >>
> > > > > > >> Signed-off-by: Jacob Trimble <modmaker at google.com>
> > > > > > >> ---
> > > > > > >>  libavcodec/avcodec.h | 70 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > > > > >>  1 file changed, 70 insertions(+)
> > > > > > >>
> > > > > > >> diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
> > > > > > >> index 5db6a81320..ccc89345e8 100644
> > > > > > >> --- a/libavcodec/avcodec.h
> > > > > > >> +++ b/libavcodec/avcodec.h
> > > > > > >> @@ -1112,6 +1112,63 @@ typedef struct AVCPBProperties {
> > > > > > >>      uint64_t vbv_delay;
> > > > > > >>  } AVCPBProperties;
> > > > > > >>  
> > > > > > >> +typedef struct AVPacketSubsampleEncryptionInfo {
> > > > > > >> +    /** The number of bytes that are clear. */
> > > > > > >> +    unsigned int bytes_of_clear_data;
> > > > > > >> +
> > > > > > >> +    /**
> > > > > > >> +     * The number of bytes that are protected.  If using pattern encryption,
> > > > > > >> +     * the pattern applies to only the protected bytes; if not using pattern
> > > > > > >> +     * encryption, all these bytes are encrypted.
> > > > > > >> +     */
> > > > > > >> +    unsigned int bytes_of_protected_data;
> > > > > > >> +} AVPacketSubsampleEncryptionInfo;
> > > > > > >> +
> > > > > > >> +/**
> > > > > > >> + * This describes encryption info for a packet.  This contains frame-specific
> > > > > > >> + * info for how to decrypt the packet before passing it to the decoder.  If this
> > > > > > >> + * side-data is present, then the packet is encrypted.
> > > > > > >> + */
> > > > > > >> +typedef struct AVPacketEncryptionInfo {
> > > > > > >> +    /** The fourcc encryption scheme. */
> > > > > > >> +    uint32_t scheme;
> > > > > > >> +
> > > > > > >> +    /** The ID of the key used to encrypt the packet. */
> > > > > > >> +    uint8_t key_id[16];
> > > > > > >> +
> > > > > > >> +    /** The initialization vector. */
> > > > > > >> +    uint8_t iv[16];
> > > > > > >> +
> > > > > > >> +    /**
> > > > > > >> +     * Only used for pattern encryption.  This is the number of 16-byte blocks
> > > > > > >> +     * that are encrypted.
> > > > > > >> +     */
> > > > > > >> +    unsigned int crypt_byte_block;
> > > > > > >> +
> > > > > > >> +    /**
> > > > > > >> +     * Only used for pattern encryption.  This is the number of 16-byte blocks
> > > > > > >> +     * that are clear.
> > > > > > >> +     */
> > > > > > >> +    unsigned int skip_byte_block;
> > > > > > >> +
> > > > > > >> +    /**
> > > > > > >> +     * The number of sub-samples in this packet.  If 0, then the whole sample
> > > > > > >> +     * is encrypted.
> > > > > > >> +     */
> > > > > > >> +    unsigned int subsample_count;
> > > > > > >> +
> > > > > > >> +    /** The subsample encryption info. */
> > > > > > >> +    AVPacketSubsampleEncryptionInfo *subsamples;      
> > > > > > > 
> > > > > > > I don't think this is sane. So far, side data could simply be copied
> > > > > > > with memcpy, and having pointers to non-static data in side data would
> > > > > > > break this completely.      
> > > > > > 
> > > > > > Even more reasons to ditch the current side data API and come up with a
> > > > > > better designed one that can also be reused for packet, frame and
> > > > > > container needs.      
> > > > > 
> > > > > yes, 
> > > > > also if its redesigned, it should become possible to pass it over the
> > > > > network. Currently all side data depends on the platform ABI, so its
> > > > > only valid on the platform its created on. That also makes it different
> > > > > from all other data, AVPacket.data and extradata are all
> > > > > platform independant. And there is no API to
> > > > > convert it into a platform ABI independant form.     
> > > > 
> > > > Why should it be transferable over network? That makes no sense. We
> > > > don't have the ability to transfer AVFrame, AVPacket, or AVCodecContext
> > > > over network either.    
> > > 
> > > If you have an application that streams video, and on a different computer
> > > an application which plays that video you need to transfer the compressed
> > > data (that is AVPackets and their side data) over the network.  
> > 
> > Use a streaming protocol instead of something hacked together.  
> 
> This is about a streaming protocol. One that supports our side data
> which no currently existing streaming protocol fully supports.

So? Side data is a ffmpeg internal concept.

If you want to create a streaming protocol that does whatever, feel
free to do it, but leave ffmpeg internals out of it.

> designing a new clean streaming protocol that would allow a
> libavcodec encoder to get all data of its packets serialized, transmitted over
> the network deserialized and decoded would be very interresting.
> The existing streaming protocols are not exactly clean.

Relying on ffmpeg internals is NOT clean.

> For example packaging data into a series of mpeg-ts files which are
> then transferred over http (which was designed for web pages not multimedia)
> also commonly called hls / http live streaming is not a clean format.
> Its also not a generic format like for example matroska or nut is for files.
> The others i know of also all suffer from other design issues.
> We could design a better one. Of course this doesnt need to happen
> on ffmpeg-devel if some people here object. Nor does this need to

My god, this is a big load of bullshit again. So because I refuse that
the ffmpeg API is made extra awkward++ we can't have a new spiffy
streaming format that will safe the world.

I guess it's completely worthless trying to argue with you, as shown
with plenty of other examples in the past. It's always the same
worthless bikeshedding, never conceding to the other side, obsessing
about minor issues, wanting to make the common case exceedingly complex
for obscure use cases, and so on.

So let me put this in another way: because of your demands there will
never be a better side data API that replace the current one (that is
full of shortcomings, as illustrated by this patch). And of course your
spiffy new streaming protocol (that suspiciously sounds like ffserver
reloaded) will never happen anyway. I mean, side data has existed for
almost a decade and was supposed to be ABI independent (aka network
safe, i.e. what you're demanding here), and it never happened.

> > > At some point you have to get the side data from one platforms format
> > > to another.  
> > 
> > No, you don't.  
> 
> You do, otherwise side data would not work at all.
> Even storing side data in a mp4 or mkv file as currently done and then reading
> this file back converts it between platforms.
> The whole system of side data wouldnt work if there was no way
> to convert it between 2 platform representations

Well, thanks for admitting that we do not need an "official" ABI
independent representation of side data, because its representations
can always be converted to whatever is needed.

> 
> If someone belives side data never needs to be converted between platforms
> consider this:
> A encoder produces side data, this is specific to the platform the encoder
> runs on currently.
> 
> If you decode this again you need the side data in the format of the platform
> the decoder runs on. This is very very basic logic. Something must convert it
> That is either file format or a network protocol or something else.
> But if it doesnt get converted then its content is lost and the purpose that
> side data had is not realized. Which may be minor or major depending on the
> side data.

And this has nothing to do with anything either.


More information about the ffmpeg-devel mailing list