[FFmpeg-devel] [RFC] avcodec/avcodec.h: Add encryption info side data
Michael Niedermayer
michael at niedermayer.cc
Tue Dec 19 01:00:15 EET 2017
On Mon, Dec 18, 2017 at 10:28:14PM +0100, wm4 wrote:
> On Mon, 18 Dec 2017 22:17:14 +0100
> Michael Niedermayer <michael at niedermayer.cc> wrote:
>
> > On Mon, Dec 18, 2017 at 10:02:52PM +0100, wm4 wrote:
> > > On Mon, 18 Dec 2017 21:38:24 +0100
> > > Michael Niedermayer <michael at niedermayer.cc> wrote:
> > >
> > > > On Mon, Dec 18, 2017 at 04:56:08PM -0300, James Almer wrote:
> > > > > On 12/18/2017 4:52 PM, wm4 wrote:
> > > > > > On Fri, 15 Dec 2017 14:24:17 -0800
> > > > > > Jacob Trimble <modmaker-at-google.com at ffmpeg.org> wrote:
> > > > > >
> > > > > >> From a1b2cbcb7da4da69685f8f1299b70b672ce448e3 Mon Sep 17 00:00:00 2001
> > > > > >> From: Jacob Trimble <modmaker at google.com>
> > > > > >> Date: Tue, 5 Dec 2017 14:52:22 -0800
> > > > > >> Subject: [PATCH] avcodec/avcodec.h: Add encryption info side data.
> > > > > >>
> > > > > >> This new side-data will contain info on how a packet is encrypted.
> > > > > >> This allows the app to handle packet decryption. To allow for a
> > > > > >> variable number of subsamples, the buffer for the side-data will be
> > > > > >> allocated to hold both the structure and the array of subsamples. So
> > > > > >> the |subsamples| member will point to right after the struct.
> > > > > >>
> > > > > >> Signed-off-by: Jacob Trimble <modmaker at google.com>
> > > > > >> ---
> > > > > >> libavcodec/avcodec.h | 70 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > > > >> 1 file changed, 70 insertions(+)
> > > > > >>
> > > > > >> diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
> > > > > >> index 5db6a81320..ccc89345e8 100644
> > > > > >> --- a/libavcodec/avcodec.h
> > > > > >> +++ b/libavcodec/avcodec.h
> > > > > >> @@ -1112,6 +1112,63 @@ typedef struct AVCPBProperties {
> > > > > >> uint64_t vbv_delay;
> > > > > >> } AVCPBProperties;
> > > > > >>
> > > > > >> +typedef struct AVPacketSubsampleEncryptionInfo {
> > > > > >> + /** The number of bytes that are clear. */
> > > > > >> + unsigned int bytes_of_clear_data;
> > > > > >> +
> > > > > >> + /**
> > > > > >> + * The number of bytes that are protected. If using pattern encryption,
> > > > > >> + * the pattern applies to only the protected bytes; if not using pattern
> > > > > >> + * encryption, all these bytes are encrypted.
> > > > > >> + */
> > > > > >> + unsigned int bytes_of_protected_data;
> > > > > >> +} AVPacketSubsampleEncryptionInfo;
> > > > > >> +
> > > > > >> +/**
> > > > > >> + * This describes encryption info for a packet. This contains frame-specific
> > > > > >> + * info for how to decrypt the packet before passing it to the decoder. If this
> > > > > >> + * side-data is present, then the packet is encrypted.
> > > > > >> + */
> > > > > >> +typedef struct AVPacketEncryptionInfo {
> > > > > >> + /** The fourcc encryption scheme. */
> > > > > >> + uint32_t scheme;
> > > > > >> +
> > > > > >> + /** The ID of the key used to encrypt the packet. */
> > > > > >> + uint8_t key_id[16];
> > > > > >> +
> > > > > >> + /** The initialization vector. */
> > > > > >> + uint8_t iv[16];
> > > > > >> +
> > > > > >> + /**
> > > > > >> + * Only used for pattern encryption. This is the number of 16-byte blocks
> > > > > >> + * that are encrypted.
> > > > > >> + */
> > > > > >> + unsigned int crypt_byte_block;
> > > > > >> +
> > > > > >> + /**
> > > > > >> + * Only used for pattern encryption. This is the number of 16-byte blocks
> > > > > >> + * that are clear.
> > > > > >> + */
> > > > > >> + unsigned int skip_byte_block;
> > > > > >> +
> > > > > >> + /**
> > > > > >> + * The number of sub-samples in this packet. If 0, then the whole sample
> > > > > >> + * is encrypted.
> > > > > >> + */
> > > > > >> + unsigned int subsample_count;
> > > > > >> +
> > > > > >> + /** The subsample encryption info. */
> > > > > >> + AVPacketSubsampleEncryptionInfo *subsamples;
> > > > > >
> > > > > > I don't think this is sane. So far, side data could simply be copied
> > > > > > with memcpy, and having pointers to non-static data in side data would
> > > > > > break this completely.
> > > > >
> > > > > Even more reasons to ditch the current side data API and come up with a
> > > > > better designed one that can also be reused for packet, frame and
> > > > > container needs.
> > > >
> > > > yes,
> > > > also if its redesigned, it should become possible to pass it over the
> > > > network. Currently all side data depends on the platform ABI, so its
> > > > only valid on the platform its created on. That also makes it different
> > > > from all other data, AVPacket.data and extradata are all
> > > > platform independant. And there is no API to
> > > > convert it into a platform ABI independant form.
> > >
> > > Why should it be transferable over network? That makes no sense. We
> > > don't have the ability to transfer AVFrame, AVPacket, or AVCodecContext
> > > over network either.
> >
> > If you have an application that streams video, and on a different computer
> > an application which plays that video you need to transfer the compressed
> > data (that is AVPackets and their side data) over the network.
>
> Use a streaming protocol instead of something hacked together.
This is about a streaming protocol. One that supports our side data
which no currently existing streaming protocol fully supports.
designing a new clean streaming protocol that would allow a
libavcodec encoder to get all data of its packets serialized, transmitted over
the network deserialized and decoded would be very interresting.
The existing streaming protocols are not exactly clean.
For example packaging data into a series of mpeg-ts files which are
then transferred over http (which was designed for web pages not multimedia)
also commonly called hls / http live streaming is not a clean format.
Its also not a generic format like for example matroska or nut is for files.
The others i know of also all suffer from other design issues.
We could design a better one. Of course this doesnt need to happen
on ffmpeg-devel if some people here object. Nor does this need to
be done by us or me but it would be beneficial if someone did design a
better streaming protocol.
The existing streaming protocols i know have all issues.
Once we have a generic and clean streaming format we need a way to put
side data in a platform independant way in it.
This is independant of how the streaming format is designed pretty much.
Its just that no current protocol supports transfering all the side data
we support.
Such a new protocol would then likely define the layout of each side data
type byte per byte like any other data in a protocol would be defined.
>
> Also, design APIs for API users and the common case, not for ultra
> obscure special cases that are just vague ideas floating around your
> brain.
>
> > You can use RT*P/hls and others but they lack the concept of our side data
> > so many cases will not work over such system, especially not if there is
> > no platform independant representation of the data which you have to
> > transmit between platforms.
>
> These protocols don't need side data.
>
> > At some point you have to get the side data from one platforms format
> > to another.
>
> No, you don't.
You do, otherwise side data would not work at all.
Even storing side data in a mp4 or mkv file as currently done and then reading
this file back converts it between platforms.
The whole system of side data wouldnt work if there was no way
to convert it between 2 platform representations
If someone belives side data never needs to be converted between platforms
consider this:
A encoder produces side data, this is specific to the platform the encoder
runs on currently.
If you decode this again you need the side data in the format of the platform
the decoder runs on. This is very very basic logic. Something must convert it
That is either file format or a network protocol or something else.
But if it doesnt get converted then its content is lost and the purpose that
side data had is not realized. Which may be minor or major depending on the
side data.
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
Opposition brings concord. Out of discord comes the fairest harmony.
-- Heraclitus
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20171219/91e2af37/attachment.sig>
More information about the ffmpeg-devel
mailing list