Author: conrad Date: Thu Jul 26 02:05:32 2007 New Revision: 532 Log: Write segment UID Modified: matroska/matroskaenc.c Modified: matroska/matroskaenc.c ============================================================================== --- matroska/matroskaenc.c (original) +++ matroska/matroskaenc.c Thu Jul 26 02:05:32 2007 @@ -20,6 +20,7 @@ */ #include "avformat.h" +#include "random.h" #include "riff.h" #include "xiph.h" #include "matroska.h" @@ -61,6 +62,8 @@ typedef struct MatroskaMuxContext { mkv_seekhead *main_seekhead; mkv_seekhead *cluster_seekhead; mkv_cues *cues; + + AVRandomState rand_state; } MatroskaMuxContext; static void put_ebml_id(ByteIOContext *pb, unsigned int id) @@ -507,6 +510,9 @@ static int mkv_write_header(AVFormatCont MatroskaMuxContext *mkv = s->priv_data; ByteIOContext *pb = &s->pb; offset_t ebml_header, segment_info; + int i; + + av_init_random(av_gettime(), &mkv->rand_state); ebml_header = start_ebml_master(pb, EBML_ID_HEADER); put_ebml_uint (pb, EBML_ID_EBMLVERSION , 1); @@ -536,11 +542,16 @@ static int mkv_write_header(AVFormatCont if (strlen(s->title)) put_ebml_string(pb, MATROSKA_ID_TITLE, s->title); if (!(s->streams[0]->codec->flags & CODEC_FLAG_BITEXACT)) { + uint8_t segmentuid[16]; + for (i = 0; i < 16; i++) + segmentuid[i] = av_random(&mkv->rand_state); + put_ebml_string(pb, MATROSKA_ID_MUXINGAPP , LIBAVFORMAT_IDENT); put_ebml_string(pb, MATROSKA_ID_WRITINGAPP, LIBAVFORMAT_IDENT); put_ebml_date (pb, MATROSKA_ID_DATEUTC , av_gettime() ); + put_ebml_binary(pb, MATROSKA_ID_SEGMENTUID, segmentuid, 16); } - // XXX: segment UID + // reserve space for the duration mkv->duration = 0; mkv->duration_offset = url_ftell(pb);
Hi On Thu, Jul 26, 2007 at 02:05:33AM +0200, conrad wrote:
Author: conrad Date: Thu Jul 26 02:05:32 2007 New Revision: 532
Log: Write segment UID [...] @@ -507,6 +510,9 @@ static int mkv_write_header(AVFormatCont MatroskaMuxContext *mkv = s->priv_data; ByteIOContext *pb = &s->pb; offset_t ebml_header, segment_info; + int i; + + av_init_random(av_gettime(), &mkv->rand_state);
leaks current time (=security risk ...) [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB There will always be a question for which you do not know the correct awnser.
On Jul 25, 2007, at 8:48 PM, Michael Niedermayer wrote:
Hi
On Thu, Jul 26, 2007 at 02:05:33AM +0200, conrad wrote:
Author: conrad Date: Thu Jul 26 02:05:32 2007 New Revision: 532
Log: Write segment UID [...] @@ -507,6 +510,9 @@ static int mkv_write_header(AVFormatCont MatroskaMuxContext *mkv = s->priv_data; ByteIOContext *pb = &s->pb; offset_t ebml_header, segment_info; + int i; + + av_init_random(av_gettime(), &mkv->rand_state);
leaks current time (=security risk ...)
What would be best way to seed the random number generator? All the other uses of av_init_random() use a constant, but the purpose of the segment UID is to be unique among segments, so that other files can refer to it. Another idea might be to use a SHA-1 hash of the frame contents, but that seems like it would slow down muxing a fair bit. -David
Hi On Sat, Jul 28, 2007 at 02:47:11AM -0400, David Conrad wrote:
On Jul 25, 2007, at 8:48 PM, Michael Niedermayer wrote:
Hi
On Thu, Jul 26, 2007 at 02:05:33AM +0200, conrad wrote:
Author: conrad Date: Thu Jul 26 02:05:32 2007 New Revision: 532
Log: Write segment UID [...] @@ -507,6 +510,9 @@ static int mkv_write_header(AVFormatCont MatroskaMuxContext *mkv = s->priv_data; ByteIOContext *pb = &s->pb; offset_t ebml_header, segment_info; + int i; + + av_init_random(av_gettime(), &mkv->rand_state);
leaks current time (=security risk ...)
What would be best way to seed the random number generator? All the other uses of av_init_random() use a constant, but the purpose of the segment UID is to be unique among segments, so that other files can refer to it.
well, if so using the current time is not a good idea either, if 2 people start encoding at the same time ...
Another idea might be to use a SHA-1 hash of the frame contents, but that seems like it would slow down muxing a fair bit.
i dont know a solution, finding a good and secure random number is hard you could use the sha1/crc of just the first frame, but that might be a all black frame and thus has a fair chance of being identical between 2 movies (if used quantizer and other encoding options match) [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Asymptotically faster algorithms should always be preferred if you have asymptotical amounts of data
Michael Niedermayer wrote:
Hi
On Sat, Jul 28, 2007 at 02:47:11AM -0400, David Conrad wrote:
On Jul 25, 2007, at 8:48 PM, Michael Niedermayer wrote:
Hi
On Thu, Jul 26, 2007 at 02:05:33AM +0200, conrad wrote:
Author: conrad Date: Thu Jul 26 02:05:32 2007 New Revision: 532
Log: Write segment UID [...] @@ -507,6 +510,9 @@ static int mkv_write_header(AVFormatCont MatroskaMuxContext *mkv = s->priv_data; ByteIOContext *pb = &s->pb; offset_t ebml_header, segment_info; + int i; + + av_init_random(av_gettime(), &mkv->rand_state); leaks current time (=security risk ...) What would be best way to seed the random number generator? All the other uses of av_init_random() use a constant, but the purpose of the segment UID is to be unique among segments, so that other files can refer to it.
well, if so using the current time is not a good idea either, if 2 people start encoding at the same time ...
Another idea might be to use a SHA-1 hash of the frame contents, but that seems like it would slow down muxing a fair bit.
i dont know a solution, finding a good and secure random number is hard you could use the sha1/crc of just the first frame, but that might be a all black frame and thus has a fair chance of being identical between 2 movies (if used quantizer and other encoding options match)
given a collision is always possible: sum the avformat struct and the current time and feed everything to the mersenne twister. lu - too much simple? -- Luca Barbato Gentoo/linux Gentoo/PPC http://dev.gentoo.org/~lu_zero
Hi On Sat, Jul 28, 2007 at 11:48:25AM +0200, Luca Barbato wrote:
Michael Niedermayer wrote:
Hi
On Sat, Jul 28, 2007 at 02:47:11AM -0400, David Conrad wrote:
On Jul 25, 2007, at 8:48 PM, Michael Niedermayer wrote:
Hi
On Thu, Jul 26, 2007 at 02:05:33AM +0200, conrad wrote:
Author: conrad Date: Thu Jul 26 02:05:32 2007 New Revision: 532
Log: Write segment UID [...] @@ -507,6 +510,9 @@ static int mkv_write_header(AVFormatCont MatroskaMuxContext *mkv = s->priv_data; ByteIOContext *pb = &s->pb; offset_t ebml_header, segment_info; + int i; + + av_init_random(av_gettime(), &mkv->rand_state); leaks current time (=security risk ...) What would be best way to seed the random number generator? All the other uses of av_init_random() use a constant, but the purpose of the segment UID is to be unique among segments, so that other files can refer to it.
well, if so using the current time is not a good idea either, if 2 people start encoding at the same time ...
Another idea might be to use a SHA-1 hash of the frame contents, but that seems like it would slow down muxing a fair bit.
i dont know a solution, finding a good and secure random number is hard you could use the sha1/crc of just the first frame, but that might be a all black frame and thus has a fair chance of being identical between 2 movies (if used quantizer and other encoding options match)
given a collision is always possible:
sum the avformat struct and the current time and feed everything to the mersenne twister.
lu - too much simple?
most stupid idea of all the struct contains pointers, so you now leak very usefull information for an exploit ... as i said getting a good and secure random number is not easy, you cannot use anything that could be usefull for an expoit or anything that could by used to identify time or place where encoding happened, not the current time, no pointers, ... you are left with the video/audio you are going to encode, and some stuff we cant easily use like the least significant bits of the time of some interrupts [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In a rich man's house there is no place to spit but his face. -- Diogenes of Sinope
On Sat, Jul 28, 2007 at 02:16:23PM +0200, Michael Niedermayer wrote:
Hi [...]
... as i said getting a good and secure random number is not easy, you cannot use anything that could be usefull for an expoit or anything that could by used to identify time or place where encoding happened, not the current time, no pointers, ... you are left with the video/audio you are going to encode, and some stuff we cant easily use like the least significant bits of the time of some interrupts
What about this? http://en.wikipedia.org/wiki/UUID
[...]
-- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
Hi On Sat, Jul 28, 2007 at 05:43:03PM +0300, Kostya wrote:
On Sat, Jul 28, 2007 at 02:16:23PM +0200, Michael Niedermayer wrote:
Hi [...]
... as i said getting a good and secure random number is not easy, you cannot use anything that could be usefull for an expoit or anything that could by used to identify time or place where encoding happened, not the current time, no pointers, ... you are left with the video/audio you are going to encode, and some stuff we cant easily use like the least significant bits of the time of some interrupts
What about this? http://en.wikipedia.org/wiki/UUID
which version of UUIDs? you have 3 choices 1/2 contains the MAC address (this is the most stupid idea so far not to mention the issue of how to find the MAC address in a portable way) 3/5 are just a md5/sha1 of a string AFAIK which was already suggested 4 is just a random number (with which we are back to square 1 just that we now can call it UUID) what about this? /dev/zero [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB While the State exists there can be no freedom; when there is freedom there will be no State. -- Vladimir Lenin
On Sat, Jul 28, 2007 at 07:09:44PM +0200, Michael Niedermayer wrote:
Hi
On Sat, Jul 28, 2007 at 05:43:03PM +0300, Kostya wrote:
On Sat, Jul 28, 2007 at 02:16:23PM +0200, Michael Niedermayer wrote:
Hi [...]
... as i said getting a good and secure random number is not easy, you cannot use anything that could be usefull for an expoit or anything that could by used to identify time or place where encoding happened, not the current time, no pointers, ... you are left with the video/audio you are going to encode, and some stuff we cant easily use like the least significant bits of the time of some interrupts
What about this? http://en.wikipedia.org/wiki/UUID
which version of UUIDs? you have 3 choices 1/2 contains the MAC address (this is the most stupid idea so far not to mention the issue of how to find the MAC address in a portable way)
That depends on what UUID is used - IIRC this one can be decoded.
3/5 are just a md5/sha1 of a string AFAIK which was already suggested 4 is just a random number (with which we are back to square 1 just that we now can call it UUID)
what about this? /dev/zero
The best seed is lower bits of timestamp, especially in nanoseconds - it's very random and you can't restore anything useful from it. /dev/zero contains a lot of values omitted from other PR sequences so it is random too.
[...]
-- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
On Sat, Jul 28, 2007 at 09:03:42PM +0300, Kostya wrote:
On Sat, Jul 28, 2007 at 07:09:44PM +0200, Michael Niedermayer wrote:
Hi
On Sat, Jul 28, 2007 at 05:43:03PM +0300, Kostya wrote:
On Sat, Jul 28, 2007 at 02:16:23PM +0200, Michael Niedermayer wrote:
Hi [...]
... as i said getting a good and secure random number is not easy, you cannot use anything that could be usefull for an expoit or anything that could by used to identify time or place where encoding happened, not the current time, no pointers, ... you are left with the video/audio you are going to encode, and some stuff we cant easily use like the least significant bits of the time of some interrupts
What about this? http://en.wikipedia.org/wiki/UUID
which version of UUIDs? you have 3 choices 1/2 contains the MAC address (this is the most stupid idea so far not to mention the issue of how to find the MAC address in a portable way)
That depends on what UUID is used - IIRC this one can be decoded.
?
3/5 are just a md5/sha1 of a string AFAIK which was already suggested 4 is just a random number (with which we are back to square 1 just that we now can call it UUID)
what about this? /dev/zero
The best seed is lower bits of timestamp, especially in nanoseconds - it's very random and you can't restore anything useful from it.
we need a _PORTABLE_ solution! me have no millisec precission timer let alone micro or nano if portability wasnt an issue we could just use /dev/random simply using a constant seed is the obvious solution why would a file need a random/unique id anyway?, especially if all? muxers create these "random" ids with a 32bit seed from the current time a 32bit seed means 1 collission in ~64000 files (assuming the 32bit seeds are really random) using srand(time(0)) you get 1 second precission, and this will likely cause additional colissions ... [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB I have often repented speaking, but never of holding my tongue. -- Xenocrates
On Jul 28, 2007, at 3:10 PM, Michael Niedermayer wrote:
On Sat, Jul 28, 2007 at 09:03:42PM +0300, Kostya wrote:
On Sat, Jul 28, 2007 at 07:09:44PM +0200, Michael Niedermayer wrote:
Hi
On Sat, Jul 28, 2007 at 05:43:03PM +0300, Kostya wrote:
3/5 are just a md5/sha1 of a string AFAIK which was already suggested 4 is just a random number (with which we are back to square 1 just that we now can call it UUID)
what about this? /dev/zero
The best seed is lower bits of timestamp, especially in nanoseconds - it's very random and you can't restore anything useful from it.
we need a _PORTABLE_ solution! me have no millisec precission timer let alone micro or nano if portability wasnt an issue we could just use /dev/random
simply using a constant seed is the obvious solution
why would a file need a random/unique id anyway?, especially if all? muxers create these "random" ids with a 32bit seed from the current time a 32bit seed means 1 collission in ~64000 files (assuming the 32bit seeds are really random) using srand(time(0)) you get 1 second precission, and this will likely cause additional colissions ...
mkvmerge uses UuidCreate() on Windows and reads from /dev/urandom on *nix, with a fallback to rand() if either fails (seeded by GetTickCount() on Windows or gettimeofday() on *nix.) The unique ID can be used by a couple of elements (PrevUID, NextUID, and ChapterSegmentUID) to refer to a segment in a different file (or the same file, there can be more than one segment per file) to create a playlist of segments. I don't think it's necessary to try for a completely unique ID, the SHA-1 hash of a couple of frames would be enough imo, given how infrequently these options are used (I'll implement this shortly if there are no objections.) I just didn't want every segment UID to be identical unless the user requests it via bitexact. -David
Hi On Sat, Jul 28, 2007 at 04:23:45PM -0400, David Conrad wrote:
On Jul 28, 2007, at 3:10 PM, Michael Niedermayer wrote:
On Sat, Jul 28, 2007 at 09:03:42PM +0300, Kostya wrote:
On Sat, Jul 28, 2007 at 07:09:44PM +0200, Michael Niedermayer wrote:
Hi
On Sat, Jul 28, 2007 at 05:43:03PM +0300, Kostya wrote:
3/5 are just a md5/sha1 of a string AFAIK which was already suggested 4 is just a random number (with which we are back to square 1 just that we now can call it UUID)
what about this? /dev/zero
The best seed is lower bits of timestamp, especially in nanoseconds - it's very random and you can't restore anything useful from it.
we need a _PORTABLE_ solution! me have no millisec precission timer let alone micro or nano if portability wasnt an issue we could just use /dev/random
simply using a constant seed is the obvious solution
why would a file need a random/unique id anyway?, especially if all? muxers create these "random" ids with a 32bit seed from the current time a 32bit seed means 1 collission in ~64000 files (assuming the 32bit seeds are really random) using srand(time(0)) you get 1 second precission, and this will likely cause additional colissions ...
mkvmerge uses UuidCreate() on Windows and reads from /dev/urandom on *nix, with a fallback to rand() if either fails (seeded by GetTickCount() on Windows or gettimeofday() on *nix.)
The unique ID can be used by a couple of elements (PrevUID, NextUID, and ChapterSegmentUID) to refer to a segment in a different file (or the same file, there can be more than one segment per file) to create a playlist of segments.
I don't think it's necessary to try for a completely unique ID, the SHA-1 hash of a couple of frames would be enough imo, given how infrequently these options are used (I'll implement this shortly if there are no objections.) I just didn't want every segment UID to be identical unless the user requests it via bitexact.
no objections ... though maybe a crc or adler32 checksum would be a better option as they are faster [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Dictatorship naturally arises out of democracy, and the most aggravated form of tyranny and slavery out of the most extreme liberty. -- Plato
On Jul 28, 2007, at 5:56 PM, Michael Niedermayer wrote:
Hi
On Sat, Jul 28, 2007 at 04:23:45PM -0400, David Conrad wrote:
On Jul 28, 2007, at 3:10 PM, Michael Niedermayer wrote:
I don't think it's necessary to try for a completely unique ID, the SHA-1 hash of a couple of frames would be enough imo, given how infrequently these options are used (I'll implement this shortly if there are no objections.) I just didn't want every segment UID to be identical unless the user requests it via bitexact.
no objections ... though maybe a crc or adler32 checksum would be a better option as they are faster
I wound up using a md5sum of the first 200 bytes of the first frame of each cluster since it was the easiest to get 16 bytes without repeating the checksum. I couldn't measure any slowdown with this method.
On Sat, Jul 28, 2007 at 09:10:15PM +0200, Michael Niedermayer wrote:
On Sat, Jul 28, 2007 at 09:03:42PM +0300, Kostya wrote:
On Sat, Jul 28, 2007 at 07:09:44PM +0200, Michael Niedermayer wrote:
Hi
On Sat, Jul 28, 2007 at 05:43:03PM +0300, Kostya wrote:
On Sat, Jul 28, 2007 at 02:16:23PM +0200, Michael Niedermayer wrote:
Hi [...]
... as i said getting a good and secure random number is not easy, you cannot use anything that could be usefull for an expoit or anything that could by used to identify time or place where encoding happened, not the current time, no pointers, ... you are left with the video/audio you are going to encode, and some stuff we cant easily use like the least significant bits of the time of some interrupts
What about this? http://en.wikipedia.org/wiki/UUID
which version of UUIDs? you have 3 choices 1/2 contains the MAC address (this is the most stupid idea so far not to mention the issue of how to find the MAC address in a portable way)
That depends on what UUID is used - IIRC this one can be decoded.
?
There is a standard way to obtain information used for generating from this UUID so this is good for verifying something but not as a random ID. [...]
-- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
Michael Niedermayer wrote:
most stupid idea of all the struct contains pointers, so you now leak very usefull information for an exploit
a _SUM_ of pointers isn't an useful information. more if it is used with the current time as a seed for a random numeber generator.... lu -- Luca Barbato Gentoo/linux Gentoo/PPC http://dev.gentoo.org/~lu_zero
Hi On Sun, Jul 29, 2007 at 02:07:46AM +0200, Luca Barbato wrote:
Michael Niedermayer wrote:
most stupid idea of all the struct contains pointers, so you now leak very usefull information for an exploit
a _SUM_ of pointers isn't an useful information.
your comments are not usefull ... just try main(){ int i; for(i=0; i<10; i++){ void *p= malloc(100*i*i+1); printf("%p\n", p); } } 2 runs will give you the exact same list of pointers in a normal gnu/ulrich drepper libc based system if you now use exec shield or grsec or similar you should see the VERY same pointers just with a constant added to them, that is the same constant to all, the constant would just differ from process to process you can now combine these pointers in any way you want, you dont gain anything by this the result still leaks the constant in a way which a 11 year old could recover
more if it is used with the current time as a seed for a random numeber generator....
well an attacker trying to exploit ffmpeg running on a remote server (maybe one of these "encode random video to flv" services) knows the time when he does the attack so its addition has zero effect [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB When the tyrant has disposed of foreign enemies by conquest or treaty, and there is nothing more to fear from them, then he is always stirring up some war or other, in order that the people may require a leader. -- Plato
Michael Niedermayer wrote:
Hi
On Sun, Jul 29, 2007 at 02:07:46AM +0200, Luca Barbato wrote:
Michael Niedermayer wrote:
most stupid idea of all the struct contains pointers, so you now leak very usefull information for an exploit a _SUM_ of pointers isn't an useful information.
your comments are not usefull ...
ok, I'm an idiot... lu -- Luca Barbato Gentoo/linux Gentoo/PPC http://dev.gentoo.org/~lu_zero
Hi On Sun, Jul 29, 2007 at 07:45:31AM +0200, Luca Barbato wrote:
Michael Niedermayer wrote:
Hi
On Sun, Jul 29, 2007 at 02:07:46AM +0200, Luca Barbato wrote:
Michael Niedermayer wrote:
most stupid idea of all the struct contains pointers, so you now leak very usefull information for an exploit a _SUM_ of pointers isn't an useful information.
your comments are not usefull ...
ok, I'm an idiot...
no you are not but iam pretty good at insulting people unintentionally ... [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In a rich man's house there is no place to spit but his face. -- Diogenes of Sinope
participants (5)
-
conrad -
David Conrad -
Kostya -
Luca Barbato -
Michael Niedermayer