[00:53:49 CET] <cone-695> ffmpeg 03Michael Niedermayer 07master:fccba32b4c4f: tools: add target_dec_fate.sh [02:28:32 CET] <ShadowOfThePast> hello guys [02:28:58 CET] <ShadowOfThePast> i'm losing my mind tracing an ENOMEM issue in libavformat [02:29:53 CET] <ShadowOfThePast> is there any sorcery involved in the av_fast_realloc function? [02:31:03 CET] <ShadowOfThePast> i added some logging and it returns 1114144 in its size argument at the call site i'm interested in [02:32:37 CET] <ShadowOfThePast> it does so hundreds of times so that it should exceed the memory of my box pretty quickly, but it only finally errors out after ~1400 of those 1 MB allocations (my box has 384 MB of ram, runs Linux 2.6.34 and has no paging from what i know, because it runs off a 32 mb nand flash) [02:33:26 CET] <ShadowOfThePast> the particular function that calls it in my case is mov_try_read_block in mov.c [02:34:00 CET] <ShadowOfThePast> i'm reading a cenc-encrypted mp4 file and it uses this function while parsing saiz atoms [02:50:45 CET] <iive> ShadowOfThePast, if i read the code correctly, it just allocates 6% bigger buffer (1/16 of the target size). [02:51:24 CET] <ShadowOfThePast> yea alright, that explains why it returns that odd number instead of 1024*1024 [02:52:42 CET] <ShadowOfThePast> i'm really mind-blown though how it works, because those blocks aren't ever freed (until it runs out of memory and stops playback) [02:53:05 CET] <ShadowOfThePast> they're only freed in mov_free_encryption_index, which isn't called unless the context is closed [02:54:00 CET] <ShadowOfThePast> and like i said, my file has 1559 of those atoms so it should be taking a ton of memory (stupidly, because the data itself is only like 50 bytes every time) [02:54:41 CET] <iive> how big is this file? [02:54:53 CET] <ShadowOfThePast> 500 MB [02:55:17 CET] <iive> so... the atoms are not really 1mb each? [02:55:30 CET] <ShadowOfThePast> yea they're just a couple of bytes [02:55:47 CET] <ShadowOfThePast> that read_block function just allocates at least a meg every time no matter what [02:56:31 CET] <iive> oh, i see. mov probably allocates max size, then reallocs to the real size. but fast-realloc never shrinks [02:58:02 CET] <iive> sorry, I'm not familiar with mov. does atoms have size parameter? [02:59:37 CET] <ShadowOfThePast> yes they do, but in this case it reads a part of the atom only, where the size of that part is specified by an integer in the atom [02:59:45 CET] <ShadowOfThePast> check out the mov_read_saiz function [03:02:02 CET] <ShadowOfThePast> a realloc to shrink it would make sense, but i don't see that happening anywhere for the auxiliary_info_sizes buffer [03:10:54 CET] <iive> so, the problem is in mov_try_read_block() [03:12:37 CET] <iive> ShadowOfThePast, wb [03:12:44 CET] <ShadowOfThePast> thanks, net died [03:12:59 CET] <iive> just put an av_realloc() before the return. [03:16:15 CET] <iive> if i read the code correctly, it should be "size" bytes long. [03:17:00 CET] <ShadowOfThePast> yea i was under the same impression [03:17:02 CET] <ShadowOfThePast> i'll try that now [03:17:40 CET] <iive> i guess that this function starts with 1mb buffer, just in case... [03:18:31 CET] <ShadowOfThePast> i'll know whether it helped in 46 minutes [03:20:20 CET] <iive> and it stays at 1mb [03:20:30 CET] <iive> well, tell me when i come back. [03:20:32 CET] <iive> gtg [03:20:40 CET] <ShadowOfThePast> aight [03:21:26 CET] <iive> don't forget to open ticket and send patch :D [03:27:33 CET] <iive> who wrote that function?... [03:28:34 CET] <iive> anyway... [03:28:39 CET] <iive> n8 ppl [04:15:01 CET] <ShadowOfThePast> shrinking the buffer did indeed fix the issue, the stream plays through fine now [04:41:17 CET] <ShadowOfThePast> there we go [04:41:20 CET] <ShadowOfThePast> good night [10:57:17 CET] <j-b> 'morning [00:00:00 CET] --- Sun Dec 30 2018