MPEG-2 Acceleration Under Mac OS X - Can We Move This Forward?
Hello, I've been a little bit involved with an interesting project called Accellent which is a libavcodec-based reverse-engineer of Mac OS X's MPEG-2 hardware acceleration API. Apple refused to release any data on this, so John decided to reverse engineer it. After a few iterations and a bit of work, we now have a nice open source application that works under Mac OS 10.4 and 10.3. Take a look at it here: http://www.defyne.org/dvb/accellent.html The performance is great. Even on G4s, full resolution HDTV decoding only consumes 15-35% of the CPU. On these same machines, ffmpeg-based players (and pretty much all other players) don't even come close to fluid playback - that's millions of Mac users who can't use their machines for HD playback. There's no reason it won't work on G3s with an acceptable graphics card as well. While newer machines are capable of software-only decoding, low end G5s sometimes have trouble with HD, and for any machine - why use the CPU, when you could use the GPU? It means that you can multi-task better, reduce power consumption (longer laptop battery life) and have a cooler machine (less fan noise, cooler lap.) We've cracked open Apple's secret API, but the problem is that we are stuck. The application is a great proof-of-concept using libavcodec. But John's busy and the application isn't really useful for any function: it's crude demuxer and a window that displays the video. It only accepts MPEG program streams, but it would be far more useful if it accepted MPEG transport streams. There are no playback controls. It does not attempt to decode audio, but it really needs to decode AC3 audio to be useful. The interactions with libavcodec are a bit of a hack, so you can't just readily drop this into another player that uses libavcodec and be on your way. I'm tossing this out to you all, as ffmpeg developers, in hopes that somebody has some interest in working further on this. It would be interesting if the functionality could somehow be integrated into libavcodec, or perhaps just simply an effective player be put together. There are millions of users who could benefit! Feel free to contact me off list if you prefer. -Galen
Hi, galenz at zinkconsulting.com a ?crit :
Hello,
I've been a little bit involved with an interesting project called Accellent which is a libavcodec-based reverse-engineer of Mac OS X's MPEG-2 hardware acceleration API. Apple refused to release any data on this, so John decided to reverse engineer it. After a few iterations and a bit of work, we now have a nice open source application that works under Mac OS 10.4 and 10.3. Take a look at it here: http://www.defyne.org/dvb/accellent.html
Would you happen to know if there's a similar MPEG-4 AVC decoding API for Apple computers? MPEG-4 AVC is even more hungry than MPEG2, so any kind of hardware acceleration would be great! [..] Guillaume
On Aug 28, 2006, at 3:27 PM, Guillaume POIRIER wrote:
Hi,
galenz at zinkconsulting.com a ?crit :
Hello, I've been a little bit involved with an interesting project called Accellent which is a libavcodec-based reverse-engineer of Mac OS X's MPEG-2 hardware acceleration API. Apple refused to release any data on this, so John decided to reverse engineer it. After a few iterations and a bit of work, we now have a nice open source application that works under Mac OS 10.4 and 10.3. Take a look at it here: http://www.defyne.org/dvb/accellent.html
Would you happen to know if there's a similar MPEG-4 AVC decoding API for Apple computers? MPEG-4 AVC is even more hungry than MPEG2, so any kind of hardware acceleration would be great!
[..]
Guillaume
I know the allure of such a function - but I question the reality of it, at least at the moment. Keep in mind, MPEG-2 is quality king at higher bitrates, and CPU usage climbs with bitrate, so it can be pretty intensive under the right circumstances. Try watching a 60 fps, 1920 x 1080 19 megabit stream - that's what they broadcast over the air in many countries - and let me know how your system fares. Now toss in the fact you may need to de-interlace this. MPEG-2 acceleration is a very real need. Don't forget, MPEG-2 is also part of the Blu-Ray/HD-DVD specification - and they allow up to 60 fps, 1920 x 1080 (progressive) at 40 megabits / 28 megabits respectively. They're using MPEG-2 in most if not all current HD/BR discs because it seems to deliver the highest level of quality, given the available space. MPEG-2 is here to stay, or so it seems. Interestingly, MPEG-2 can also be competitive at lower bitrates, just check out these pretty impressive samples which are purported to rival H.264: http://www.digigami.com/megapeg/hdtv_samples.php Take a look at John's earlier post, he details how his acceleration works. Accellent is so effective precisely because there's an API for MPEG-2. At best, attempting to use MPEG-2 functions for other codecs would somewhat make piecemeal of the the performance gain, and at worst it would be useless. As for whether Apple has an API for MPEG-4/AVC acceleration, I'm not certain, but I'm a bit doubtful - at least for the moment. Only really new GPUs would support such a thing, meaning that even if it existed, the installed base would be tiny. Not that there wouldn't be benefit to using the GPU when available, but at the moment, more machines have sufficient CPUs than have sufficient GPUs, yet every machine that has a sufficient GPU has a sufficient CPU. So there's not a major impetus here quite yet. Furthermore, AVC acceleration isn't very impressive at the moment. The best case scenario is with the ATI X1600 used in the MacBook Pro and iMac. It supports up to 720p AVC acceleration. Using Windows drivers, it only reduces CPU usage by 39%. Not very impressive. Previous ATI GPUs (before the x1xxx series) don't support AVC acceleration at all. The numbers I have seen for NVidia's approach are even less impressive - we're talking 5% or less reduction in average CPU usage using Windows. The only bright spot is that a wider range of NVidia GPUs support this (including some used in previous Mac models), but I seriously doubt there's much point in even trying to support it considering the lackluster benefits. The MacBook and Mac Mini machines use the GMA 950, which while it supports full resolution MPEG-2 HD decoding, it doesn't touch AVC. It seems the time for AVC acceleration hasn't yet arrived. Conversely, millions upon millions of Apple machines exist that can't play HD MPEG-2 in any form, yet have the requisite GPU to do so. MPEG-2 accelleration can drop CPU usage by 75% or more - videos that don't come close to playing on my 1.33 GHz G4 in VLC consume less than 25% of my CPU in Accellent. And pretty much every GPU, even the lowest end GMA 950, is capable of full MPEG-2 HD playback with acceleration. Because of this, frankly, I see MPEG-2 acceleration as being a lot more widely useful technology to pursue. The need, the API and hardware is here and now, just adapt Accellent, and pretty much every machine shipped in the past few years can benefit immensely. MPEG-4/AVC is something to keep your eyes on - maybe in Leopard, and as better GPUs come along - but for now, let's focus on MPEG-2. Just my 2?. -Galen
Hi galenz at zinkconsulting.com wrote:
[...]
I know the allure of such a function - but I question the reality of it, at least at the moment. Keep in mind, MPEG-2 is quality king at higher bitrates, and CPU usage climbs with bitrate, so it can be pretty intensive under the right circumstances. Try watching a 60 fps, 1920 x 1080 19 megabit stream - that's what they broadcast over the air in many countries - and let me know how your system fares. Now toss in the fact you may need to de-interlace this. MPEG-2 acceleration is a very real need.
Does anyone actually broadcast at 1080p ? I only know broadcasters at 720 60p(ESPN, ABC ?) and 1080 60i. -- Baptiste COUDURIER GnuPG Key Id: 0x5C1ABAAA SMARTJOG S.A. http://www.smartjog.com Key fingerprint 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA Phone: +33 1 49966312
On Aug 29, 2006, at 4:31 AM, Baptiste Coudurier wrote:
Hi
galenz at zinkconsulting.com wrote:
[...]
I know the allure of such a function - but I question the reality of it, at least at the moment. Keep in mind, MPEG-2 is quality king at higher bitrates, and CPU usage climbs with bitrate, so it can be pretty intensive under the right circumstances. Try watching a 60 fps, 1920 x 1080 19 megabit stream - that's what they broadcast over the air in many countries - and let me know how your system fares. Now toss in the fact you may need to de-interlace this. MPEG-2 acceleration is a very real need.
Does anyone actually broadcast at 1080p ? I only know broadcasters at 720 60p(ESPN, ABC ?) and 1080 60i.
-- Baptiste COUDURIER GnuPG Key Id: 0x5C1ABAAA SMARTJOG S.A. http:// www.smartjog.com Key fingerprint 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA Phone: +33 1 49966312
Sorry, I wasn't completely clear there - I only made an offhand reference to the fact the content is interlaced. ("Now toss in the fact you may need to de-interlace this.") 1080p is outside the specification for terrestrial broadcasts in the US. Everything is 720p (up to 60 fps) or 1080i (up to 60 fps). I've seen 1080i used in such a way that 30 fps progressive content is broadcast as interlaced 60 fps content, effectively meaning that you can de-interlace without quality loss. That said, 1080p is not outside the specifications for MPEG-2 on HD/ BR discs. -Galen
On Tue, Aug 29, 2006 at 04:26:26AM -0700, galenz at zinkconsulting.com wrote:
Don't forget, MPEG-2 is also part of the Blu-Ray/HD-DVD specification - and they allow up to 60 fps, 1920 x 1080 (progressive)
A common misconception. Neither HD DVD nor Blu-Ray video allows 1080p60. This doesn't really affect your point, though...
On Aug 29, 2006, at 6:19 AM, Jeff Clagg wrote:
On Tue, Aug 29, 2006 at 04:26:26AM -0700, galenz at zinkconsulting.com wrote:
Don't forget, MPEG-2 is also part of the Blu-Ray/HD-DVD specification - and they allow up to 60 fps, 1920 x 1080 (progressive)
A common misconception. Neither HD DVD nor Blu-Ray video allows 1080p60. This doesn't really affect your point, though...
So you mean they are doing 1080p30? My apologies if I'm wrong, although as you mentioned, it really isn't the point of my post so I didn't fact check that before posting.
galenz at zinkconsulting.com said:
On Aug 29, 2006, at 6:19 AM, Jeff Clagg wrote:
On Tue, Aug 29, 2006 at 04:26:26AM -0700, galenz at zinkconsulting.com wrote:
Don't forget, MPEG-2 is also part of the Blu-Ray/HD-DVD specification - and they allow up to 60 fps, 1920 x 1080 (progressive)
A common misconception. Neither HD DVD nor Blu-Ray video allows 1080p60. This doesn't really affect your point, though...
So you mean they are doing 1080p30? My apologies if I'm wrong, although as you mentioned, it really isn't the point of my post so I didn't fact check that before posting.
Film is shot at 24fps, so there's hardly any point encoding it at 60fps. -- M?ns Rullg?rd mru at inprovide.com
Hello, galenz at zinkconsulting.com wrote:
I'm tossing this out to you all, as ffmpeg developers, in hopes that somebody has some interest in working further on this. It would be interesting if the functionality could somehow be integrated into libavcodec, or perhaps just simply an effective player be put together. There are millions of users who could benefit!
I'm interested in this as I have a MacBook and such things would be useful. I doubt I have the same level of skills as yourself and John but I might be able to offer some suggestions. My initial thought was that you might try to make it into a library such that it can be easily used by any project. MPlayerOSX or VLC might be other points of interest if you don't want to make it into a library. Also, is it possible to extend to functionality of the code to allow decoding of other codecs than MPEG-2? Regards, Rob
Robert, I hate to admit it, but I highly doubt I have immensely more skill than you do. John is the real talent behind this. I'm just passionate, tenacious and (sadly?) headed a slightly different direction with my professional work, so while I keep up with computers, I don't have enough time and (most importantly) "mental money" to get into new programming - at last not of the complexity required to really turn Accellent into a portable library or integrate it with a mainstream player all by myself. What got this going back in January was that I started thinking about some tricks I could pull and I contrived a successful way to get DVD Player.app to render HD MPEG-2. Once I did that, John was able to get Accellent up and running under 10.4 in record time and actually improve it substantially. Now we have a fairly portable proof-of- concept application. To give you an idea of the performance, I tested VLC (Intel build) versus Accellent running some 1920 x 1080 HD content: VLC used 90-110% of the CPU on a 2 GHz Core Duo MacBook. On a dramatically slower 1.33 GHz PowerBook G4, the same clip played back using no more than 25% of the CPU. Now that's what I call a performance boost! The challenge is going to be that this isn't just a library you feed the bits into as MPEG-2 and have them dumped out as a full resolution, uncompressed video. It actually basically takes the data, does some pre-processing, and then hands the data off to the Apple stuff (it's the handoff that's the tricky part) which then actually dumps it to the display. That's how you get such amazing performance. That's also why it's a bit tricky, you would almost certainly have to modify the display code in your player to get it working. As for using it with other codecs, I think that would require another layer of reverse engineering, and I think that you wouldn't see such great performance gains. Despite being an "older" codec, MPEG-2 is the quality leader at high bitrates is part of the Blu-ray and HD-DVD spec, and is currently used for HDTV broadcast over the air in the US and many other countries. You can get TV tuners that plug into your machine via FireWire and dump an HD stream. The problem for slower machines is playing back the video. And even for fast machines, heat/ noise/power consumption/multi-tasking can be a problem. I've also been in touch with a the VLC developers list (I vastly prefer VLC's OS X interface over MPlayer's) and there is a modicum of interest, but DJ would like to see more documentation about Accellent - which really needs to come from John. I've written John and we'll see if we can't get a bit more documentation out of him. If you feel inclined, drop John a friendly email expressing the value you would see in him providing a bit more documentation so this project can move forward - johnd at defyne.org -Galen On Aug 28, 2006, at 5:58 PM, Robert Swain wrote:
Hello,
galenz at zinkconsulting.com wrote:
I'm tossing this out to you all, as ffmpeg developers, in hopes that somebody has some interest in working further on this. It would be interesting if the functionality could somehow be integrated into libavcodec, or perhaps just simply an effective player be put together. There are millions of users who could benefit!
I'm interested in this as I have a MacBook and such things would be useful. I doubt I have the same level of skills as yourself and John but I might be able to offer some suggestions. My initial thought was that you might try to make it into a library such that it can be easily used by any project. MPlayerOSX or VLC might be other points of interest if you don't want to make it into a library. Also, is it possible to extend to functionality of the code to allow decoding of other codecs than MPEG-2?
Regards, Rob _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel at mplayerhq.hu http://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-devel
Also, is it possible to extend to functionality of the code to allow decoding of other codecs than MPEG-2?
Anything is possible (the GPU vendors talk about MPEG4 assist too), but I think that is very unlikely. From my understanding of it, Apple only uses HW assist for DVDs. John has reverse-engineered this, and it is possible he knows what parts of the API do aspects like ICT (useful for some other codecs), but that is a lot more work. -- Nigel Pearson, nigel at ind.tansu.com.au|"Gentlemen, Telstra Net. Eng., Sydney, Australia | you can't fight in here Office: 9202 3900 Fax: 9261 3912 | - this is the war room!" Mobile: 0408 664435 Home: 9792 6998 | Dr Strangelove
On Aug 28, 2006, at 7:09 PM, Nigel Pearson wrote:
Also, is it possible to extend to functionality of the code to allow decoding of other codecs than MPEG-2?
Anything is possible (the GPU vendors talk about MPEG4 assist too), but I think that is very unlikely. From my understanding of it, Apple only uses HW assist for DVDs.
John has reverse-engineered this, and it is possible he knows what parts of the API do aspects like ICT (useful for some other codecs), but that is a lot more work.
-- Nigel Pearson, nigel at ind.tansu.com.au|"Gentlemen, Telstra Net. Eng., Sydney, Australia | you can't fight in here Office: 9202 3900 Fax: 9261 3912 | - this is the war room!" Mobile: 0408 664435 Home: 9792 6998 | Dr Strangelove
My understanding of John's implementation is that he's handing off the MPEG-2 in a fairly whole manner. I believe we had discussed this and you would have to do another layer of reverse engineering to get down to the very specific functions like IDCT. I have a hard time imagining the gains being nearly so great, however, unless you were accessing GPU functions specifically for MPEG-4, h.264, etc. And as we all know, graphics vendors aren't terribly helpful with documentation. With Accellent, it's nice that by relying on Apple's API, it abstracts the code from the specific GPU, so it makes the application a lot more portable between systems, new and old. I'm presently reinstalling the developer tools (don't ask why) and once that finishes in a few minutes, I will look over the code again. But frankly, I'm not likely to glean much inspiration because I'm not qualified to turn this into something very useful unless somebody volunteers to let me ask countless stupid questions and I try to figure out what end is up. Once upon a time ago, I worked full time doing behind the scenes web programming, but that's a far leap from this. Sure, it's still object-oriented programming, but that's about it... -Galen
On Tue, 29 Aug 2006, Nigel Pearson wrote:
Also, is it possible to extend to functionality of the code to allow decoding of other codecs than MPEG-2?
Anything is possible (the GPU vendors talk about MPEG4 assist too), but I think that is very unlikely. From my understanding of it, Apple only uses HW assist for DVDs.
Yes I'm sure the GPU manufacturers would be quite happy to make an API suited to MPEG4 for Apple, but since it's not in Apple's interests it's very unlikely to happen. Apple want to sell more and faster computers. Their latest computers don't have too much trouble with MPEG4 / H264 in software so it becomes another reason to upgrade. The only reason we have an MPEG2-on-GPU API is because the first Macs that could play DVDs really really needed the help. (This is all IMHO of course.) One thing that might be easily salvagable from the DVDVideo API is its deinterlacing. Although this is not really a topic for ffmpeg. Apart from that I fear it's too MPEG2-specific to be of use elsewhere. I don't know much about the internals of codecs 'tho so maybe someone else has a better idea. Here are the structures ffmpeg currently fills in: --------------------------------------------------------------- typedef struct DVDV_Frame { int intra_vlc_format; int alternate_scan; } DVDV_Frame; typedef struct DVDV_MBInfo { int16_t frwd_mv[2]; int16_t back_mv[2]; int16_t frwd_mv_field[2]; int16_t back_mv_field[2]; uint8_t field_select[4]; // 0/1 in order of mvs above uint8_t mb_type; // |1 = frwd, |2 = back, |4 = use fields uint8_t ildct; // 0/1 uint8_t num_coded_elts[6]; // for each block incl. dc } DVDV_MBInfo; typedef struct DVDV_DCTElt { uint8_t run_sub_one; uint8_t unused; uint16_t elt; // adjusted for first coded elt in block: } DVDV_DCTElt; // intra sub 0x0400, else add 0x0400 typedef uint8_t DVDV_CPB; --------------------------------------------------------------- Hopefully they are self-explanatory for those familiar with MPEG2. Do they look like they could be adapted for use in other codecs to anyone? {P^/
On Tue, Aug 29, 2006 at 01:58:19AM +0100, Robert Swain wrote:
Hello,
galenz at zinkconsulting.com wrote:
I'm tossing this out to you all, as ffmpeg developers, in hopes that somebody has some interest in working further on this. It would be interesting if the functionality could somehow be integrated into libavcodec, or perhaps just simply an effective player be put together. There are millions of users who could benefit!
I'm interested in this as I have a MacBook and such things would be useful. I doubt I have the same level of skills as yourself and John but I might be able to offer some suggestions. My initial thought was that you might try to make it into a library such that it can be easily used by any project. MPlayerOSX or VLC might be other points of interest if you don't want to make it into a library. Also, is it possible to extend to functionality of the code to allow decoding of other codecs than MPEG-2?
The best way to use this IMO is to make a vo driver for it in MPlayer, possibly merging it with an existing relevant MPlayer vo driver. FFmpeg libs should communicate with the driver the same way they do for xvmc. Rich
On Aug 28, 2006, at 7:10 PM, Rich Felker wrote:
On Tue, Aug 29, 2006 at 01:58:19AM +0100, Robert Swain wrote:
Hello,
galenz at zinkconsulting.com wrote:
I'm tossing this out to you all, as ffmpeg developers, in hopes that somebody has some interest in working further on this. It would be interesting if the functionality could somehow be integrated into libavcodec, or perhaps just simply an effective player be put together. There are millions of users who could benefit!
I'm interested in this as I have a MacBook and such things would be useful. I doubt I have the same level of skills as yourself and John but I might be able to offer some suggestions. My initial thought was that you might try to make it into a library such that it can be easily used by any project. MPlayerOSX or VLC might be other points of interest if you don't want to make it into a library. Also, is it possible to extend to functionality of the code to allow decoding of other codecs than MPEG-2?
The best way to use this IMO is to make a vo driver for it in MPlayer, possibly merging it with an existing relevant MPlayer vo driver. FFmpeg libs should communicate with the driver the same way they do for xvmc.
Rich
Any volunteers? :) -Galen
Hi, On Mon, 28 Aug 2006, Rich Felker wrote:
On Tue, Aug 29, 2006 at 01:58:19AM +0100, Robert Swain wrote:
galenz at zinkconsulting.com wrote:
I'm tossing this out to you all, as ffmpeg developers, in hopes that somebody has some interest in working further on this. It would be interesting if the functionality could somehow be integrated into libavcodec, or perhaps just simply an effective player be put together. There are millions of users who could benefit!
I'm interested in this as I have a MacBook and such things would be useful. I doubt I have the same level of skills as yourself and John but I might be able to offer some suggestions. My initial thought was that you might try to make it into a library such that it can be easily used by any project. MPlayerOSX or VLC might be other points of interest if you don't want to make it into a library. Also, is it possible to extend to functionality of the code to allow decoding of other codecs than MPEG-2?
The best way to use this IMO is to make a vo driver for it in MPlayer, possibly merging it with an existing relevant MPlayer vo driver. FFmpeg libs should communicate with the driver the same way they do for xvmc.
Firstly I should say that I am the John that Galen is talking about (and I'm definitely not famous/rare enough to omit my last name :). I haven't worked on this any further since January when it was technically finished, and probably won't for quite some time so anyone who wants to run with it is most welcome. I know at least one person has started on a patch... As Rich says, the ffmpeg portion is quite small; all it has to do is get the data out in the right way. I did of course use a modified file from ffmpeg to do this (mpeg12.c) in Accellent. IIRC the problem is that the data expected by the Apple API is one level higher than XVMC. It takes the un-VLC'd run/level values plus the macroblock type (to determine the scantable) whereas XVMC just takes the whole 64 element block. This is great from a bandwidth point of view but doesn't integrate well with the existing mpeg decoding routine in ffmpeg. I didn't come up with a solution that doesn't either: 1. Add lots of #ifdefs uglifying the code (and preventing runtime selection), or 2. Add lots of ifs slowing down the code, or 3. Duplicate significant portions of the code. Whoever takes this on will first have to decide which of these is the lesser evil (probably 3? or 2 if ifs prove predictable?). And then making a coherent case to the list. Good luck! :) {P^/
Hi On Tue, Aug 29, 2006 at 06:03:06PM +1000, John Dalgliesh wrote:
Hi,
On Mon, 28 Aug 2006, Rich Felker wrote:
On Tue, Aug 29, 2006 at 01:58:19AM +0100, Robert Swain wrote:
galenz at zinkconsulting.com wrote:
I'm tossing this out to you all, as ffmpeg developers, in hopes that somebody has some interest in working further on this. It would be interesting if the functionality could somehow be integrated into libavcodec, or perhaps just simply an effective player be put together. There are millions of users who could benefit!
I'm interested in this as I have a MacBook and such things would be useful. I doubt I have the same level of skills as yourself and John but I might be able to offer some suggestions. My initial thought was that you might try to make it into a library such that it can be easily used by any project. MPlayerOSX or VLC might be other points of interest if you don't want to make it into a library. Also, is it possible to extend to functionality of the code to allow decoding of other codecs than MPEG-2?
The best way to use this IMO is to make a vo driver for it in MPlayer, possibly merging it with an existing relevant MPlayer vo driver. FFmpeg libs should communicate with the driver the same way they do for xvmc.
Firstly I should say that I am the John that Galen is talking about (and I'm definitely not famous/rare enough to omit my last name :). I haven't worked on this any further since January when it was technically finished, and probably won't for quite some time so anyone who wants to run with it is most welcome. I know at least one person has started on a patch...
As Rich says, the ffmpeg portion is quite small; all it has to do is get the data out in the right way. I did of course use a modified file from ffmpeg to do this (mpeg12.c) in Accellent.
IIRC the problem is that the data expected by the Apple API is one level higher than XVMC. It takes the un-VLC'd run/level values plus the macroblock type (to determine the scantable) whereas XVMC just takes the whole 64 element block. This is great from a bandwidth point of view but doesn't integrate well with the existing mpeg decoding routine in ffmpeg.
I didn't come up with a solution that doesn't either: 1. Add lots of #ifdefs uglifying the code (and preventing runtime selection), or 2. Add lots of ifs slowing down the code, or 3. Duplicate significant portions of the code.
Whoever takes this on will first have to decide which of these is the lesser evil (probably 3? or 2 if ifs prove predictable?). And then making a coherent case to the list. Good luck! :)
there are many ways to do this A. always_inline decode_block(..., int unpack){ ... if(unpack){ //unquantize level= ((level*2+1)*qscale*quant_matrix[j])>>5; level= (level-1)|1; } ... if(unpack) block[j] = level; else{ block[2*i ] = level; block[2*i+1] = run; } ... } if(foobar) decode_block(..., 0); else decode_block(..., 1); or B. #define decode_block_internal(..., unpack){\ ... if(unpack){ //unquantize\ level= ((level*2+1)*qscale*quant_matrix[j])>>5;\ level= (level-1)|1;\ }\ ... if(unpack) block[j] = level;\ else{\ block[2*i ] = level;\ block[2*i+1] = run;\ }\ ... } decode_block(...){ decode_block_internal(..., 1); } decode_block_unvlc(...){ decode_block_internal(..., 0); } or other variations of that, most important though is to carefully benchmark it (see START/STOP_TIMER) to ensure everything works as expected, the inline based variants depend on gcc not messing but they are cleaner and more readable ... PS: also if anyone does that, please merge the decode_block_foobar() and decode_block_foobar_fast() functions they are nearly identical and i should have implemented them like descibed above furthermore merging them will half the number of functions which need to be changed for apple hw decoding support [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In the past you could go to a library and read, borrow or copy any book Today you'd get arrested for mere telling someone where the library is
Hi On Tue, Aug 29, 2006 at 11:15:49AM +0200, Michael Niedermayer wrote: [...]
PS: also if anyone does that, please merge the decode_block_foobar() and decode_block_foobar_fast() functions they are nearly identical and i should have implemented them like descibed above furthermore merging them will half the number of functions which need to be changed for apple hw decoding support
additional tip, to merge the functions, dump them to a file each and run diff over them [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In the past you could go to a library and read, borrow or copy any book Today you'd get arrested for mere telling someone where the library is
Aeons ago, Michael wrote:
Subject: Re: [Ffmpeg-devel] MPEG-2 Acceleration Under Mac OS X - Can We Move This Forward?
Hi
On Tue, Aug 29, 2006 at 11:15:49AM +0200, Michael Niedermayer wrote: [...]
PS: also if anyone does that, please merge the decode_block_foobar () and decode_block_foobar_fast() functions they are nearly identical and i should have implemented them like descibed above furthermore merging them will half the number of functions which need to be changed for apple hw decoding support
additional tip, to merge the functions, dump them to a file each and run diff over them
Well, I finally got around to trying this. I know the attached diff isn't commit ready (I have purposely used bad formatting, and left dead code there, to minimise the size of the diff), but I would appreciate comments. e.g. Is this what was meant? Changes seem to pass regression tests, but I suspect that these don't use CODEC_FLAG2_FAST? I have not benchmarked these (still haven't worked out how). They may be marginally slower due to the extra conditional - if (fast) - inside some of the loops. If anyone has a clever way to optimise this, please share! Change 1: mpeg2_fast_decode_block_intra(...) => mpeg2_decode_block_intra(..., DECODE_FAST) mpeg2_decode_block_intra(...) => mpeg2_decode_block_intra(..., DECODE_SLOW) There were only trivial diffs between these functions. Fast version used a pointer instead of an extra scantable index int, and the non-fast version has some debug, damaged ac-tex checking and mismatched block stuff. Change 2: mpeg1_fast_decode_block_inter(...) => mpeg1_decode_block_inter(..., DECODE_FAST) mpeg1_decode_block_inter(...) => mpeg1_decode_block_inter(..., DECODE_SLOW) mpeg2_fast_decode_block_non_intra(...) => mpeg2_decode_block_non_intra (..., DECODE_FAST) mpeg2_decode_block_non_intra(...) => mpeg2_decode_block_non_intra (..., DECODE_SLOW) These function pairs were also nearly identical. Fast versions didn't use quant_matrix[], check for damaged ac-tex or deal with mismatched blocks. % svn update At revision 7850. % svn diff libavcodec/mpeg12.c >mpeg12.c-refactor-1.diff % -- Nigel Pearson, nigel at ind.tansu.com.au|"Gentlemen, Telstra Net. Eng., Sydney, Australia | you can't fight in here- Office: 9202 3900 Fax: 9261 3912 | this is the war room!" Mobile: 0408 664435 Home: 9792 6998 | Dr Strangelove
Hi On Wed, Feb 07, 2007 at 11:19:08AM +1100, Nigel Pearson wrote:
Aeons ago, Michael wrote:
Subject: Re: [Ffmpeg-devel] MPEG-2 Acceleration Under Mac OS X - Can We Move This Forward?
Hi
On Tue, Aug 29, 2006 at 11:15:49AM +0200, Michael Niedermayer wrote: [...]
PS: also if anyone does that, please merge the decode_block_foobar () and decode_block_foobar_fast() functions they are nearly identical and i should have implemented them like descibed above furthermore merging them will half the number of functions which need to be changed for apple hw decoding support
additional tip, to merge the functions, dump them to a file each and run diff over them
Well, I finally got around to trying this. I know the attached diff isn't commit ready (I have purposely used bad formatting,
it looks pretty good, formatting changes must be in a seperate patch anyway and removing the old code can be done in a seperate change too ...
and left dead code there, to minimise the size of the diff), but I would appreciate comments. e.g. Is this what was meant?
yes!
Changes seem to pass regression tests, but I suspect that these don't use CODEC_FLAG2_FAST?
then test it (the mpeg1 and 2 files in tests/data/ can simply be used with rawvideo output and md5sum)
I have not benchmarked these (still haven't worked out how).
see START/STOP_TIMER
They may be marginally slower due to the extra conditional - if (fast) - inside some of the loops. If anyone has a clever way to optimise this, please share!
gcc will remove this if the code gets inlined which it should ... well theoretically [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB There will always be a question for which you do not know the correct awnser.
I didn't come up with a solution that doesn't either: 1. Add lots of #ifdefs uglifying the code (and preventing runtime selection), or 2. Add lots of ifs slowing down the code, or 3. Duplicate significant portions of the code.
Here are my slight changes to your mpeg12.c patches. My understanding of that file, (or how Michael N. would like to factor out the hardware accel. stuff) isn't great, but I hope this can be a start for discussion/review. In terms of the concerns above: 1) There are lots of #ifdefs, but not many more than the existing HAVE_XVMC blocks? 2) Similarly with the ifs for runtime selection. 3) I tried to move the large blocks into a separate file, but there are still lots of sections that are in similar locations to the XVMC code. I think we also need to settle on one namespace. Either DVDV or Accellent (at the moment there are both). (Note that this also includes some other MythTV additions- VIA XVMC hardware accel, and a closed captioning decoder) Index: libavcodec/mpeg12.c =================================================================== --- libavcodec/mpeg12.c (revision 6123) +++ libavcodec/mpeg12.c (working copy) @@ -33,6 +33,9 @@ //#undef NDEBUG //#include <assert.h> +#ifdef HAVE_DVDV +#include "dvdv.h" +#endif /* Start codes. */ #define SEQ_END_CODE 0x000001b7 @@ -83,6 +86,18 @@ extern void XVMC_init_block(MpegEncContext *s);//set s->block #endif +#ifdef HAVE_XVMC_VLD +extern int XVMC_VLD_field_start(MpegEncContext *s, AVCodecContext *avctx); +extern int XVMC_VLD_field_end(MpegEncContext *s); +extern int XVMC_VLD_decode_slice(MpegEncContext *s, int start_code, + uint8_t *buffer, int buf_size); +#endif + +#ifdef HAVE_DVDV +extern void DVDV_init_block(MpegEncContext *s); +extern void DVDV_decode_mb(MpegEncContext *s); +#endif + const enum PixelFormat pixfmt_yuv_420[]= {PIX_FMT_YUV420P,-1}; const enum PixelFormat pixfmt_yuv_422[]= {PIX_FMT_YUV422P,-1}; const enum PixelFormat pixfmt_yuv_444[]= {PIX_FMT_YUV444P,-1}; @@ -1124,6 +1139,11 @@ s->mb_skipped = 1; } +#ifdef HAVE_DVDV + if (s->avctx->accellent_enabled) + *DVDV_CBP(s)++ = 0; +#endif + return 0; } @@ -1198,6 +1218,10 @@ } } #endif +#ifdef HAVE_DVDV + if (s->avctx->accellent_enabled) + *DVDV_CBP(s)++ = 0x3F; +#endif if (s->codec_id == CODEC_ID_MPEG2VIDEO) { if(s->flags2 & CODEC_FLAG2_FAST){ @@ -1409,6 +1433,10 @@ } } #endif +#ifdef HAVE_DVDV + if (s->avctx->accellent_enabled) + *DVDV_CBP(s)++ = cbp; +#endif if (s->codec_id == CODEC_ID_MPEG2VIDEO) { if(s->flags2 & CODEC_FLAG2_FAST){ @@ -1456,6 +1484,10 @@ } } }else{ +#ifdef HAVE_DVDV + if (s->avctx->accellent_enabled) + *DVDV_CBP(s)++ = 0; +#endif for(i=0;i<12;i++) s->block_last_index[i] = -1; } @@ -1751,6 +1783,9 @@ const uint16_t *quant_matrix; const int qscale= s->qscale; int mismatch; +#ifdef HAVE_DVDV + int dct_count = 0; +#endif mismatch = 1; @@ -1770,6 +1805,15 @@ level= -level; block[0] = level; mismatch ^= level; +#ifdef HAVE_DVDV + if (s->avctx->accellent_enabled) + { + DVDV_DCT(s)->run_sub_one = 0; + DVDV_DCT(s)->elt = level; + DVDV_DCT(s)++; + dct_count++; + } +#endif i++; SKIP_BITS(re, &s->gb, 2); if(((int32_t)GET_CACHE(re, &s->gb)) <= (int32_t)0xBFFFFFFF) @@ -1808,6 +1852,16 @@ mismatch ^= level; block[j] = level; +#ifdef HAVE_DVDV + if (s->avctx->accellent_enabled) + { + //needn't write block at all but keep it for sanity + DVDV_DCT(s)->run_sub_one = run-1; + DVDV_DCT(s)->elt = level; + DVDV_DCT(s)++; + dct_count++; + } +#endif if(((int32_t)GET_CACHE(re, &s->gb)) <= (int32_t)0xBFFFFFFF) break; UPDATE_CACHE(re, &s->gb); @@ -1819,6 +1873,10 @@ block[63] ^= (mismatch & 1); s->block_last_index[n] = i; +#ifdef HAVE_DVDV + if (s->avctx->accellent_enabled) + DVDV_MB(s)->num_coded_elts[n] = dct_count; +#endif return 0; } @@ -1830,6 +1888,9 @@ RLTable *rl = &rl_mpeg1; uint8_t * const scantable= s->intra_scantable.permutated; const int qscale= s->qscale; +#ifdef HAVE_DVDV + int dct_count = 0; +#endif OPEN_READER(re, &s->gb); i = -1; @@ -1840,6 +1901,15 @@ if(GET_CACHE(re, &s->gb)&0x40000000) level= -level; block[0] = level; +#ifdef HAVE_DVDV + if (s->avctx->accellent_enabled) + { + DVDV_DCT(s)->run_sub_one = 0; + DVDV_DCT(s)->elt = level; + DVDV_DCT(s)++; + dct_count++; + } +#endif i++; SKIP_BITS(re, &s->gb, 2); if(((int32_t)GET_CACHE(re, &s->gb)) <= (int32_t)0xBFFFFFFF) @@ -1873,6 +1943,16 @@ } block[j] = level; +#ifdef HAVE_DVDV + if (s->avctx->accellent_enabled) + { + //needn't write block at all but keep it for sanity + DVDV_DCT(s)->run_sub_one = run-1; + DVDV_DCT(s)->elt = level; + DVDV_DCT(s)++; + dct_count++; + } +#endif if(((int32_t)GET_CACHE(re, &s->gb)) <= (int32_t)0xBFFFFFFF) break; UPDATE_CACHE(re, &s->gb); @@ -1881,6 +1961,10 @@ LAST_SKIP_BITS(re, &s->gb, 2); CLOSE_READER(re, &s->gb); s->block_last_index[n] = i; +#ifdef HAVE_DVDV + if (s->avctx->accellent_enabled) + DVDV_MB(s)->num_coded_elts[n] = dct_count; +#endif return 0; } @@ -1896,6 +1980,9 @@ const uint16_t *quant_matrix; const int qscale= s->qscale; int mismatch; +#ifdef HAVE_DVDV + int dct_count = 0; +#endif /* DC coef */ if (n < 4){ @@ -1914,6 +2001,15 @@ block[0] = dc << (3 - s->intra_dc_precision); dprintf("dc=%d\n", block[0]); mismatch = block[0] ^ 1; +#ifdef HAVE_DVDV + if (s->avctx->accellent_enabled) + { + DVDV_DCT(s)->run_sub_one = 0; + DVDV_DCT(s)->elt = block[0] - 0x0400; + DVDV_DCT(s)++; + dct_count++; + } +#endif i = 0; if (s->intra_vlc_format) rl = &rl_mpeg2; @@ -1951,17 +2047,35 @@ } if (i > 63){ av_log(s->avctx, AV_LOG_ERROR, "ac-tex damaged at %d %d\n", s->mb_x, s->mb_y); +#ifdef HAVE_DVDV + if (s->avctx->accellent_enabled) + DVDV_MB(s)->num_coded_elts[n] = dct_count; +#endif return -1; } mismatch^= level; block[j] = level; +#ifdef HAVE_DVDV + if (s->avctx->accellent_enabled) + { + //needn't write block at all but keep it for sanity + DVDV_DCT(s)->run_sub_one = run-1; + DVDV_DCT(s)->elt = level; + DVDV_DCT(s)++; + dct_count++; + } +#endif } CLOSE_READER(re, &s->gb); } block[63]^= mismatch&1; s->block_last_index[n] = i; +#ifdef HAVE_DVDV + if (s->avctx->accellent_enabled) + DVDV_MB(s)->num_coded_elts[n] = dct_count; +#endif return 0; } @@ -1975,6 +2089,9 @@ uint8_t * scantable= s->intra_scantable.permutated; const uint16_t *quant_matrix; const int qscale= s->qscale; +#ifdef HAVE_DVDV + int dct_count = 0; +#endif /* DC coef */ if (n < 4){ @@ -1991,6 +2108,15 @@ dc += diff; s->last_dc[component] = dc; block[0] = dc << (3 - s->intra_dc_precision); +#ifdef HAVE_DVDV + if (s->avctx->accellent_enabled) + { + DVDV_DCT(s)->run_sub_one = 0; + DVDV_DCT(s)->elt = block[0] - 0x0400; + DVDV_DCT(s)++; + dct_count++; + } +#endif if (s->intra_vlc_format) rl = &rl_mpeg2; else @@ -2027,11 +2153,25 @@ } block[j] = level; +#ifdef HAVE_DVDV + if (s->avctx->accellent_enabled) + { + //needn't write block at all but keep it for sanity + DVDV_DCT(s)->run_sub_one = run-1; + DVDV_DCT(s)->elt = level; + DVDV_DCT(s)++; + dct_count++; + } +#endif } CLOSE_READER(re, &s->gb); } s->block_last_index[n] = scantable - s->intra_scantable.permutated; +#ifdef HAVE_DVDV + if (s->avctx->accellent_enabled) + DVDV_MB(s)->num_coded_elts[n] = dct_count; +#endif return 0; } @@ -2174,6 +2314,9 @@ if( avctx->idct_algo == FF_IDCT_AUTO ) avctx->idct_algo = FF_IDCT_SIMPLE; + if( avctx->xvmc_vld_hwslice == 1) + avctx->idct_algo = FF_IDCT_LIBMPEG2MMX; + //quantization matrixes may need reordering //if dct permutation is changed memcpy(old_permutation,s->dsp.idct_permutation,64*sizeof(uint8_t)); @@ -2413,6 +2556,13 @@ dprintf("alternate_scan=%d\n", s->alternate_scan); dprintf("frame_pred_frame_dct=%d\n", s->frame_pred_frame_dct); dprintf("progressive_frame=%d\n", s->progressive_frame); +#ifdef HAVE_DVDV + if (s->avctx->accellent_enabled) + { + DVDV_FRAME(s)->intra_vlc_format = s->intra_vlc_format; + DVDV_FRAME(s)->alternate_scan = s->alternate_scan; + } +#endif } static void mpeg_decode_extension(AVCodecContext *avctx, @@ -2496,6 +2646,10 @@ if(s->avctx->xvmc_acceleration) XVMC_field_start(s,avctx); #endif +#ifdef HAVE_XVMC_VLD + if(s->avctx->xvmc_vld_hwslice) + XVMC_VLD_field_start(s, avctx); +#endif return 0; } @@ -2525,6 +2679,16 @@ return -1; } +#ifdef HAVE_XVMC_VLD + if (s->avctx->xvmc_vld_hwslice){ + int used = XVMC_VLD_decode_slice(s, mb_y, *buf, buf_size); + if (used < 0) + return DECODE_SLICE_ERROR; + *buf += used - 1; + return DECODE_SLICE_OK; + } +#endif + init_get_bits(&s->gb, *buf, buf_size*8); ff_mpeg1_clean_buffers(s); @@ -2583,6 +2747,10 @@ if(s->avctx->xvmc_acceleration > 1) XVMC_init_block(s);//set s->block #endif +#ifdef HAVE_DVDV + if (s->avctx->accellent_enabled) + DVDV_init_block(s); +#endif ret = mpeg_decode_mb(s, s->block); s->chroma_qscale= s->qscale; @@ -2626,7 +2794,14 @@ s->dest[1] += 16 >> (s->chroma_x_shift + lowres); s->dest[2] += 16 >> (s->chroma_x_shift + lowres); +#ifdef HAVE_DVDV + if (s->avctx->accellent_enabled) + DVDV_decode_mb(s); + else + MPV_decode_mb(s, s->block); +#else MPV_decode_mb(s, s->block); +#endif if (++s->mb_x >= s->mb_width) { const int mb_size= 16>>s->avctx->lowres; @@ -2737,6 +2912,11 @@ if(s->avctx->xvmc_acceleration) XVMC_field_end(s); #endif +#ifdef HAVE_XVMC_VLD + if(s->avctx->xvmc_vld_hwslice) + XVMC_VLD_field_end(s); +#endif + /* end of slice reached */ if (/*s->mb_y<<field_pic == s->mb_height &&*/ !s->first_field) { /* end of image */ @@ -2898,6 +3078,9 @@ if( avctx->idct_algo == FF_IDCT_AUTO ) avctx->idct_algo = FF_IDCT_SIMPLE; + if( avctx->xvmc_vld_hwslice == 1) + avctx->idct_algo = FF_IDCT_LIBMPEG2MMX; + if (MPV_common_init(s) < 0) return -1; exchange_uv(s);//common init reset pblocks, so we swap them here @@ -3306,6 +3489,85 @@ #endif +#ifdef HAVE_XVMC_VLD +static int mpeg_xvmc_vld_decode_init(AVCodecContext *avctx){ + Mpeg1Context *s; + + if( avctx->thread_count > 1) + return -1; + if( !(avctx->slice_flags & SLICE_FLAG_CODED_ORDER) ) + return -1; + if( !(avctx->slice_flags & SLICE_FLAG_ALLOW_FIELD) ) + dprintf("mpeg12.c: XVMC_VLD decoder will work better if SLICE_FLAG_ALLOW_FIELD is set\n"); + + mpeg_decode_init(avctx); + s = avctx->priv_data; + + avctx->xvmc_vld_hwslice = 1; + + return 0; +} + +AVCodec mpeg_xvmc_vld_decoder = { + "mpegvideo_xvmc_vld", + CODEC_TYPE_VIDEO, + CODEC_ID_MPEG2VIDEO_XVMC_VLD, + sizeof(Mpeg1Context), + mpeg_xvmc_vld_decode_init, + NULL, + mpeg_decode_end, + mpeg_decode_frame, + CODEC_CAP_DRAW_HORIZ_BAND | CODEC_CAP_DR1 | CODEC_CAP_TRUNCATED| CODEC_CAP_HWACCEL | CODEC_CAP_DELAY, + .flush= ff_mpeg_flush, +}; + +#endif + +#ifdef HAVE_DVDV +static int mpeg_accellent_decode_init(AVCodecContext *avctx){ + Mpeg1Context *s; + // We only try to decode one stream at once: + if( avctx->thread_count > 1) + return -1; + + // We cannot handle non-ordered slices? + //if( !(avctx->slice_flags & SLICE_FLAG_CODED_ORDER) ) + //return -1; + + //if( !(avctx->slice_flags & SLICE_FLAG_ALLOW_FIELD) ) + //dprintf("mpeg12.c: Accellent decoder will work better if SLICE_FLAG_ALLOW_FIELD is set\n"); + + mpeg_decode_init(avctx); + s = avctx->priv_data; + + avctx->accellent = av_malloc(sizeof(DVDV_CurPtrs)); + if (!avctx->accellent) + return -1; + + // Indicate to callers that we are initialised? + avctx->accellent_enabled = 1; + + // For now, don't actually do anything? + //avctx->parse_only = 1; + + return 0; +} + +AVCodec mpeg_accellent_decoder = { + "mpegvideo_accellent", + CODEC_TYPE_VIDEO, + CODEC_ID_MPEG2VIDEO, // Do we need a custom codec type? + sizeof(Mpeg1Context), + mpeg_accellent_decode_init, + NULL, + mpeg_decode_end, + mpeg_decode_frame, + CODEC_CAP_DRAW_HORIZ_BAND | CODEC_CAP_DR1 | CODEC_CAP_TRUNCATED| CODEC_CAP_HWACCEL | CODEC_CAP_DELAY, + .flush= ff_mpeg_flush, +}; + +#endif + /* this is ugly i know, but the alternative is too make hundreds of vars global and prefix them with ff_mpeg1_ which is far uglier. */ Index: libavcodec/dvdv.c =================================================================== --- libavcodec/dvdv.c (revision 0) +++ libavcodec/dvdv.c (revision 0) @@ -0,0 +1,66 @@ +/* + * dvdv.c - Mac Hardware MPEG2 decoding. + * + * Based on modifications to ffmpeg's libavcodec/mpeg12.c by John Dagliesh. + * See http://www.defyne.org/dvb/accellent.html + * + * TO be added to the MythTV project's copy of libavcodec? + */ + +#include "avcodec.h" +#include "mpegvideo.h" +#include "dvdv.h" + +void DVDV_init_block(MpegEncContext *s) +{ + DVDV_MBInfo * mb = DVDV_MB(s); + int ret; + + for (ret=0; ret<6; ret++) + mb->num_coded_elts[ret] = 0; +} + +void DVDV_decode_mb(MpegEncContext *s) +{ + DVDV_MBInfo * mb = DVDV_MB(s); + + mb->mb_type = 0; + if (!s->mb_intra) + { + if (s->mv_dir & MV_DIR_FORWARD) + { + mb->mb_type |= 1; + mb->frwd_mv[0] = s->mv[0][0][0]; + mb->frwd_mv[1] = s->mv[0][0][1]; + mb->frwd_mv_field[0] = s->mv[0][1][0]; + mb->frwd_mv_field[1] = s->mv[0][1][1]; + } + if (s->mv_dir & MV_DIR_BACKWARD) + { + mb->mb_type |= 2; + mb->back_mv[0] = s->mv[1][0][0]; + mb->back_mv[1] = s->mv[1][0][1]; + mb->back_mv_field[0] = s->mv[1][1][0]; + mb->back_mv_field[1] = s->mv[1][1][1]; + } + if (s->mv_type != MV_TYPE_16X16) + { + mb->mb_type |= 4; + mb->field_select[0] = s->field_select[0][0]; + mb->field_select[1] = s->field_select[1][0]; + mb->field_select[2] = s->field_select[0][1]; + mb->field_select[3] = s->field_select[1][1]; + } + } + mb->ildct = s->interlaced_dct; + DVDV_MB(s)++; + + // side effects of MPV_decode_mb: + if (!s->mb_intra) + { + s->last_dc[0] = + s->last_dc[1] = + s->last_dc[2] = 128 << s->intra_dc_precision; + } + s->mb_skipped = 0; +} Property changes on: libavcodec/dvdv.c ___________________________________________________________________ Name: svn:keywords + Date Revision Author Name: svn:eol-style + native Index: libavcodec/dvdv.h =================================================================== --- libavcodec/dvdv.h (revision 0) +++ libavcodec/dvdv.h (revision 0) @@ -0,0 +1,64 @@ +/* + * This header is adapted from Accellent by John Dagliesh: + * http://www.defyne.org/dvb/accellent.html + */ + +#ifndef __FF_DVDV__ +#define __FF_DVDV__ + +/* +Header file containing structures for writing data to Apple's +private hardware MPEG2 decoding API, that I'm calling DVDVideo +*/ + +#ifdef __cplusplus +extern "C" { +#endif + +typedef struct DVDV_MBInfo +{ + int16_t frwd_mv[2]; + int16_t back_mv[2]; + int16_t frwd_mv_field[2]; + int16_t back_mv_field[2]; + + uint8_t field_select[4]; // 0/1 in order of mvs above + + uint8_t mb_type; // |1 = frwd, |2 = back, |4 = use fields + uint8_t ildct; // 0/1 + + uint8_t num_coded_elts[6]; // for each block incl. dc +} DVDV_MBInfo; + +typedef struct DVDV_DCTElt +{ + uint8_t run_sub_one; + uint8_t unused; + uint16_t elt; // adjusted for first coded elt in block: +} DVDV_DCTElt; // intra sub 0x0400, else add 0x0400 + +typedef struct DVDV_Frame +{ + int intra_vlc_format; + int alternate_scan; +} DVDV_Frame; + +typedef struct DVDV_CurPtrs +{ + DVDV_MBInfo * mb; + DVDV_DCTElt * dct; + uint8_t * cbp; + DVDV_Frame * frame; +} DVDV_CurPtrs; + +// Convenience macro to get the above from an MpegEncContext +#define DVDV_MB(x) (((DVDV_CurPtrs *)(x->avctx->accellent))->mb) +#define DVDV_DCT(x) (((DVDV_CurPtrs *)(x->avctx->accellent))->dct) +#define DVDV_CBP(x) (((DVDV_CurPtrs *)(x->avctx->accellent))->cbp) +#define DVDV_FRAME(x) (((DVDV_CurPtrs *)(x->avctx->accellent))->frame) + +#ifdef __cplusplus +} +#endif + +#endif // __FF_DVDV__ Property changes on: libavcodec/dvdv.h ___________________________________________________________________ Name: svn:keywords + Date Revision Author Name: svn:eol-style + native Index: libavcodec/Makefile =================================================================== --- libavcodec/Makefile (revision 6123) +++ libavcodec/Makefile (working copy) @@ -281,6 +281,8 @@ OBJS-$(HAVE_XVMC_ACCEL) += xvmcvideo.o +OBJS-$(HAVE_MAC_ACCELLENT) += dvdv.o + ifneq ($(CONFIG_SWSCALER),yes) OBJS += imgresample.o endif Index: libavcodec/avcodec.h =================================================================== --- libavcodec/avcodec.h (revision 6123) +++ libavcodec/avcodec.h (working copy) @@ -32,6 +32,7 @@ CODEC_ID_MPEG1VIDEO, CODEC_ID_MPEG2VIDEO, /* prefered ID for MPEG Video 1 or 2 decoding */ CODEC_ID_MPEG2VIDEO_XVMC, + CODEC_ID_MPEG2VIDEO_XVMC_VLD, CODEC_ID_H261, CODEC_ID_H263, CODEC_ID_RV10, @@ -627,7 +628,15 @@ * - encoding: set by user\ * - decoding: set by lavc\ */\ - int8_t *ref_index[2]; + int8_t *ref_index[2];\ +\ + /** ATSC CC data\ + * - encoding: unused\ + * - decoding: set by lavc\ + */\ + uint8_t atsc_cc_buf[1024];\ + int atsc_cc_len;\ +\ #define FF_QSCALE_TYPE_MPEG1 0 #define FF_QSCALE_TYPE_MPEG2 1 @@ -2024,6 +2033,29 @@ * - decoding: unused. */ int max_partition_order; + + /** + * XVMC_VLD (VIA CLE266) Hardware MPEG decoding + * - encoding: forbidden + * - decoding: set by decoder + */ + int xvmc_vld_hwslice; + + /** + * Closed Caption decoder + * - encoding: forbidden + * - decoding: set by decoder + */ + void (*decode_cc_dvd)(struct AVCodecContext *c, const uint8_t *buf, int buf_size); + /** + * Mac OS X Hardware MPEG decoding - Accellent by John Dagliesh: + * http://www.defyne.org/dvb/accellent.html + * - encoding: forbidden + * - decoding: set by decoder + */ + int accellent_enabled; + void *accellent; // This is actually a pointer to a DVDV_CurPtrs + } AVCodecContext; /** @@ -2158,6 +2190,8 @@ extern AVCodec mpeg2video_decoder; extern AVCodec mpegvideo_decoder; extern AVCodec mpeg_xvmc_decoder; +extern AVCodec mpeg_xvmc_vld_decoder; +extern AVCodec mpeg_accellent_decoder; extern AVCodec h263i_decoder; extern AVCodec flv_decoder; extern AVCodec rv10_decoder; @@ -2631,6 +2665,11 @@ extern unsigned int av_xiphlacing(unsigned char *s, unsigned int v); +void avcodec_default_decode_cc_dvd(AVCodecContext *c, const uint8_t *buf, int buf_size); +void avcodec_default_decode_cc_atsc(AVCodecContext *c, const uint8_t *buf, int buf_size); +const char *codec_id_string(enum CodecID codec_id); +const char *codec_type_string(enum CodecType codec_type); + #ifdef __cplusplus } #endif Index: configure =================================================================== --- configure (revision 19589) +++ configure (working copy) @@ -1700,6 +1700,7 @@ _macosx=auto _macosx_finder_support=no _macosx_bundle=auto +_accellent=no _sortsub=yes _freetypeconfig='freetype-config' _fribidi=auto @@ -3533,7 +3534,7 @@ _aosrc="$_aosrc ao_macosx.c" _aomodules="macosx $_aomodules" _vosrc="$_vosrc vo_quartz.c" - _vomodules="quartz $_vomodules" + _vomodules="quartz accellent $_vomodules" else _macosx=no _def_macosx='#undef MACOSX' @@ -3550,6 +3551,8 @@ _vomodules="macosx $_vomodules" _macosx_frameworks="$_macosx_frameworks -framework Cocoa -framework QuartzCore -framework OpenGL" _def_macosx_corevideo='#define MACOSX_COREVIDEO 1' + _def_macosx_accellent='#define HAVE_DVDV 1' + _accellent=yes _macosx_corevideo=yes else _novomodules="macosx $_novomodules" @@ -7380,6 +7383,8 @@ HAVE_XVMC_ACCEL = $_xvmc +HAVE_MAC_ACCELLENT = $_accellent + # for FFmpeg SRC_PATH=.. LIBPREF=lib @@ -7834,6 +7839,7 @@ $_def_macosx_finder_support $_def_macosx_bundle $_def_macosx_corevideo +$_def_macosx_accellent /* Build our Win32-loader */ $_def_win32_loader -- Nigel Pearson, nigel at ind.tansu.com.au| "They did the Telstra Net. Eng., Sydney, Australia | white man power dance, Office: 9202 3900 Fax: 9261 3912 | and ... Shazam" Mobile: 0408 664435 Home: 9792 6998 | Lois Lane - Smallville
Hi Nigel, On Wed, 30 Aug 2006, Nigel Pearson wrote:
I didn't come up with a solution that doesn't either: 1. Add lots of #ifdefs uglifying the code (and preventing runtime selection), or 2. Add lots of ifs slowing down the code, or 3. Duplicate significant portions of the code.
Here are my slight changes to your mpeg12.c patches.
They are a lot more than slight! Maybe they are slight vs someone else's modification of the patch.
My understanding of that file, (or how Michael N. would like to factor out the hardware accel. stuff) isn't great, but I hope this can be a start for discussion/review.
I don't think it will be reviewed unless you at least follow Michael's suggestion of collapsing the _fast methods. Using diff it should not be hard to merge them correctly.
In terms of the concerns above: 1) There are lots of #ifdefs, but not many more than the existing HAVE_XVMC blocks? 2) Similarly with the ifs for runtime selection.
But they are in a (more) inner loop.
3) I tried to move the large blocks into a separate file, but there are still lots of sections that are in similar locations to the XVMC code.
IMHO the two fns you moved aren't large enough to warrant this.
I think we also need to settle on one namespace. Either DVDV or Accellent (at the moment there are both).
I never used 'accellent' in my ffmpeg patch, someone else must have done that. Accellent is just the silly name of the demo app, DVDV(ideo) is the correct name for the acceleration method.
(Note that this also includes some other MythTV additions- VIA XVMC hardware accel, and a closed captioning decoder)
A patch with multiple unrelated changes like this will never be accepted. I know you're just putting it out there for discussion, but you should make ppl's life easier if you can. Other quick comments: - The dct elts should not be set into the blocks for sanity in real code; that was just for testing. - The way the stream pointers are accessed appears inefficient. - Don't need two vars in the avctx, just have the pointer NULL if not enabled. {P^/
Thanks for the quick review, John!
2) Similarly with the ifs for runtime selection.
But they are in a (more) inner loop.
Aah. Yes. Good point. ...
I think we also need to settle on one namespace. Either DVDV or Accellent (at the moment there are both).
I never used 'accellent' in my ffmpeg patch, someone else must have done that.
Probably me. The changes in MythTV go well beyond just libavcodec (to configure, and all the calling code), so I think the name crept in there as well, and then down into the lib.
Accellent is just the silly name of the demo app, DVDV(ideo) is the correct name for the acceleration method.
OK. I will start correcting this. Short is good.
- Don't need two vars in the avctx, just have the pointer NULL if not enabled.
I wasn't sure if the DVDV stuff allowed multiple clients (e.g. MythTV does PiP) so I wanted a variable that could be incremented for each simultaneous client. -- Nigel Pearson, nigel at ind.tansu.com.au|"Things you own Telstra Net. Eng., Sydney, Australia | end up owning you" Office: 9202 3900 Fax: 9261 3912 | Tyler, Mobile: 0408 664435 Home: 9792 6998 | Fight Club
On Wed, 30 Aug 2006, Nigel Pearson wrote:
- Don't need two vars in the avctx, just have the pointer NULL if not enabled.
I wasn't sure if the DVDV stuff allowed multiple clients (e.g. MythTV does PiP) so I wanted a variable that could be incremented for each simultaneous client.
Yes you do need to put the pointer to the DVDV streams in the context (or somewhere - definitely not a global like I had it for testing) that's good. I just don't think you need more than one such addition. If the user sets the pointer to non-NULL then that will do fine to indicate that dvdv acceleration should be used for that context. {P^/
At 04:36 28/08/2006, you wrote:
Hello,
I've been a little bit involved with an interesting project called Accellent which is a libavcodec-based reverse-engineer of Mac OS X's MPEG-2 hardware acceleration API. Apple refused to release any data
.........
erest in working further on this. It would be interesting if the functionality could somehow be integrated into libavcodec, or perhaps just simply an effective player be put together. There are millions of users who could benefit!
Well, not only MPEG-2, you can use the GPU for accelerate almost any mathematical function. For example, check, this pdf http://www.vis.uni-stuttgart.de/ger/research/pub/pub2000/VisSym00hopf.pdf#se... where a method for calculate wavelet using OpenGL is showed (note, only mathematical and some timings, not code). Using GLSL (OpenGL shading language) you can make a GPU program that can calculate the wavelet of any image. At http://www.cse.cuhk.edu.hk/~ttwong/demo/dwtgpu/dwtgpu.html you have opensource c++ code for implement DWT on Nvidias GPU and a JPEG2000 library, jasper, that uses it. HTH
participants (11)
-
baptiste.coudurier＠smartjog.com -
dalias＠aerifal.cx -
galenz＠zinkconsulting.com -
gpoirier＠mplayerhq.hu -
johnd＠defyne.org -
michaelni＠gmx.at -
mru＠inprovide.com -
nec556＠retena.com -
nigel＠ind.tansu.com.au -
robert.swain＠gmail.com -
snacky＠ikaruga.co.uk