Hi On Tuesday 05 July 2005 10:17, Andy Parkins wrote: [...]

I don't know for sure is my answer. However, I am having success treating the units to av_seek_frame() as being in AV_TIME_BASE units (at present this is a 64 bit integer representing time in microseconds passed AVFormatContext.start_time)

yes it uses AV_TIME_BASE units if no stream is specified, otherwise the timebase of the stream is used

As to seeking to a particular (non-key) frame, I believe the suggestion from Rich Felker was that you seek to the nearest key frame using av_seek_frame() then enable hurry_up mode until you hit the frame you want. e.g. (error checking and initialisation ommitted)

av_seek_frame( fmtCtx, -1, TARGET_PTS, AVSEEK_FLAG_BACKWARD ); CodecCtx->hurry_up = 1; do { av_read_frame( fmtCtx, &Packet ); // should really be checking that this is a video packet MyPts = Packet.pts / Packet.duration * AV_TIME_BASE / av_q2d( Stream->r_frame_rate);

this is completely wrong, the division by Packet.duration makes no sens and its 1/Stream.time_base not Stream->r_frame_rate which should be used [...] -- Michael

On Tue, Jul 05, 2005 at 11:25:05AM +0100, Andy Parkins wrote:

This obviously still contains the dodgy " / Packet.duration"; which I'm using to create a frame counter - is there a frame count already in the packet?

This is a perfect frame counter for fixed-framerate files. For files without a framerate, there is no frame counter available, naturally, since there's no way to know a frame number without counting manually from the beginning. If your program decodes the whole stream without ever seeking you can just keep this counter yourself... Rich

Hi On Tuesday 05 July 2005 12:36, Andy Parkins wrote:

On Tuesday 2005 July 05 10:14, Michael Niedermayer wrote:

...

...

av_seek_frame( fmtCtx, -1, TARGET_PTS, AVSEEK_FLAG_BACKWARD ); CodecCtx->hurry_up = 1; do { av_read_frame( fmtCtx, &Packet ); // should really be checking that this is a video packet MyPts = Packet.pts / Packet.duration * AV_TIME_BASE / av_q2d( Stream->r_frame_rate);

this is completely wrong, the division by Packet.duration makes no sens and its 1/Stream.time_base not Stream->r_frame_rate which should be used

How's this?

MyPts = av_rescale( Packet.pts, AV_TIME_BASE * (int64_t) Stream->time_base.num, Stream->time_base.den );

looks good [...] -- Michael

Hi Ian, I think what Rich is saying is that there is no formula for accurately mapping a time to a frame number for all formats, because some formats do not have a constant frame rate. I ran into this issue myself using the Java Media Framework (JMF), which I'm much more familiar with. It is better to refer to timestamps rather than frames, because there is always a frame to fetch for a given timestamp. If you try to fetch a frame by sequence number, you have to count from the beginning to be sure its correct for all formats, and that gets messy if you need to seek. With this in mind, I am looking for the "best practice" way to do just that...fetch a frame for a given time. This is proving difficult for me, because the library can only seek to the nearest keyframe. I need the exact frame for a given time...not just the nearest key frame. I'm trying out the suggestions I've been given on this list, and I'll post my progress if that helps you. It could be another week, though. I'm a Java programmer, and I'm struggling to understand both the avformat/avcodec library and also C/C++ at large. Oh yeah, I'm also using Windows! So far, I'm successfully fetching one frame at a time in sequence as an RGB image (which is what I need), so I just have to figure out an efficient way of random-access into the stream. By the way, if you are certain that all the formats you will use will be constant frame rate, the timestamp method can still help you: frame_number = frame_rate * time ...where frame_number and time are in the same units, such as seconds or ms. Best regards, Steve Ian Gowen wrote:

While providing lots of useful info, this still doesn't address my original question: given the frame number, what is the formula to find the correct timestamp for a particular stream? (i.e. in AVStream.time_base units, not AV_TIME_BASE).

--Ian

On 7/5/05, Michael Niedermayer <michaelni at gmx.at> wrote:

...

Hi

On Tuesday 05 July 2005 12:36, Andy Parkins wrote:

...

On Tuesday 2005 July 05 10:14, Michael Niedermayer wrote:

...

...

av_seek_frame( fmtCtx, -1, TARGET_PTS, AVSEEK_FLAG_BACKWARD ); CodecCtx->hurry_up = 1; do { av_read_frame( fmtCtx, &Packet ); // should really be checking that this is a video packet MyPts = Packet.pts / Packet.duration * AV_TIME_BASE / av_q2d( Stream->r_frame_rate);

this is completely wrong, the division by Packet.duration makes no sens and its 1/Stream.time_base not Stream->r_frame_rate which should be used

How's this?

MyPts = av_rescale( Packet.pts, AV_TIME_BASE * (int64_t) Stream->time_base.num, Stream->time_base.den );

looks good

[...] -- Michael

_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel at mplayerhq.hu http://mplayerhq.hu/mailman/listinfo/ffmpeg-devel

_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel at mplayerhq.hu http://mplayerhq.hu/mailman/listinfo/ffmpeg-devel

On Wednesday 2005 July 06 07:10, Ian Gowen wrote:

Thus, time = frame_number / frame_rate. However, what I don't understand is how to put that into AVStream.time_base units, as the docs say to do. Do I multiply it by the time_base? Divide? I don't get it, and I'm pretty sure this is where I'm screwing up.

I'm not sure why you would need to do this. What source for frame_number are you using - it's not in any of the AV structures. If you want the time simply read Packet.pts; why calculate it when it's already there? If you are trying to seek, you would want to seek a time anyway. Maybe a percentage through the file? ffplay does this like this: seekPoint = (int64_t)(formatContext->start_time+frac*formatContext->duration); Andy -- Dr Andrew Parkins, M Eng (hons), AMIEE

On Wednesday 2005 July 06 14:27, Steve Willis wrote:

Let's say I want to seek to a frame that is 14.25 seconds into a stream. Would someone please post actual code showing (1) how to calculate the value for 14.25 seconds to pass to av_seek_frame to seek to the nearest key frame (Ian's question), and (2) a recommended way to actually seek to the precise frame for 14.25 seconds, regardless of whether it is a keyframe (my question).

My example code from earlier in this thread, with the correction suggested by Michael... // normal frame loop av_read_frame() done above here if( seekRequested ) { TARGET_PTS = 14.25 * AV_TIME_BASE; av_seek_frame( fmtCtx, -1, TARGET_PTS, AVSEEK_FLAG_BACKWARD ); CodecCtx->hurry_up = 1; do { av_read_frame( fmtCtx, &Packet ); // should really be checking that this is a video packet MyPts = av_rescale( Packet.pts, AV_TIME_BASE * (int64_t) Stream->time_base.num, Stream->time_base.den ); // Once we pass the target point, break from the loop if( MyPts >= TARGET_PTS ) break; avcodec_decode_video( CodecCtx, pFrame, &gotFrame, Packet.data, Packet.size ); av_free_packet( &Packet ); } while(1); CodecCtx->hurry_up = 0; } // Continue with normal frame loop (i.e. decode, show and free) The documentation for av_seek_frame() (and the code) says that if the stream is not given (i.e. -1) then AV_TIME_BASE units are used, otherwise the time base for the particular stream is used - in which case you would need to rescale TARGET_PTS to stream units before calling it. This would be done as the inverse of the MyPts calculation above. Looking at the code again, it would also be better to use the stream time base throughout, which would then save the repeated call to av_rescale() every loop. I've only recently started working with the libav* libraries myself, but have found that everything I want is there; it's finding the thing that's the trick :-) Andy -- Dr Andrew Parkins, M Eng (hons), AMIEE andyparkins at gmail.com

On 7/6/05, Andy Parkins <andyparkins at gmail.com> wrote:

I'm not sure why you would need to do this. What source for frame_number are you using - it's not in any of the AV structures.

I'm working on a video editing application. If the user wants to preview frame 320, I need it to seek to frame 320, not the nearest keyframe or a frame in the general vicinity of frame 320. Anyway, I've seen a lot of helpful suggestions here so far. I'll try rescaling the calculated timestamp by the time base and check back here if it doesn't work. thanks --Ian

Nathan Kurz wrote:

On Wed, Jul 06, 2005 at 12:04:44PM -0700, Ian Gowen wrote:

...

On 7/6/05, Andy Parkins <andyparkins at gmail.com> wrote:

...

I'm not sure why you would need to do this. What source for frame_number are you using - it's not in any of the AV structures.

I'm working on a video editing application. If the user wants to preview frame 320, I need it to seek to frame 320, not the nearest keyframe or a frame in the general vicinity of frame 320.

Not to be too discouraging, but you should also know that seek is badly broken for many formats. It's quite possible that you are doing things correctly, but are being bitten by bugs. If you search the archives from (I think) the end of last year you'll find a fairly elaborate test program I wrote to test which formats work. I had submitted it as a regression test, but as it pointed out new bugs rather than regressions it wasn't included in the test suite.

Nathan Kurz nate at verse.com

_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel at mplayerhq.hu http://mplayerhq.hu/mailman/listinfo/ffmpeg-devel

It sounds like Ian and I are both working on applications that need precise frame positioning. In light of what Nathan said above, can anyone offer recommendations for the best way to ensure that the user is looking at the frame he or she thinks they should be looking at? Is there a reliable way to detect when seek has gone wrong ("I expected to be looking at the frame for time 13.125 seconds, but I actually got the frame for time X...error!")? In response to my original question a couple of weeks ago, someone suggested using a format like MJPEG that treats all frames as keyframes. If there is not a more general solution for precicely positioning arbitrarily-formatted streams, could someone give me an expert opinion on the best format to transcode all input videos into? The videos I am dealing with are printed from high speed film (1000 fps) for scientific analysis frame-by-frame. I'm looking for a reasonable balance between decoding speed and size. Image quality is of course important for seeing detail in each frame of interest. Most important is that the frame shown is actually correct for the time in question to match up with external sensor data, so a format that does not use keyframes would also be important. Thanks, Steve

Gary Corcoran wrote:

Steve Willis wrote:

...

Nathan Kurz wrote:

...

On Wed, Jul 06, 2005 at 12:04:44PM -0700, Ian Gowen wrote:

...

On 7/6/05, Andy Parkins <andyparkins at gmail.com> wrote:

...

I'm not sure why you would need to do this. What source for frame_number are you using - it's not in any of the AV structures.

I'm working on a video editing application. If the user wants to preview frame 320, I need it to seek to frame 320, not the nearest keyframe or a frame in the general vicinity of frame 320.

Not to be too discouraging, but you should also know that seek is badly broken for many formats. It's quite possible that you are doing things correctly, but are being bitten by bugs. If you search the archives from (I think) the end of last year you'll find a fairly elaborate test program I wrote to test which formats work. I had submitted it as a regression test, but as it pointed out new bugs rather than regressions it wasn't included in the test suite.

Nathan Kurz nate at verse.com

_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel at mplayerhq.hu http://mplayerhq.hu/mailman/listinfo/ffmpeg-devel

It sounds like Ian and I are both working on applications that need precise frame positioning. In light of what Nathan said above, can anyone offer recommendations for the best way to ensure that the user is looking at the frame he or she thinks they should be looking at? Is there a reliable way to detect when seek has gone wrong ("I expected to be looking at the frame for time 13.125 seconds, but I actually got the frame for time X...error!")?

In response to my original question a couple of weeks ago, someone suggested using a format like MJPEG that treats all frames as keyframes. If there is not a more general solution for precicely positioning arbitrarily-formatted streams, could someone give me an expert opinion on the best format to transcode all input videos into? The videos I am dealing with are printed from high speed film (1000 fps) for scientific analysis frame-by-frame. I'm looking for a reasonable balance between decoding speed and size. Image quality is of course important for seeing detail in each frame of interest. Most important is that the frame shown is actually correct for the time in question to match up with external sensor data, so a format that does not use keyframes would also be important.

Couldn't you convert it to MPEG, specifying that only I-frames (keyframes) be used? Then the "seek only directly works to I-frames" feature wouldn't be a problem...

I don't think MPEG supports arbitrary frame rates. The FAQ on the ffmpeg Web site mentions that only a small number of frame rates are allowed. The videos are printed from film. They are usually in the ballpark of 1000 fps. No player can realistically play them at this speed, but they work very well for frame-by-frame analysis. One other requirement for a format I should mention is the ability to encode an arbitrary number of video streams. Some of these videos contain multiple video tracks with different angles of the same event. I need a format that lets me preserve this in a single file. Thanks for your suggestion, and please, let me know if I'm wrong about MPEG and framerates! Steve

Gary Corcoran wrote:

Steve Willis wrote:

...

Gary Corcoran wrote:

...

Steve Willis wrote:

...

Nathan Kurz wrote:

...

On Wed, Jul 06, 2005 at 12:04:44PM -0700, Ian Gowen wrote:

...

On 7/6/05, Andy Parkins <andyparkins at gmail.com> wrote:

> I'm not sure why you would need to do this. What source for > frame_number are > you using - it's not in any of the AV structures.

I'm working on a video editing application. If the user wants to preview frame 320, I need it to seek to frame 320, not the nearest keyframe or a frame in the general vicinity of frame 320.

Not to be too discouraging, but you should also know that seek is badly broken for many formats. It's quite possible that you are doing things correctly, but are being bitten by bugs. If you search the archives from (I think) the end of last year you'll find a fairly elaborate test program I wrote to test which formats work. I had submitted it as a regression test, but as it pointed out new bugs rather than regressions it wasn't included in the test suite.

Nathan Kurz nate at verse.com

_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel at mplayerhq.hu http://mplayerhq.hu/mailman/listinfo/ffmpeg-devel

It sounds like Ian and I are both working on applications that need precise frame positioning. In light of what Nathan said above, can anyone offer recommendations for the best way to ensure that the user is looking at the frame he or she thinks they should be looking at? Is there a reliable way to detect when seek has gone wrong ("I expected to be looking at the frame for time 13.125 seconds, but I actually got the frame for time X...error!")?

In response to my original question a couple of weeks ago, someone suggested using a format like MJPEG that treats all frames as keyframes. If there is not a more general solution for precicely positioning arbitrarily-formatted streams, could someone give me an expert opinion on the best format to transcode all input videos into? The videos I am dealing with are printed from high speed film (1000 fps) for scientific analysis frame-by-frame. I'm looking for a reasonable balance between decoding speed and size. Image quality is of course important for seeing detail in each frame of interest. Most important is that the frame shown is actually correct for the time in question to match up with external sensor data, so a format that does not use keyframes would also be important.

Couldn't you convert it to MPEG, specifying that only I-frames (keyframes) be used? Then the "seek only directly works to I-frames" feature wouldn't be a problem...

I don't think MPEG supports arbitrary frame rates. The FAQ on the ffmpeg Web site mentions that only a small number of frame rates are allowed. The videos are printed from film. They are usually in the ballpark of 1000 fps. No player can realistically play them at this speed, but they work very well for frame-by-frame analysis. One other requirement for a format I should mention is the ability to encode an arbitrary number of video streams. Some of these videos contain multiple video tracks with different angles of the same event. I need a format that lets me preserve this in a single file.

Thanks for your suggestion, and please, let me know if I'm wrong about MPEG and framerates!

Well, since it sounds like you do NOT require realtime playback of the video - you want to edit and/or look frame-by-frame, right? - then who cares what framerate the MPEG is? That is, couldn't you just convert every frame of your original video to MPEG at say 30 fps, and then edit/examine frames? Sure if you tried to play it realtime it would be 33 times too slow - but wouldn't that be an advantage for scientific analysis? :) Then after editing you could always convert to some other format that supported 1000fps (but isn't seekable), if that's needed.

But I don't know anything about encoding multiple video streams in a single file...

Of course you are right. That was stupid of me. As to encoding multiple streams, I believe MPEG-2 is an excellent choice. If I recall my course in multimedia information systems, MPEG-2 improved upon the 3-stream model of MPEG-1 (video, audio, system) by allowing multiple streams of arbitrary type to be interleaved. I'll have to verify this, though (comments or corrections are much appreciated.) If I can't find a reliable way to position media in other formats to precise frames, I'll probably go this route. I don't know if this helps Ian, though. For editing, he probably won't want the quality loss of converting to MPEG (lossy) then back to another format. Does anyone have any insight on how "professional" video editing programs like Adobe Premiere allow users to treat any frame as a keyframe? Steve

Hi, I'm working on an video editing application, and I notice that using av_seek_frame() with AVSEEK_FLAG_BACKWARD on MPEG format don't gives the same result than on AVI format. On AVI it seek on the nearest key-frame, but on MPEG it seems that it don't seek on key-frame. Can I have more explaination on this please. Thanks