[PATCH] api-example for libavfilter
Hi. The attached patch adds an api-example.c file to libavfilter, to help application programmers get a quick start on how to use the new library. This file imitates what is done inside ffmpeg.c itself; I would like confirmation that everything is done the preferred way though. Regards, -- Nicolas George
On date Wednesday 2010-12-08 12:40:23 +0100, Nicolas George encoded:
Hi.
The attached patch adds an api-example.c file to libavfilter, to help application programmers get a quick start on how to use the new library.
This file imitates what is done inside ffmpeg.c itself; I would like confirmation that everything is done the preferred way though.
Regards,
-- Nicolas George
libavfilter/Makefile | 4 + libavfilter/api-example.c | 227 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 231 insertions(+), 0 deletions(-)
diff --git a/libavfilter/Makefile b/libavfilter/Makefile index aece3ab..3c804cb 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -56,4 +56,8 @@ OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o
DIRS = x86
+EXAMPLES = api + include $(SUBDIR)../subdir.mak + +$(SUBDIR)api-example$(EXESUF): ELIBS = -lavformat -lavcodec -lavcore -lavutil $(FFEXTRALIBS) diff --git a/libavfilter/api-example.c b/libavfilter/api-example.c new file mode 100644 index 0000000..878d7cb --- /dev/null +++ b/libavfilter/api-example.c @@ -0,0 +1,227 @@ +/* + * API example for libavfilter + * Copyright (c) 2010 Nicolas George + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#define _XOPEN_SOURCE 600 /* for usleep */
+#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h>
These headers shouldn't be required.
+#include <libavformat/avformat.h> +#include <libavcodec/avcodec.h>
I'd prefer to avoid to rely on other FFmpeg libraries, currently lavfi only depends on libavutil, libavcore and libswscale, and API example should be minimal so should be as strict as possible in terms of requirements. Also consider that it is possible to install and compile libavfilter without lavc/lavf, so I'd like to make the API example working even in that case. Also the code could be simpler if you implement it this way... implement a video sink for printing to console a textual representation of the image (which could be useful even in libavfilter), and let the user specify the input for the input source in the filtergraph description.
+#include <libavfilter/avfiltergraph.h> +#include <libavfilter/vsrc_buffer.h> + +const char *filter_descr = "scale=78:24,format=gray"; + +static AVFormatContext *avf; +static AVCodecContext *video_dec; +AVFilterContext *video_in_filter; +AVFilterContext *video_out_filter; +AVFilterGraph *filter_graph; +static int video_stream = -1; +static int64_t last_pts = AV_NOPTS_VALUE; + +static void fatal_libav_error(const char *tag, int r) +{ + char buf[1024]; + + av_strerror(r, buf, sizeof(buf)); + fprintf(stderr, "%s: %s\n", tag, buf); + exit(1); +}
you can use the log system (with the fancy coloring support). [...] -- FFmpeg = Funny & Fostering Multimedia Purposeless Ecumenical Glue
L'octidi 18 frimaire, an CCXIX, Stefano Sabatini a ?crit?:
+#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> These headers shouldn't be required.
string.h was forgotten from a previous version. unistd.h is necessary for usleep, compilation breaks without it. stdio.h and stdlib.h are indirectly included by libav* headers, but AFAIK, that fact is not explicitly stated, and that could possibly change.
I'd prefer to avoid to rely on other FFmpeg libraries, currently lavfi only depends on libavutil, libavcore and libswscale, and API example should be minimal so should be as strict as possible in terms of requirements.
I think that illustrating how the filtering system interacts with the rest of the libraries is important: which time base should be used, how to insert an AVFrame in the system, etc. See libavformat/output-example.c: to give a complete working example, before outputing with libavformat, it scales with libswscale and encodes with libavcodec.
Also the code could be simpler if you implement it this way... implement a video sink
This is a point that I wanted clarified when I started writing this program: it seems that anyone who wants to use libavfilter is almost expected to write its own designated sink. Is it on purpose? Should there not be some kind of "buffer" sink just like there is a "buffer" source?
for printing to console a textual representation of the image (which could be useful even in libavfilter),
Agreed, I will try to submit something like that sometime. But that will not do for an example, because an example needs to show how the main program is supposed to get the filtered data back from libavfilter.
and let the user specify the input for the input source in the filtergraph description.
As above: an API example needs to illustrate how the main program is supposed to inject the video/audio it has into libavfilter. (And at this time, video sources are rather scarce.)
you can use the log system (with the fancy coloring support).
I think that a simple fprintf is more readable for this case. Anyway, I temporarily withdraw my patch: I forgot all cleanup code. I will submit again when it is done. But in the meantime, I am interested in clarifications on the previous points, especially the question of the sink. Regards, -- Nicolas George
On Wed, 2010-12-08 at 20:37 +0100, Nicolas George wrote:
L'octidi 18 frimaire, an CCXIX, Stefano Sabatini a ?crit :
I'd prefer to avoid to rely on other FFmpeg libraries, currently lavfi only depends on libavutil, libavcore and libswscale, and API example should be minimal so should be as strict as possible in terms of requirements.
I think that illustrating how the filtering system interacts with the rest of the libraries is important: which time base should be used, how to insert an AVFrame in the system, etc.
See libavformat/output-example.c: to give a complete working example, before outputing with libavformat, it scales with libswscale and encodes with libavcodec.
libavformat depends on libavcodec though, so it might as well use it. However, libavformat could use a pure remuxing example. I would also like to see a more pure libavfilter example that doesn't depend on libavformat and libavcodec. However, I don't think it has to be mutually exclusive to an example that does use them like this one. In short: this example seems fair. IMO we could use another one that only does filtering on generated data. /Tomas
Le nonidi 19 frimaire, an CCXIX, Tomas H?rdin a ?crit?:
I would also like to see a more pure libavfilter example that doesn't depend on libavformat and libavcodec. However, I don't think it has to be mutually exclusive to an example that does use them like this one.
In short: this example seems fair. IMO we could use another one that only does filtering on generated data.
Thanks for your advice. Here is an updated proposal: - Renamed to "decode_filter-example.c". - Removed useless "string.h" include. - Closed and freed everything. On the other hand, I stick to using nullsink to terminate the chain, unless someone who knows tells me that there is another preferred method. I keep in mind submitting a trivial ASCII-art sink for sometime soon. Regards, -- Nicolas George
On Fri, Dec 10, 2010 at 07:11:05PM +0100, Nicolas George wrote:
Le nonidi 19 frimaire, an CCXIX, Tomas H?rdin a ?crit?:
I would also like to see a more pure libavfilter example that doesn't depend on libavformat and libavcodec. However, I don't think it has to be mutually exclusive to an example that does use them like this one.
In short: this example seems fair. IMO we could use another one that only does filtering on generated data.
Thanks for your advice. Here is an updated proposal:
- Renamed to "decode_filter-example.c".
- Removed useless "string.h" include.
- Closed and freed everything.
On the other hand, I stick to using nullsink to terminate the chain, unless someone who knows tells me that there is another preferred method.
I keep in mind submitting a trivial ASCII-art sink for sometime soon.
Regards,
-- Nicolas George
Makefile | 4 decode_filter-example.c | 232 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 236 insertions(+) 53ae1c5a6e66e764314633428f85b5629e735b18 ffmpeg-vf-test-20101210-1903-01-vf-test.diff libavfilter/Makefile | 4 + libavfilter/decode_filter-example.c | 232 +++++++++++++++++++++++++++++++++++
what follows is more a review of the APIs than the api example, as i think its a great oppertunity to review the APIs based on the example ...
2 files changed, 236 insertions(+), 0 deletions(-)
diff --git a/libavfilter/Makefile b/libavfilter/Makefile index aece3ab..b5d9176 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -56,4 +56,8 @@ OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o
DIRS = x86
+EXAMPLES = api + include $(SUBDIR)../subdir.mak + +$(SUBDIR)decode_filter-example$(EXESUF): ELIBS = -lavformat -lavcodec -lavcore -lavutil $(FFEXTRALIBS) diff --git a/libavfilter/decode_filter-example.c b/libavfilter/decode_filter-example.c new file mode 100644 index 0000000..526111a --- /dev/null +++ b/libavfilter/decode_filter-example.c @@ -0,0 +1,232 @@ +/* + * API example for libavfilter + * Copyright (c) 2010 Nicolas George + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#define _XOPEN_SOURCE 600 /* for usleep */ +#include <stdio.h> +#include <stdlib.h> +#include <unistd.h> +#include <libavformat/avformat.h> +#include <libavcodec/avcodec.h> +#include <libavfilter/avfiltergraph.h> +#include <libavfilter/vsrc_buffer.h> + +const char *filter_descr = "scale=78:24,format=gray"; + +static AVFormatContext *avf; +static AVCodecContext *video_dec; +AVFilterContext *video_in_filter; +AVFilterContext *video_out_filter; +AVFilterGraph *filter_graph; +static int video_stream = -1; +static int64_t last_pts = AV_NOPTS_VALUE; + +static void fatal_libav_error(const char *tag, int r) +{ + char buf[1024]; + + av_strerror(r, buf, sizeof(buf)); + fprintf(stderr, "%s: %s\n", tag, buf); + exit(1); +} + +static void open_input_file(const char *filename) +{ + int r, i; + AVCodec *codec; + AVCodecContext *avc; +
+ avcodec_register_all(); + av_register_all();
already called in main()
+ r = av_open_input_file(&avf, filename, NULL, 0, NULL); + if (r < 0) + fatal_libav_error("av_open_input_file", r); + r = av_find_stream_info(avf); + if (r < 0) + fatal_libav_error("av_find_stream_info", r); +
+ /* Find a video stream */ + for (i = 0; i < (int)avf->nb_streams; i++) { + avc = avf->streams[i]->codec; + if (!video_dec && avc->codec_type == CODEC_TYPE_VIDEO) { + video_dec = avc; + video_stream = i; + } + }
We have code in ffplay.c that selects the "best" stream of a kind (see surrounding code when you grep for st_best_packet_count) this should be moved into the public API of libavformat and used here and from ffplay Selecting the first stream leads to known bugs (dont remember the issue #)
+ /* Init the video decoder */ + if (video_dec) { + codec = avcodec_find_decoder(video_dec->codec_id); + if (!codec) { + fprintf(stderr, "Unable to find video decoder\n"); + exit(1); + }
+ r = avcodec_open(video_dec, codec);
make libavcodec call avcodec_find_decoder() when codec=NULL and this becomes simpler
+ if (r < 0) + fatal_libav_error("avcodec_open", r); + } +} + +static void init_filters(const char *filters) +{ + char args[256]; + int r; + AVFilter *vf_buffer = avfilter_get_by_name("buffer"); + AVFilter *vf_nullsink = avfilter_get_by_name("nullsink"); + AVFilterInOut *outputs = av_malloc(sizeof(AVFilterInOut)); + AVFilterInOut *inputs = av_malloc(sizeof(AVFilterInOut)); + + + filter_graph = avfilter_graph_alloc(); + + /* Buffer video source: the decoded frames from the codec will be + * inserted here. */ + snprintf(args, sizeof(args), "%d:%d:%d:%d:%d", video_dec->width, + video_dec->height, video_dec->pix_fmt, + video_dec->time_base.num, video_dec->time_base.den);
+ r = avfilter_graph_create_filter(&video_in_filter, vf_buffer, "src", args, + NULL, filter_graph); + if (r < 0) + fatal_libav_error("avfilter_graph_create_filter: buffer", r); + + /* Null video sink: to terminate the filter chain. */ + r = avfilter_graph_create_filter(&video_out_filter, vf_nullsink, "out", + NULL, NULL, filter_graph); + if (r < 0) + fatal_libav_error("avfilter_graph_create_filter: nullsink", r); +
+ /* Endpoints for the filter graph. */ + outputs->name = av_strdup("in"); + outputs->filter_ctx = video_in_filter; + outputs->pad_idx = 0; + outputs->next = NULL; + inputs->name = av_strdup("out"); + inputs->filter_ctx = video_out_filter; + inputs->pad_idx = 0; + inputs->next = NULL;
These 2 blocks seems somewhat redundant the avfilter_graph_create_filter() could do that already somehow, maybe if we move the 2 AVFilterInOut into filter_graph
+ r = avfilter_graph_parse(filter_graph, filters, inputs, outputs, NULL); + if (r < 0) + fatal_libav_error("avfilter_graph_parse", r); + + r = avfilter_graph_config(filter_graph, NULL); + if (r < 0) + fatal_libav_error("avfilter_graph_config", r); +} + +static void display_frame(AVFilterLink *link) +{ + AVFilterBufferRef *ob; + int x, y; + uint8_t *p0, *p; + int64_t delay; + + ob = link->cur_buf; + if (ob->pts != AV_NOPTS_VALUE) { + if (last_pts != AV_NOPTS_VALUE) { + /* sleep roughly the right amount of time; + * usleep is in microseconds, just like AV_TIME_BASE. */ + delay = av_rescale_q(ob->pts - last_pts, + link->time_base, + AV_TIME_BASE_Q); + if (delay > 0 && delay < 1000000) + usleep(delay); + } + last_pts = ob->pts; + } + /* ob->data, ob->linesize and ob->pts could be copied to an AVFrame + * structure. */ + /* Trivial ASCII grayscale display. */
+ p0 = ob->data[0]; + puts("\033c"); + for (y = 0; y < link->h; y++) { + p = p0; + for (x = 0; x < link->w; x++) + putchar(" .-+#"[*(p++) / 52]);
p[x] would avoid the temp p vs p0 varible
+ putchar('\n'); + p0 += ob->linesize[0]; + } + fflush(stdout); + avfilter_unref_buffer(ob); +} + +static void got_video_frame(AVFrame *frame) +{ + int r; + AVFilterLink *out = video_out_filter->inputs[0]; + + av_vsrc_buffer_add_frame(video_in_filter, frame, frame->pts, + video_dec->sample_aspect_ratio); + while (avfilter_poll_frame(out)) { + r = avfilter_request_frame(out); + if (r < 0) + fatal_libav_error("avfilter_request_frame", r); + if (!out->cur_buf) + fatal_libav_error("avfilter_request_frame", AVERROR(ENOENT)); + display_frame(out); + } +} + +int main(int argc, char **argv) +{ + int r; + AVPacket packet; + AVFrame frame; + int got_frame; + + if (argc != 2) { + fprintf(stderr, "Usage: api_example file\n"); + exit(1); + }
+ avcodec_register_all();
unneeded
+ avfilter_register_all(); + av_register_all(); + open_input_file(argv[1]); + init_filters(filter_descr); + /* Read all packets. */ + while (1) {
+ r = av_read_frame(avf, &packet); + if (r < 0) + break;
EAGAIN handling is missing [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB The real ebay dictionary, page 2 "100% positive feedback" - "All either got their money back or didnt complain" "Best seller ever, very honest" - "Seller refunded buyer after failed scam"
Le primidi 21 frimaire, an CCXIX, Michael Niedermayer a ?crit?:
what follows is more a review of the APIs than the api example, as i think its a great oppertunity to review the APIs based on the example ...
Thanks for that review.
2 files changed, 236 insertions(+), 0 deletions(-)
diff --git a/libavfilter/Makefile b/libavfilter/Makefile index aece3ab..b5d9176 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -56,4 +56,8 @@ OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o
DIRS = x86
+EXAMPLES = api + include $(SUBDIR)../subdir.mak + +$(SUBDIR)decode_filter-example$(EXESUF): ELIBS = -lavformat -lavcodec -lavcore -lavutil $(FFEXTRALIBS) diff --git a/libavfilter/decode_filter-example.c b/libavfilter/decode_filter-example.c new file mode 100644 index 0000000..526111a --- /dev/null +++ b/libavfilter/decode_filter-example.c @@ -0,0 +1,232 @@ +/* + * API example for libavfilter + * Copyright (c) 2010 Nicolas George + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#define _XOPEN_SOURCE 600 /* for usleep */ +#include <stdio.h> +#include <stdlib.h> +#include <unistd.h> +#include <libavformat/avformat.h> +#include <libavcodec/avcodec.h> +#include <libavfilter/avfiltergraph.h> +#include <libavfilter/vsrc_buffer.h> + +const char *filter_descr = "scale=78:24,format=gray"; + +static AVFormatContext *avf; +static AVCodecContext *video_dec; +AVFilterContext *video_in_filter; +AVFilterContext *video_out_filter; +AVFilterGraph *filter_graph; +static int video_stream = -1; +static int64_t last_pts = AV_NOPTS_VALUE; + +static void fatal_libav_error(const char *tag, int r) +{ + char buf[1024]; + + av_strerror(r, buf, sizeof(buf)); + fprintf(stderr, "%s: %s\n", tag, buf); + exit(1); +} + +static void open_input_file(const char *filename) +{ + int r, i; + AVCodec *codec; + AVCodecContext *avc; +
+ avcodec_register_all(); + av_register_all(); already called in main()
Fixed in my working repository.
We have code in ffplay.c that selects the "best" stream of a kind (see surrounding code when you grep for st_best_packet_count) this should be moved into the public API of libavformat and used here and from ffplay
The logic seems to be to select the stream with the maximum value for codec_info_nb_frames. May I suggest: int av_find_best_streams(AVFormatContext *ic, int flags, int *video_stream, int *audio_stream, int *subtitles_stream, int flags);
make libavcodec call avcodec_find_decoder() when codec=NULL and this becomes simpler
Good idea indeed.
+ /* Endpoints for the filter graph. */ + outputs->name = av_strdup("in"); + outputs->filter_ctx = video_in_filter; + outputs->pad_idx = 0; + outputs->next = NULL; + inputs->name = av_strdup("out"); + inputs->filter_ctx = video_out_filter; + inputs->pad_idx = 0; + inputs->next = NULL; These 2 blocks seems somewhat redundant the avfilter_graph_create_filter() could do that already somehow, maybe if we move the 2 AVFilterInOut into filter_graph
I will leave Stefano answer to that one.
+ putchar(" .-+#"[*(p++) / 52]); p[x] would avoid the temp p vs p0 varible
Changed in my working repository.
EAGAIN handling is missing
Changed in my working repository. I would like another point about the API:
if (packet.pts != AV_NOPTS_VALUE) video_dec->reordered_opaque = av_rescale_q(packet.pts, avf->streams[video_stream]->time_base, video_dec->time_base); r = avcodec_decode_video2(video_dec, &frame, &got_frame, &packet); if (r < 0) fatal_libav_error("avcodec_decode_video2", r); if (got_frame) { if (frame.pts == AV_NOPTS_VALUE) frame.pts = frame.reordered_opaque; got_video_frame(&frame); }
I think a wrapper around avcodec_decode_video2 that takes care of the reordered_opaque stuff to fill in PTS using the packet data would be useful for a lot of applications. It could include the guess_correct_pts logic too. Regards, -- Nicolas George
On Sun, Dec 12, 2010 at 05:45:19PM +0100, Nicolas George wrote:
Le primidi 21 frimaire, an CCXIX, Michael Niedermayer a ?crit?:
what follows is more a review of the APIs than the api example, as i think its a great oppertunity to review the APIs based on the example ...
Thanks for that review.
2 files changed, 236 insertions(+), 0 deletions(-)
diff --git a/libavfilter/Makefile b/libavfilter/Makefile index aece3ab..b5d9176 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -56,4 +56,8 @@ OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o
DIRS = x86
+EXAMPLES = api + include $(SUBDIR)../subdir.mak + +$(SUBDIR)decode_filter-example$(EXESUF): ELIBS = -lavformat -lavcodec -lavcore -lavutil $(FFEXTRALIBS) diff --git a/libavfilter/decode_filter-example.c b/libavfilter/decode_filter-example.c new file mode 100644 index 0000000..526111a --- /dev/null +++ b/libavfilter/decode_filter-example.c @@ -0,0 +1,232 @@ +/* + * API example for libavfilter + * Copyright (c) 2010 Nicolas George + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#define _XOPEN_SOURCE 600 /* for usleep */ +#include <stdio.h> +#include <stdlib.h> +#include <unistd.h> +#include <libavformat/avformat.h> +#include <libavcodec/avcodec.h> +#include <libavfilter/avfiltergraph.h> +#include <libavfilter/vsrc_buffer.h> + +const char *filter_descr = "scale=78:24,format=gray"; + +static AVFormatContext *avf; +static AVCodecContext *video_dec; +AVFilterContext *video_in_filter; +AVFilterContext *video_out_filter; +AVFilterGraph *filter_graph; +static int video_stream = -1; +static int64_t last_pts = AV_NOPTS_VALUE; + +static void fatal_libav_error(const char *tag, int r) +{ + char buf[1024]; + + av_strerror(r, buf, sizeof(buf)); + fprintf(stderr, "%s: %s\n", tag, buf); + exit(1); +} + +static void open_input_file(const char *filename) +{ + int r, i; + AVCodec *codec; + AVCodecContext *avc; +
+ avcodec_register_all(); + av_register_all(); already called in main()
Fixed in my working repository.
We have code in ffplay.c that selects the "best" stream of a kind (see surrounding code when you grep for st_best_packet_count) this should be moved into the public API of libavformat and used here and from ffplay
The logic seems to be to select the stream with the maximum value for codec_info_nb_frames.
May I suggest:
int av_find_best_streams(AVFormatContext *ic, int flags, int *video_stream, int *audio_stream, int *subtitles_stream, int flags);
looks good
make libavcodec call avcodec_find_decoder() when codec=NULL and this becomes simpler
Good idea indeed.
+ /* Endpoints for the filter graph. */ + outputs->name = av_strdup("in"); + outputs->filter_ctx = video_in_filter; + outputs->pad_idx = 0; + outputs->next = NULL; + inputs->name = av_strdup("out"); + inputs->filter_ctx = video_out_filter; + inputs->pad_idx = 0; + inputs->next = NULL; These 2 blocks seems somewhat redundant the avfilter_graph_create_filter() could do that already somehow, maybe if we move the 2 AVFilterInOut into filter_graph
I will leave Stefano answer to that one.
+ putchar(" .-+#"[*(p++) / 52]); p[x] would avoid the temp p vs p0 varible
Changed in my working repository.
EAGAIN handling is missing
Changed in my working repository.
I would like another point about the API:
if (packet.pts != AV_NOPTS_VALUE) video_dec->reordered_opaque = av_rescale_q(packet.pts, avf->streams[video_stream]->time_base, video_dec->time_base); r = avcodec_decode_video2(video_dec, &frame, &got_frame, &packet); if (r < 0) fatal_libav_error("avcodec_decode_video2", r); if (got_frame) { if (frame.pts == AV_NOPTS_VALUE) frame.pts = frame.reordered_opaque; got_video_frame(&frame); }
I think a wrapper around avcodec_decode_video2 that takes care of the reordered_opaque stuff to fill in PTS using the packet data would be useful for a lot of applications. It could include the guess_correct_pts logic too.
Yes, i agree [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB The real ebay dictionary, page 3 "Rare item" - "Common item with rare defect or maybe just a lie" "Professional" - "'Toy' made in china, not functional except as doorstop" "Experts will know" - "The seller hopes you are not an expert"
Michael Niedermayer <michaelni <at> gmx.at> writes:
We have code in ffplay.c that selects the "best" stream of a kind (see surrounding code when you grep for st_best_packet_count) this should be moved into the public API of libavformat and used here and from ffplay Selecting the first stream leads to known bugs (dont remember the issue #)
Issues 1499 and 1668. Carl Eugen
On Thu, Dec 09, 2010 at 11:23:00AM +0100, Tomas H?rdin wrote:
On Wed, 2010-12-08 at 20:37 +0100, Nicolas George wrote:
L'octidi 18 frimaire, an CCXIX, Stefano Sabatini a ?crit :
I'd prefer to avoid to rely on other FFmpeg libraries, currently lavfi only depends on libavutil, libavcore and libswscale, and API example should be minimal so should be as strict as possible in terms of requirements.
I think that illustrating how the filtering system interacts with the rest of the libraries is important: which time base should be used, how to insert an AVFrame in the system, etc.
See libavformat/output-example.c: to give a complete working example, before outputing with libavformat, it scales with libswscale and encodes with libavcodec.
libavformat depends on libavcodec though, so it might as well use it. However, libavformat could use a pure remuxing example.
I would also like to see a more pure libavfilter example that doesn't depend on libavformat and libavcodec. However, I don't think it has to
i agree, a pure libavfilter example is definitly needed [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB The real ebay dictionary, page 3 "Rare item" - "Common item with rare defect or maybe just a lie" "Professional" - "'Toy' made in china, not functional except as doorstop" "Experts will know" - "The seller hopes you are not an expert"
On date Wednesday 2010-12-08 20:37:44 +0100, Nicolas George encoded:
L'octidi 18 frimaire, an CCXIX, Stefano Sabatini a ?crit?:
+#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> These headers shouldn't be required.
string.h was forgotten from a previous version.
unistd.h is necessary for usleep, compilation breaks without it.
stdio.h and stdlib.h are indirectly included by libav* headers, but AFAIK, that fact is not explicitly stated, and that could possibly change.
I'd prefer to avoid to rely on other FFmpeg libraries, currently lavfi only depends on libavutil, libavcore and libswscale, and API example should be minimal so should be as strict as possible in terms of requirements.
I think that illustrating how the filtering system interacts with the rest of the libraries is important: which time base should be used, how to insert an AVFrame in the system, etc.
See libavformat/output-example.c: to give a complete working example, before outputing with libavformat, it scales with libswscale and encodes with libavcodec.
Also the code could be simpler if you implement it this way... implement a video sink
This is a point that I wanted clarified when I started writing this program: it seems that anyone who wants to use libavfilter is almost expected to write its own designated sink.
Is it on purpose?
Should there not be some kind of "buffer" sink just like there is a "buffer" source?
That may be possible, on the other hand such a sink is rather trivial to write (check cmdutils.c:ffsink). In general for integrating your application you need to write a source (if you need to inject data) and a sink (if you need to pass it filtered to the application), for the source you may use vsrc_buffer and its interface in vsrc_buffer.h, for the sink we don't have nothing of that kind yet (but may have in the future). Note that as the library keeps adding functionality the need for injecting/extracting data may decrease, for example you could implement a video player simply connecting a movie source to a video display sink. Also note that vsrc_buffer is quite generic so in the present form doesn't allow the kind of strict integration which is possible in ffplay.c (required for enabling direct rendering).
for printing to console a textual representation of the image (which could be useful even in libavfilter),
Agreed, I will try to submit something like that sometime.
But that will not do for an example, because an example needs to show how the main program is supposed to get the filtered data back from libavfilter.
and let the user specify the input for the input source in the filtergraph description.
OK but I'm still thinking that having such example (using lavf API) in libavfilter is a bit misleading, maybe would be better to have an examples dir containing all the examples, rather than having them scattered in the tree.
As above: an API example needs to illustrate how the main program is supposed to inject the video/audio it has into libavfilter.
(And at this time, video sources are rather scarce.)
you can use the log system (with the fancy coloring support).
I think that a simple fprintf is more readable for this case.
I suggested this because I was thinking to implement that in ff*, anyway this is bikeshed so do as you prefer.
Anyway, I temporarily withdraw my patch: I forgot all cleanup code. I will submit again when it is done. But in the meantime, I am interested in clarifications on the previous points, especially the question of the sink. -- FFmpeg = Fanciful and Funny Marvellous Pitiless Egregious Gymnast
Le decadi 20 frimaire, an CCXIX, Stefano Sabatini a ?crit?:
In general for integrating your application you need to write a source (if you need to inject data) and a sink (if you need to pass it filtered to the application), for the source you may use vsrc_buffer and its interface in vsrc_buffer.h, for the sink we don't have nothing of that kind yet (but may have in the future).
I gathered as much, but your last parenthesised statement leaves the following question: What do we _want_: do we want this output-buffer sink, or do we want people who write applications to write their own sink each time? It we want the buffer sink, then I can try to make a proposal in a near future. If we want a dedicated sink in each application, I shall change the example in accordance.
library keeps adding functionality the need for injecting/extracting data may decrease, for example you could implement a video player simply connecting a movie source to a video display sink.
Indeed, but not always. As a matter of fact, one of my purposes when writing this example is to learn enough about lavfi API to write the "-vf lavfi" option for mplayer. Unfortunately, I do not think that lavfi sinks will compare with mplayer libvo in any foreseeable future.
Also note that vsrc_buffer is quite generic so in the present form doesn't allow the kind of strict integration which is possible in ffplay.c (required for enabling direct rendering).
I do not think I quite understand what you mean here.
OK but I'm still thinking that having such example (using lavf API) in libavfilter is a bit misleading, maybe would be better to have an examples dir containing all the examples, rather than having them scattered in the tree.
I agree with that; "doc/examples" would probably be a good place. Regards, -- Nicolas George
On date Sunday 2010-12-12 17:21:36 +0100, Nicolas George encoded:
Le decadi 20 frimaire, an CCXIX, Stefano Sabatini a ?crit?:
In general for integrating your application you need to write a source (if you need to inject data) and a sink (if you need to pass it filtered to the application), for the source you may use vsrc_buffer and its interface in vsrc_buffer.h, for the sink we don't have nothing of that kind yet (but may have in the future).
I gathered as much, but your last parenthesised statement leaves the following question:
What do we _want_: do we want this output-buffer sink, or do we want people who write applications to write their own sink each time?
It we want the buffer sink, then I can try to make a proposal in a near future. If we want a dedicated sink in each application, I shall change the example in accordance.
If you want to dedicate some time on this that's definitively welcome, I still don't consider it an high-priority (as I want to fix all the various regressions). I suppose the cmdutils.c:ffsink and get_filtered_video_frame() may be used as a base for creating such an API.
library keeps adding functionality the need for injecting/extracting data may decrease, for example you could implement a video player simply connecting a movie source to a video display sink.
Indeed, but not always. As a matter of fact, one of my purposes when writing this example is to learn enough about lavfi API to write the "-vf lavfi" option for mplayer.
Unfortunately, I do not think that lavfi sinks will compare with mplayer libvo in any foreseeable future.
Can you elaborate on this? Does it depend on architectural issues or is a problem of missing components (which will eventually be implemented)?
Also note that vsrc_buffer is quite generic so in the present form doesn't allow the kind of strict integration which is possible in ffplay.c (required for enabling direct rendering).
I do not think I quite understand what you mean here.
I mean that vsrc_buffer.c in the present form is not acceptable, we should either make it a libavfilter-pure source (with no references to lavc/lavf symbols), or turn it into something more similar to the ffplay.c:input_filter, which allows direct rendering, or have both of them (note that I already posted some patches for this).
OK but I'm still thinking that having such example (using lavf API) in libavfilter is a bit misleading, maybe would be better to have an examples dir containing all the examples, rather than having them scattered in the tree.
I agree with that; "doc/examples" would probably be a good place. -- FFmpeg = Foolish and Funny Most Proud Ecletic Goblin
Le duodi 22 frimaire, an CCXIX, Stefano Sabatini a ?crit?:
If you want to dedicate some time on this that's definitively welcome, I still don't consider it an high-priority (as I want to fix all the various regressions). I suppose the cmdutils.c:ffsink and get_filtered_video_frame() may be used as a base for creating such an API.
I gave some more thought about that point: ffsink and get_filtered_video_frame were what I imitated in the first place. As far as I can tell, the only difference between ffsink and nullsink is that ffsink only accepts one pixel format. It would be reasonably easy to add this feature to nullsink or to rely on an additional "format" filter. Or is there something I am missing? (As for me, I am still trying to figure out what ffplay's input_filter does.) Regards, -- Nicolas George
On date Sunday 2010-12-26 12:23:12 +0100, Nicolas George encoded:
Le duodi 22 frimaire, an CCXIX, Stefano Sabatini a ?crit?:
If you want to dedicate some time on this that's definitively welcome, I still don't consider it an high-priority (as I want to fix all the various regressions). I suppose the cmdutils.c:ffsink and get_filtered_video_frame() may be used as a base for creating such an API.
I gave some more thought about that point: ffsink and get_filtered_video_frame were what I imitated in the first place.
As far as I can tell, the only difference between ffsink and nullsink is that ffsink only accepts one pixel format. It would be reasonably easy to add this feature to nullsink or to rely on an additional "format" filter.
Think nullsink as a sort of /dev/null (which can sometimes be useful for testing purposes), so it should not be modified, but it shoul be pretty straightforward to make ffsink public (buffersink?).
Or is there something I am missing?
(As for me, I am still trying to figure out what ffplay's input_filter does.)
input_filter requests video frames through get_video_frame(), and wraps them around a video buffer reference which is passed to the filterchain. Strict integration/direct rendering with the decoder is performed through the get/release/reget buffer libavcodec API, so that the decoder uses the same buffers provided by the filterchain by calling avfilter_get_video_buffer(), thus avoiding unnecessary memcpies. If you have specific questions I can try to address them. -- FFmpeg = Faithless and Funny Miracolous Powered Eccentric Governor
On date Sunday 2010-12-26 12:23:12 +0100, Nicolas George wrote:
Le duodi 22 frimaire, an CCXIX, Stefano Sabatini a écrit :
If you want to dedicate some time on this that's definitively welcome, I still don't consider it an high-priority (as I want to fix all the various regressions). I suppose the cmdutils.c:ffsink and get_filtered_video_frame() may be used as a base for creating such an API.
I gave some more thought about that point: ffsink and get_filtered_video_frame were what I imitated in the first place.
As far as I can tell, the only difference between ffsink and nullsink is that ffsink only accepts one pixel format. It would be reasonably easy to add this feature to nullsink or to rely on an additional "format" filter.
Or is there something I am missing?
Updated patch, not yet ready. In particular, there is some issue with the display_picref() timing and we need to poll from the input link attached to the sink, maybe we should provide an interface for that?
Le tridi 3 messidor, an CCXIX, Stefano Sabatini a écrit :
Updated patch, not yet ready.
Thank you to have taken up the task.
doc/examples/decoding+filtering-example.c | 232 +++++++++++++++++++++++++++++
Is the "-example" part really necessary, considering the file is itself in the examples directory; decoding_and_filtering.c would be IMHO more elegant.
+ * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version.
It may be a good idea to set a less restrictive licence on simple examples programs, possibly even Public Domain.
+ /* select the video stream */ + if ((ret = av_find_best_stream(fmt_ctx, AVMEDIA_TYPE_VIDEO, + -1, -1, NULL, 0)) < 0) { + av_log(NULL, AV_LOG_ERROR, "Cannot find a video stream in the input file\n"); + return ret; + } + video_stream_index = ret; + dec_ctx = fmt_ctx->streams[video_stream_index]->codec; + + /* init the video decoder */ + if (dec_ctx) { + dec = avcodec_find_decoder(dec_ctx->codec_id); + if (!dec) { + av_log(NULL, AV_LOG_ERROR, "Unable to find video decoder\n"); + return AVERROR(EINVAL); + }
This could be made slightly simpler by using something like that: ret = av_find_best_stream(fmt_ctx, AVMEDIA_TYPE_VIDEO, -1, -1, &dec, 0);
+ ret = avcodec_decode_video2(dec_ctx, &frame, &got_frame, &packet); + if (ret < 0) {
Nit: The code sometimes uses "if ((ret = ...) < 0)" and sometimes this version.
+ av_free_packet(&packet);
Couldn't it be nearer avcodec_decode_video2 (this was a flaw in the original version). Regards, -- Nicolas George
On date Tuesday 2011-06-21 17:21:43 +0200, Nicolas George encoded:
Le tridi 3 messidor, an CCXIX, Stefano Sabatini a écrit :
Updated patch, not yet ready.
Thank you to have taken up the task.
doc/examples/decoding+filtering-example.c | 232 +++++++++++++++++++++++++++++
Is the "-example" part really necessary, considering the file is itself in the examples directory; decoding_and_filtering.c would be IMHO more elegant.
Changed to filtering.c.
+ * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version.
It may be a good idea to set a less restrictive licence on simple examples programs, possibly even Public Domain.
Done (same license as muxing.c and metadata.c in the same dir).
+ /* select the video stream */ + if ((ret = av_find_best_stream(fmt_ctx, AVMEDIA_TYPE_VIDEO, + -1, -1, NULL, 0)) < 0) { + av_log(NULL, AV_LOG_ERROR, "Cannot find a video stream in the input file\n"); + return ret; + } + video_stream_index = ret; + dec_ctx = fmt_ctx->streams[video_stream_index]->codec; + + /* init the video decoder */ + if (dec_ctx) { + dec = avcodec_find_decoder(dec_ctx->codec_id); + if (!dec) { + av_log(NULL, AV_LOG_ERROR, "Unable to find video decoder\n"); + return AVERROR(EINVAL); + }
This could be made slightly simpler by using something like that:
ret = av_find_best_stream(fmt_ctx, AVMEDIA_TYPE_VIDEO, -1, -1, &dec, 0);
+ ret = avcodec_decode_video2(dec_ctx, &frame, &got_frame, &packet); + if (ret < 0) {
Yes, fixed.
Nit: The code sometimes uses "if ((ret = ...) < 0)" and sometimes this version.
+ av_free_packet(&packet);
Couldn't it be nearer avcodec_decode_video2 (this was a flaw in the original version).
Done. Also fixed the timestamp issue, the warning issue was fixed in a separate vsink_buffer patch. -- FFmpeg = Fundamentalist and Forgiving Multimedia Puristic Extroverse Gargoyle
On date Friday 2011-07-08 11:17:12 +0200, Stefano Sabatini encoded:
On date Tuesday 2011-06-21 17:21:43 +0200, Nicolas George encoded:
Le tridi 3 messidor, an CCXIX, Stefano Sabatini a écrit :
Updated patch, not yet ready.
Thank you to have taken up the task.
doc/examples/decoding+filtering-example.c | 232 +++++++++++++++++++++++++++++
Is the "-example" part really necessary, considering the file is itself in the examples directory; decoding_and_filtering.c would be IMHO more elegant.
Changed to filtering.c.
+ * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version.
It may be a good idea to set a less restrictive licence on simple examples programs, possibly even Public Domain.
Done (same license as muxing.c and metadata.c in the same dir).
+ /* select the video stream */ + if ((ret = av_find_best_stream(fmt_ctx, AVMEDIA_TYPE_VIDEO, + -1, -1, NULL, 0)) < 0) { + av_log(NULL, AV_LOG_ERROR, "Cannot find a video stream in the input file\n"); + return ret; + } + video_stream_index = ret; + dec_ctx = fmt_ctx->streams[video_stream_index]->codec; + + /* init the video decoder */ + if (dec_ctx) { + dec = avcodec_find_decoder(dec_ctx->codec_id); + if (!dec) { + av_log(NULL, AV_LOG_ERROR, "Unable to find video decoder\n"); + return AVERROR(EINVAL); + }
This could be made slightly simpler by using something like that:
ret = av_find_best_stream(fmt_ctx, AVMEDIA_TYPE_VIDEO, -1, -1, &dec, 0);
+ ret = avcodec_decode_video2(dec_ctx, &frame, &got_frame, &packet); + if (ret < 0) {
Yes, fixed.
Nit: The code sometimes uses "if ((ret = ...) < 0)" and sometimes this version.
+ av_free_packet(&packet);
Couldn't it be nearer avcodec_decode_video2 (this was a flaw in the original version).
Done.
Also fixed the timestamp issue, the warning issue was fixed in a separate vsink_buffer patch. -- FFmpeg = Fundamentalist and Forgiving Multimedia Puristic Extroverse Gargoyle
From 0f302f57e0bc52791fddb3da0728f84da7425a9c Mon Sep 17 00:00:00 2001 From: Stefano Sabatini <stefano.sabatini-lala@poste.it> Date: Tue, 21 Jun 2011 00:33:37 +0200 Subject: [PATCH] doc/examples: add decoding/filtering example program
--- doc/examples/Makefile | 2 +- doc/examples/filtering.c | 230 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 231 insertions(+), 1 deletions(-) create mode 100644 doc/examples/filtering.c
Updated again, no need for the other patch, call avfilter_poll_frame() in the external look rather than require a possible pointless avfilter_poll_frame() call in vsink_buffer (this is the same technique adopted in ffmpeg.c). I'll push in a few days if I see no comments. -- FFmpeg = Furious & Free Minimal Ponderous Explosive Gymnast
On date Saturday 2011-07-09 15:18:15 +0200, Stefano Sabatini encoded:
On date Friday 2011-07-08 11:17:12 +0200, Stefano Sabatini encoded: [...]
doc/examples/Makefile | 2 +- doc/examples/filtering.c | 230 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 231 insertions(+), 1 deletions(-) create mode 100644 doc/examples/filtering.c
Updated again, no need for the other patch, call avfilter_poll_frame() in the external look rather than require a possible pointless avfilter_poll_frame() call in vsink_buffer (this is the same technique adopted in ffmpeg.c).
I'll push in a few days if I see no comments.
Applied. -- FFmpeg = Fancy & Faithful Martial Power Encoding/decoding Gorilla
participants (7)
-
cehoyos@ag.or.at -
michaelni@gmx.at -
Nicolas George -
nicolas.george@normalesup.org -
Stefano Sabatini -
stefano.sabatini-lala@poste.it -
tomas.hardin@codemill.se