[FFmpeg-devel] [patch] glob matching for image series

Alexander Strasser eclipse7 at gmx.net
Sun Mar 11 17:46:08 CET 2012


Hi Nicolas,

  I assume you read my reply to Brian in the meanwhile, therefore
some relevant parts of this mail will be left uncommented.

Nicolas George wrote:
> Le nonidi 9 ventôse, an CCXX, Alexander Strasser a écrit :
> >   1. Support globbing not only for picture input.
> 
> How do you imagine it should be handled, from the API point of view? We are
> nowhere near supporting demuxer-level concatenation, IMHO, so the syntax for
> various parts of the command line will have to change four or five times
> before we are able of implementing anything like that.
> 
> >   2. Do it only if it was explicitly enabled
> 
> I find that rather user-unfriendly. OTOH, having options to ensure the mode
> of expansion would be useful for robust automated scripts:

  I see. What do you think about the attached patch? It builds directly on
Brian's work and tries to be user-friendly and consistent while not making
currently working use cases more hard/impossible when globbing is compiled
in.

  The few examples are given in the patch commit message. I also attached
my complete test log so you can dig in deeper. Beware the examples found
in the test log are rather extreme to test more corner cases.

> ffmpeg -f img2 -expand printf photo%05d.jpg
> ffmpeg -f img2 -expand glob   photo\*.jpg

  Looks like a nice idea. I am OK with adding such features later on.

> >   3. Agree on a set of features like ?*[]{} and fail if is not available
> >      (maybe with the additional possibility to force it on with what is
> >       available)
> 
> I do not like it at all in principle: if we were to cripple features just to
> be compatible with the lowest common denominator, we could just all install
> windows and be done with it.
> 
> On the other hand, for this particular case:
> 
> - GLOB_BRACE, could be marginally useful, but also cause trouble (the files
>   that randomly appears on people's hard drives tend to have braces or
>   parentheses to indicate the checksum, name of the team or whatever, or so
>   I have been told).

  This tends to be no problem with my proposed patch. Very useful glob
chars, [], are also used frequently for the same purposes you quote above.
This works out too with my patch.

> - GLOB_TILDE(_CHECK): this is not related to globbing at all, this is some
>   kind of special syntax for a bizarre environment variable expansion (~
>   expands like $HOME, but only at the start; ~foo would expand like
>   $HOME_FOO, assuming the contents of passwd is part of the environment).
>   Also, while it is perfectly normal for 'foo*.jpg' to be supported only by
>   the img2 demuxer, there is no logic of being able to write ~/image.jpg and
>   not ~/video.avi.
> 
>   Also, for the command-line tool, the user can just not escape the ~ and
>   let their shell expand it.

  So you think we can just drop it?

> - GLOB_NOMAGIC should be useless, since glob is called only if is_glob has
>   already detected metacharacters.

  Agreed. Though didn't change it in this patch though.

> So in this particular case, I do not mind sticking to the standard features
> at all.

  Ok, tell me if I should prepare a patch removing the GLOB_BRACE,
GLOB_TILDE(_CHECK), GLOB_NOMAGIC.
 
> >   4. Tell the user if he requested to enable globbing but it wasn't
> >      available at all in his lavf binary.
> 
> I second that.

  I wanted to do this as an incremental improvement to not further
delay Brian's work getting into ffmpeg.

  I thought about printing a warning if the user used glob chars in
image2 input path but glob support was not compiled in. Did you have
something else in mind?

  If we decide to keep GLOB_BRACE, that should be taken into account too.

  Also I think these warnings will have much lower chance of unexpected
occurrence after my patch, because only enabled glob chars would count
(e.g. %[).

  BTW if my patch passes review I can of course write an update for the
documentation and resubmit.

  While writing this a further question pops up, should ] and } be
removed in is_glob() detection code? If don't think it will make much
sense to try to glob it if we only found closing brackets/braces.

> >   The described behaviour would allow for implementation of applications
> > that behave consistently across platforms. That is a property I appreciate
> > very much and that I would like to see from FFmpeg programs and libraries.
> 
> Unfortunately, consistency is often at odds with feature-fullness, and
> someone who decided to use one particular platform because it has some
> interesting extensions should be able to benefit from these extensions.
> 
> What can be done, OTOH, is to make sure that someone who wants to stick to
> the common denominator can avoid triggering unexpected extensions.

  Not sure I can fully agree. It is very broad statement and therefore
not easy to comment on specifically. But in any case I think we should
get along with each other :)

  Alexander
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-img2dec-Recognize-glob-meta-chars-only-if-prefixed-b.patch
Type: text/x-diff
Size: 2742 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20120311/020fb41d/attachment.bin>
-------------- next part --------------
Test run for `glob' from 11.03.2012 at 16:33:57

./test/glob/0-all.sh
  generate test input image files

    cmd line: `mkdir /tmp/ffglobtest'
    cmd exit value:   0

    cmd line: `./ffmpeg -loglevel error -ss 630 -frames 100 -i '../samples/img2_glob/input.mkv' -f image2 -frames 100 '/tmp/ffglobtest/%%%03d-frame%%d[ABC].jpg''
    cmd exit value:   0

    cmd line: `./ffmpeg -loglevel error -ss 630 -frames 10 -i '../samples/img2_glob/input.mkv' -f image2 -frames 10 '/tmp/ffglobtest/%03d-]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]].jpg''
    cmd exit value:   0

    cmd line: `./ffmpeg -loglevel error -ss 630 -frames 10 -i '../samples/img2_glob/input.mkv' -f image2 -frames 10 '/tmp/ffglobtest/%03d-%%\.jpg''
    cmd exit value:   0

    cmd line: `./ffmpeg -loglevel error -ss 630 -frames 10 -i '../samples/img2_glob/input.mkv' -f image2 -codec mjpeg -frames 10 '/tmp/ffglobtest/%03d-%%''
    cmd exit value:   0

  result after 10s: success


  generate movie from image series with % enabled globbing

    cmd line: `git checkout img2_glob-inrecmental_patch'
    cmd standard error:
Already on 'img2_glob-inrecmental_patch'
    cmd exit value:   0

    cmd line: `make -j3'
    cmd standard output:
CC	cmdutils.o
CC	ffprobe.o
LD	ffmpeg_g
LD	ffplay_g
LD	ffserver_g
CP	ffmpeg
STRIP	ffmpeg
LD	ffprobe_g
CP	ffserver
CP	ffplay
STRIP	ffserver
STRIP	ffplay
CP	ffprobe
STRIP	ffprobe
    cmd exit value:   0

    cmd line: `rm -rf $TEST_DIR/video'
    cmd exit value:   0

    cmd line: `mkdir $TEST_DIR/video'
    cmd exit value:   0

    cmd line: `./ffmpeg -loglevel error -y -f image2 -i /tmp/ffglobtest/%\*-frame%%d\[ABC\].jpg /tmp/ffglobtest/video/X.avi'
    cmd exit value:   0

    cmd line: `./ffmpeg -loglevel error -y -f image2 -i /tmp/ffglobtest/%%*-frame%%d\[ABC\].jpg /tmp/ffglobtest/video/Y.avi'
    cmd standard error:
/tmp/ffglobtest/%%*-frame%%d[ABC].jpg: No such file or directory
    cmd exit value:   1

    cmd line: `./ffmpeg -loglevel error -y -f image2 -i /tmp/ffglobtest/%%%*-frame%%d\[%\[A%\]%\[AB%\]%\[ABC%\]\].jpg /tmp/ffglobtest/video/Y.avi'
    cmd exit value:   0

    cmd line: `./ffmpeg -loglevel error -y -f image2 -i /tmp/ffglobtest/%%%*-frame%%d\[ABC\].jpg /tmp/ffglobtest/video/Z.avi'
    cmd exit value:   0

    cmd line: `./ffmpeg -loglevel error -y -f image2 -i /tmp/ffglobtest/%*-]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]].jpg -f avi /dev/null'
    cmd exit value:   0

    cmd line: `./ffmpeg -loglevel error -y -f image2 -i /tmp/ffglobtest/%*-%\\.jpg -f avi /dev/null'
    cmd exit value:   0

    cmd line: `./ffmpeg -loglevel error -y -f image2 -codec mjpeg -i /tmp/ffglobtest/%*-%% -f avi /dev/null'
    cmd exit value:   0

    cmd line: `./ffmpeg -loglevel error -y -f image2 -i /tmp/ffglobtest/%%%03d-frame%%d\[ABC\].jpg /tmp/ffglobtest/video/C.avi'
    cmd exit value:   0

  result after 30s: success


  generate movie from image series with \ escaped globbing

    cmd line: `git checkout img2_glob_bolson'
    cmd standard error:
Switched to branch 'img2_glob_bolson'
    cmd exit value:   0

    cmd line: `make -j3'
    cmd standard output:
CC	libavformat/img2dec.o
CC	cmdutils.o
CC	ffprobe.o
AR	libavformat/libavformat.a
LD	ffmpeg_g
LD	ffplay_g
LD	ffserver_g
CP	ffplay
STRIP	ffplay
LD	ffprobe_g
CP	ffmpeg
STRIP	ffmpeg
CP	ffserver
STRIP	ffserver
CP	ffprobe
STRIP	ffprobe
    cmd exit value:   0

    cmd line: `git checkout img2_glob-inrecmental_patch'
    cmd standard error:
Switched to branch 'img2_glob-inrecmental_patch'
    cmd exit value:   0

    cmd line: `./ffmpeg -loglevel error -y -f image2 -i /tmp/ffglobtest/\*-frame%d\\[ABC\\].jpg /tmp/ffglobtest/video/S.avi'
    cmd exit value:   0

    cmd line: `./ffmpeg -loglevel error -y -f image2 -i /tmp/ffglobtest/\\\*-frame%d\\[ABC\\].jpg /tmp/ffglobtest/video/T.avi'
    cmd standard error:
[image2 @ 0x2ad43e0] Could not open file : /tmp/ffglobtest/\*-frame%d\[ABC\].jpg
/tmp/ffglobtest/\*-frame%d\[ABC\].jpg: could not find codec parameters
    cmd exit value:   1

    cmd line: `./ffmpeg -loglevel error -y -f image2 -i /tmp/ffglobtest/%\*-frame%d\\[\[A\]\[AB\]\[ABC\]\\].jpg /tmp/ffglobtest/video/T.avi'
    cmd exit value:   0

    cmd line: `./ffmpeg -loglevel error -y -f image2 -i /tmp/ffglobtest/%\*-frame%d\\[ABC\\].jpg /tmp/ffglobtest/video/U.avi'
    cmd exit value:   0

    cmd line: `./ffmpeg -loglevel error -y -f image2 -i /tmp/ffglobtest/%%%03d-frame%%d\\[ABC\\].jpg /tmp/ffglobtest/video/B.avi'
    cmd standard error:
[image2 @ 0x29c73e0] Could not open file : /tmp/ffglobtest/%%%03d-frame%%d\[ABC\].jpg
/tmp/ffglobtest/%%%03d-frame%%d\[ABC\].jpg: could not find codec parameters
    cmd exit value:   1

  result after 25s: success


  generate movie from image series without globbing

    cmd line: `git checkout 305e4b3'
    cmd standard error:
Note: checking out '305e4b3'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at 305e4b3... Merge remote-tracking branch 'qatar/master'
    cmd exit value:   0

    cmd line: `make -j3'
    cmd standard output:
CC	libavformat/img2dec.o
HTML	doc/ffmpeg.html
POD	doc/ffmpeg.pod
CC	cmdutils.o
CC	ffprobe.o
AR	libavformat/libavformat.a
MAN	doc/ffmpeg.1
LD	ffmpeg_g
LD	ffplay_g
LD	ffprobe_g
LD	ffserver_g
CP	ffmpeg
STRIP	ffmpeg
CP	ffprobe
STRIP	ffprobe
CP	ffplay
STRIP	ffplay
CP	ffserver
STRIP	ffserver
    cmd exit value:   0

    cmd line: `git checkout img2_glob-inrecmental_patch'
    cmd standard error:
Previous HEAD position was 305e4b3... Merge remote-tracking branch 'qatar/master'
Switched to branch 'img2_glob-inrecmental_patch'
    cmd exit value:   0

    cmd line: `./ffmpeg -loglevel error -y -f image2 -i /tmp/ffglobtest/%%%03d-frame%%d\[ABC\].jpg /tmp/ffglobtest/video/A.avi'
    cmd exit value:   0

  result after 20s: success


  checking results

    cmd line: `cmp /tmp/ffglobtest/video/A.avi /tmp/ffglobtest/video/C.avi'
    cmd exit value:   0

    cmd line: `cmp /tmp/ffglobtest/video/S.avi /tmp/ffglobtest/video/T.avi'
    cmd exit value:   0

    cmd line: `cmp /tmp/ffglobtest/video/X.avi /tmp/ffglobtest/video/Y.avi'
    cmd exit value:   0

    cmd line: `cmp /tmp/ffglobtest/video/Y.avi /tmp/ffglobtest/video/Z.avi'
    cmd exit value:   0

  result after 0s: success


  checking working copy state

    cmd line: `git branch | grep '*' | awk "{ print \$2 }"'
    cmd standard output:
img2_glob-inrecmental_patch
    cmd exit value:   0

    cmd line: `make -j3'
    cmd standard output:
CC	libavformat/img2dec.o
POD	doc/ffmpeg.pod
HTML	doc/ffmpeg.html
CC	cmdutils.o
CC	ffprobe.o
AR	libavformat/libavformat.a
MAN	doc/ffmpeg.1
LD	ffmpeg_g
LD	ffplay_g
LD	ffprobe_g
LD	ffserver_g
CP	ffmpeg
STRIP	ffmpeg
CP	ffplay
STRIP	ffplay
CP	ffprobe
STRIP	ffprobe
CP	ffserver
STRIP	ffserver
    cmd exit value:   0

  result after 16s: success




More information about the ffmpeg-devel mailing list