[FFmpeg-trac] #3056(undetermined:new): Non-ASCII chars in -passlogfile on Windows
FFmpeg
trac at avcodec.org
Fri Oct 18 00:08:33 CEST 2013
#3056: Non-ASCII chars in -passlogfile on Windows
-------------------------------------+-------------------------------------
Reporter: lelegard | Type: defect
Status: new | Priority: normal
Component: | Version: git-
undetermined | master
Keywords: | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-------------------------------------+-------------------------------------
This report describes a problem with ffmpeg on Windows with the
-passlogfile option using non-ASCII characters in the file path.
The problem was originally presented in the FFmpeg-user mailing list as
seen at [http://ffmpeg-users.933282.n4.nabble.com/Weird-characters-in-
created-passlogfile-on-Windows-td4661845.html]
I have not tested it yet on Linux but a reply in the mailing list suggests
that the problem does not exist on FreeBSD. Given the nature of the
problem, it is probably Windows-specific.
Consider a file path containing non-ASCII characters like è (e grave) in a
file or directory name. There are (at least) three different use cases for
a file name: input file, output file, pass log file.
In the first two cases, the non-ASCII characters in file names are
processed correctly. But in the case of the pass log file, the file name
is transformed into some weird sequence (typical untranslated UTF-8
sequence). The strange thing is that ffmpeg creates two files, the pass
log file and the output file, but processes the characters in the
corresponding file names quite differently. The way the two files are
created must be completely different.
See the following example. The host system is Windows 7 64-bit with some
Unix-like command line tools.
{{{
> ls -l x*
-rw-rw-rw- 1 user group 9400000 Oct 16 17:08 xè.ts
> ffmpeg -i xè.ts -pass 1 -passlogfile xè xè.mpg
ffmpeg version N-57176-g64b3aaf Copyright (c) 2000-2013 the FFmpeg
developers
built on Oct 15 2013 21:47:01 with gcc 4.8.1 (GCC)
configuration: --enable-gpl --enable-version3 --disable-w32threads
--enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r
--enable-gnutls --enable-iconv --enable-libass --enable-libbluray
--enable-libcaca --enable-libfreetype --enable-libgsm --enable-libilbc
--enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb
--enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-
librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex
--enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-
aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx
--enable-libwavpack --enable-libx264 --enable-libxavs --enable-libxvid
--enable-zlib
libavutil 52. 46.101 / 52. 46.101
libavcodec 55. 37.100 / 55. 37.100
libavformat 55. 19.102 / 55. 19.102
libavdevice 55. 4.100 / 55. 4.100
libavfilter 3. 88.101 / 3. 88.101
libswscale 2. 5.101 / 2. 5.101
libswresample 0. 17.103 / 0. 17.103
libpostproc 52. 3.100 / 52. 3.100
[h264 @ 000000000249e820] non-existing PPS referenced
[h264 @ 000000000249e820] non-existing PPS 0 referenced
[h264 @ 000000000249e820] decode_slice_header error
[h264 @ 000000000249e820] no frame!
[h264 @ 000000000249e820] non-existing PPS referenced
[h264 @ 000000000249e820] non-existing PPS 0 referenced
[h264 @ 000000000249e820] decode_slice_header error
[h264 @ 000000000249e820] no frame!
[h264 @ 000000000249e820] non-existing PPS referenced
[h264 @ 000000000249e820] non-existing PPS 0 referenced
[h264 @ 000000000249e820] decode_slice_header error
[h264 @ 000000000249e820] no frame!
[h264 @ 000000000249e820] non-existing PPS referenced
[h264 @ 000000000249e820] non-existing PPS 0 referenced
[h264 @ 000000000249e820] decode_slice_header error
[h264 @ 000000000249e820] no frame!
[h264 @ 000000000249e820] mmco: unref short failure
Last message repeated 2 times
[mpegts @ 00000000024aebe0] PES packet size mismatch
Last message repeated 2 times
Input #0, mpegts, from 'xè.ts':
Duration: 00:00:37.04, start: 42851.202589, bitrate: 2030 kb/s
Program 4006
Metadata:
service_name : ARTE
service_provider: Harmonic
Stream #0:0[0x424]: Video: h264 (Main) ([27][0][0][0] / 0x001B),
yuv420p(tv, bt470bg), 720x576 [SAR 16:11 DAR 20:11], 25 fps, 25
tbr, 90k tbn, 50 tbc
Stream #0:1[0x425](fra): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz,
stereo, s16p, 128 kb/s
Stream #0:2[0x426](eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz,
stereo, s16p, 128 kb/s
Stream #0:3[0x427](deu): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz,
mono, s16p, 64 kb/s
Stream #0:4[0x42b](qad): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz,
mono, s16p, 64 kb/s (visual impaired)
Stream #0:5[0x42c](fra): Subtitle: dvb_teletext ([6][0][0][0] /
0x0006)
[mpeg @ 00000000048e9980] VBV buffer size not set, muxing may fail
Output #0, mpeg, to 'xè.mpg':
Metadata:
encoder : Lavf55.19.102
Stream #0:0: Video: mpeg1video, yuv420p, 720x576 [SAR 16:11 DAR
20:11], q=2-31, pass 1, 200 kb/s, 90k tbn, 25 tbc
Stream #0:1(fra): Audio: mp2, 48000 Hz, stereo, s16, 128 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (h264 -> mpeg1video)
Stream #0:1 -> #0:1 (mp2 -> mp2)
Press [q] to stop, [?] for help
[h264 @ 0000000004bb2280] reference picture missing during reorder
Last message repeated 1 times
[h264 @ 0000000004bb2280] Missing reference picture, default is 0
[h264 @ 0000000004bb2280] decode_slice_header error
[h264 @ 00000000048cf8a0] mmco: unref short failure
[h264 @ 0000000004bb2280] mmco: unref short failure
[mpegts @ 00000000024aebe0] PES packet size mismatch0:34.60 bitrate=
451.3kbits/s dup=18 drop=0
[h264 @ 00000000048d01c0] error while decoding MB 22 28, bytestream
(-5).8kbits/s dup=18 drop=0
[mp2 @ 000000000249ec80] incomplete frame
[h264 @ 00000000048d01c0] concealing 387 DC, 387 AC, 387 MV errors in B
frame
Error while decoding stream #0:1: Invalid data found when processing input
frame= 923 fps=186 q=31.0 Lsize= 2046kB time=00:00:36.88 bitrate=
454.5kbits/s dup=20 drop=0
video:1455kB audio:571kB subtitle:0 global headers:0kB muxing overhead
0.990720%
> ls -l x*
-rw-rw-rw- 1 user group 128580 Oct 16 17:15 xA"-0.log
-rw-rw-rw- 1 user group 2095104 Oct 16 17:15 xè.mpg
-rw-rw-rw- 1 user group 9400000 Oct 16 17:08 xè.ts
}}}
The command contains "-passlogfile xè xè.mpg" but the "xè" in the two
created files resulted in two different encodings.
Of course, Windows is subject to the old "character code page" issues. But
this cannot be the sole explanation. FFmpeg seems to correctly interpret
the command line arguments since it successfully opens the input file and
creates the output file with the right encoding. There is no rational
reason that the arguments "-passlogfile xè" would not be read the same
way. However, when it comes to creating the log file, the name "xè" is
specifically transformed into "xA"". When creating the output file
"xè.mpg", such name transformation does not occur.
The problem is that the specified file name on the command line does NOT
correspond to the actual file name in the file system. One may say that we
don't care since this is a temporary log file.
But there is a real functional problem when the non-ASCII character is in
the name of a directory. In that case, ffmpeg transforms the directory
name and the resulting path is invalid since the transformed path is a
non-existent directory. And ffmpeg fails...
See this example:
{{{
> mkdir xèdir
> ls -ld x*
-rw-rw-rw- 1 user group 9400000 Oct 16 17:08 xè.ts
drwxrwxrwx 1 user group 0 Oct 16 17:15 xèdir
> ffmpeg -i xè.ts -pass 1 -passlogfile xèdir\log xè.mpg
ffmpeg version N-57176-g64b3aaf Copyright (c) 2000-2013 the FFmpeg
developers
built on Oct 15 2013 21:47:01 with gcc 4.8.1 (GCC)
configuration: --enable-gpl --enable-version3 --disable-w32threads
--enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r
--enable-gnutls --enable-iconv --enable-libass --enable-libbluray
--enable-libcaca --enable-libfreetype --enable-libgsm --enable-libilbc
--enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb
--enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-
librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex
--enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-
aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx
--enable-libwavpack --enable-libx264 --enable-libxavs --enable-libxvid
--enable-zlib
libavutil 52. 46.101 / 52. 46.101
libavcodec 55. 37.100 / 55. 37.100
libavformat 55. 19.102 / 55. 19.102
libavdevice 55. 4.100 / 55. 4.100
libavfilter 3. 88.101 / 3. 88.101
libswscale 2. 5.101 / 2. 5.101
libswresample 0. 17.103 / 0. 17.103
libpostproc 52. 3.100 / 52. 3.100
[h264 @ 000000000031aa40] non-existing PPS referenced
[h264 @ 000000000031aa40] non-existing PPS 0 referenced
[h264 @ 000000000031aa40] decode_slice_header error
[h264 @ 000000000031aa40] no frame!
[h264 @ 000000000031aa40] non-existing PPS referenced
[h264 @ 000000000031aa40] non-existing PPS 0 referenced
[h264 @ 000000000031aa40] decode_slice_header error
[h264 @ 000000000031aa40] no frame!
[h264 @ 000000000031aa40] non-existing PPS referenced
[h264 @ 000000000031aa40] non-existing PPS 0 referenced
[h264 @ 000000000031aa40] decode_slice_header error
[h264 @ 000000000031aa40] no frame!
[h264 @ 000000000031aa40] non-existing PPS referenced
[h264 @ 000000000031aa40] non-existing PPS 0 referenced
[h264 @ 000000000031aa40] decode_slice_header error
[h264 @ 000000000031aa40] no frame!
[h264 @ 000000000031aa40] mmco: unref short failure
Last message repeated 2 times
[mpegts @ 000000000031a4a0] PES packet size mismatch
Last message repeated 2 times
Input #0, mpegts, from 'xè.ts':
Duration: 00:00:37.04, start: 42851.202589, bitrate: 2030 kb/s
Program 4006
Metadata:
service_name : ARTE
service_provider: Harmonic
Stream #0:0[0x424]: Video: h264 (Main) ([27][0][0][0] / 0x001B),
yuv420p(tv, bt470bg), 720x576 [SAR 16:11 DAR 20:11], 25 fps, 25
tbr, 90k tbn, 50 tbc
Stream #0:1[0x425](fra): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz,
stereo, s16p, 128 kb/s
Stream #0:2[0x426](eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz,
stereo, s16p, 128 kb/s
Stream #0:3[0x427](deu): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz,
mono, s16p, 64 kb/s
Stream #0:4[0x42b](qad): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz,
mono, s16p, 64 kb/s (visual impaired)
Stream #0:5[0x42c](fra): Subtitle: dvb_teletext ([6][0][0][0] /
0x0006)
Cannot write log file 'xèdir\log-0.log' for pass-1 encoding: No such file
or directory
>
}}}
In this case, ffmpeg fails since the log file cannot be created.
Obvious workarounds exist, including using the old DOS 8.3 "short path
name" for the directory parts which contain non-ASCII characters.
Example:
{{{
> dir/x x*
2013-10-16 17:08 9 400 000 X2691~1.TS xè.ts
2013-10-16 17:15 <DIR> XDIR~1 xèdir
> ffmpeg -i xè.ts -pass 1 -passlogfile xdir~1\log xè.mpg
ffmpeg version N-57176-g64b3aaf Copyright (c) 2000-2013 the FFmpeg
developers
built on Oct 15 2013 21:47:01 with gcc 4.8.1 (GCC)
configuration: --enable-gpl --enable-version3 --disable-w32threads
--enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r
--enable-gnutls --enable-iconv --enable-libass --enable-libbluray
--enable-libcaca --enable-libfreetype --enable-libgsm --enable-libilbc
--enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb
--enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-
librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex
--enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-
aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx
--enable-libwavpack --enable-libx264 --enable-libxavs --enable-libxvid
--enable-zlib
libavutil 52. 46.101 / 52. 46.101
libavcodec 55. 37.100 / 55. 37.100
libavformat 55. 19.102 / 55. 19.102
libavdevice 55. 4.100 / 55. 4.100
libavfilter 3. 88.101 / 3. 88.101
libswscale 2. 5.101 / 2. 5.101
libswresample 0. 17.103 / 0. 17.103
libpostproc 52. 3.100 / 52. 3.100
[h264 @ 00000000003aaa40] non-existing PPS referenced
[h264 @ 00000000003aaa40] non-existing PPS 0 referenced
[h264 @ 00000000003aaa40] decode_slice_header error
[h264 @ 00000000003aaa40] no frame!
[h264 @ 00000000003aaa40] non-existing PPS referenced
[h264 @ 00000000003aaa40] non-existing PPS 0 referenced
[h264 @ 00000000003aaa40] decode_slice_header error
[h264 @ 00000000003aaa40] no frame!
[h264 @ 00000000003aaa40] non-existing PPS referenced
[h264 @ 00000000003aaa40] non-existing PPS 0 referenced
[h264 @ 00000000003aaa40] decode_slice_header error
[h264 @ 00000000003aaa40] no frame!
[h264 @ 00000000003aaa40] non-existing PPS referenced
[h264 @ 00000000003aaa40] non-existing PPS 0 referenced
[h264 @ 00000000003aaa40] decode_slice_header error
[h264 @ 00000000003aaa40] no frame!
[h264 @ 00000000003aaa40] mmco: unref short failure
Last message repeated 2 times
[mpegts @ 00000000003aa4a0] PES packet size mismatch
Last message repeated 2 times
Input #0, mpegts, from 'xè.ts':
Duration: 00:00:37.04, start: 42851.202589, bitrate: 2030 kb/s
Program 4006
Metadata:
service_name : ARTE
service_provider: Harmonic
Stream #0:0[0x424]: Video: h264 (Main) ([27][0][0][0] / 0x001B),
yuv420p(tv, bt470bg), 720x576 [SAR 16:11 DAR 20:11], 25 fps, 25
tbr, 90k tbn, 50 tbc
Stream #0:1[0x425](fra): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz,
stereo, s16p, 128 kb/s
Stream #0:2[0x426](eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz,
stereo, s16p, 128 kb/s
Stream #0:3[0x427](deu): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz,
mono, s16p, 64 kb/s
Stream #0:4[0x42b](qad): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz,
mono, s16p, 64 kb/s (visual impaired)
Stream #0:5[0x42c](fra): Subtitle: dvb_teletext ([6][0][0][0] /
0x0006)
[mpeg @ 0000000004b662c0] VBV buffer size not set, muxing may fail
Output #0, mpeg, to 'xè.mpg':
Metadata:
encoder : Lavf55.19.102
Stream #0:0: Video: mpeg1video, yuv420p, 720x576 [SAR 16:11 DAR
20:11], q=2-31, pass 1, 200 kb/s, 90k tbn, 25 tbc
Stream #0:1(fra): Audio: mp2, 48000 Hz, stereo, s16, 128 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (h264 -> mpeg1video)
Stream #0:1 -> #0:1 (mp2 -> mp2)
Press [q] to stop, [?] for help
[h264 @ 0000000004a074c0] reference picture missing during reorder
Last message repeated 1 times
[h264 @ 0000000004a074c0] Missing reference picture, default is 0
[h264 @ 0000000004a074c0] decode_slice_header error
[h264 @ 0000000004a07de0] mmco: unref short failure
[h264 @ 0000000004a074c0] mmco: unref short failure
[mpegts @ 00000000003aa4a0] PES packet size mismatch0:35.36 bitrate=
453.2kbits/s dup=18 drop=0
[mp2 @ 000000000249ec40] incomplete frame
Error while decoding stream #0:1: Invalid data found when processing input
[h264 @ 0000000004a08700] error while decoding MB 22 28, bytestream (-5)
[h264 @ 0000000004a08700] concealing 387 DC, 387 AC, 387 MV errors in B
frame
frame= 923 fps=432 q=31.0 Lsize= 2046kB time=00:00:36.88 bitrate=
454.5kbits/s dup=20 drop=0
video:1455kB audio:571kB subtitle:0 global headers:0kB muxing overhead
0.990720%
> ls -ld x*
-rw-rw-rw- 1 user group 2095104 Oct 16 17:40 xè.mpg
-rw-rw-rw- 1 user group 9400000 Oct 16 17:08 xè.ts
drwxrwxrwx 1 user group 0 Oct 16 17:40 xèdir
> ls -l xèdir
total 126
-rw-rw-rw- 1 user group 128580 Oct 16 17:40 log-0.log
>
}}}
I understand that this kind of Windows oddity may not be too interesting
to FFmpeg developers but it can be quite annoying for Windows users.
As far as I am concerned, I use ffmpeg on Windows mainly through the GUI
front-end named QtlMovie. All ffmpeg commands are generated by the front-
end and I implemented a specific workaround for this problem. In the
specific case of -passlogfile on Windows, I first translate the directory
path of the log file with its DOS 8.3 counterpart. But this is just a non-
portable workaround, pushing the problem into the ffmpeg user's code.
--
Ticket URL: <https://ffmpeg.org/trac/ffmpeg/ticket/3056>
FFmpeg <http://ffmpeg.org>
FFmpeg issue tracker
More information about the FFmpeg-trac
mailing list