V360 stabilization
Hello all, I've programmed a C# workaround for stabilization of 360° videos. The procedure is as follows: 1. FFmpeg: From each frame of the equirectangular input video, extract two small images which are 90° apart in the input video. I call them A and B images. 2. C# code: Analyze the x and y image shift from subsequent A and B images. Calculate how the equirectangular frames must be rotated (yaw, pitch, roll) to compensate the image shifts. This part wasn't easy. Two rotation matrices and one matrix multiplication are required. Write the results to a *.cmd file. 3. FFmpeg: Read the *.cmd file and apply the rotations with the v360 filter. The output video is stabilized. For details and source code please have a look at chapter 2.78 in my book: http://www.astro-electronic.de/FFmpeg_Book.pdf If anyone wants to implement this in FFmpeg, please feel free to do it. Michael
On Tue, Sep 29, 2020 at 10:54:54PM +0200, Michael Koch wrote:
Hello all,
I've programmed a C# workaround for stabilization of 360° videos. The procedure is as follows:
1. FFmpeg: From each frame of the equirectangular input video, extract two small images which are 90° apart in the input video. I call them A and B images.
2. C# code: Analyze the x and y image shift from subsequent A and B images. Calculate how the equirectangular frames must be rotated (yaw, pitch, roll) to compensate the image shifts. This part wasn't easy. Two rotation matrices and one matrix multiplication are required. Write the results to a *.cmd file.
3. FFmpeg: Read the *.cmd file and apply the rotations with the v360 filter. The output video is stabilized.
For details and source code please have a look at chapter 2.78 in my book: http://www.astro-electronic.de/FFmpeg_Book.pdf
If anyone wants to implement this in FFmpeg, please feel free to do it.
Better upload DNG files that do not decode with FFmpeg.
Am 01.10.2020 um 22:11 schrieb Paul B Mahol:
On Tue, Sep 29, 2020 at 10:54:54PM +0200, Michael Koch wrote:
Hello all,
I've programmed a C# workaround for stabilization of 360° videos. The procedure is as follows:
1. FFmpeg: From each frame of the equirectangular input video, extract two small images which are 90° apart in the input video. I call them A and B images.
2. C# code: Analyze the x and y image shift from subsequent A and B images. Calculate how the equirectangular frames must be rotated (yaw, pitch, roll) to compensate the image shifts. This part wasn't easy. Two rotation matrices and one matrix multiplication are required. Write the results to a *.cmd file.
3. FFmpeg: Read the *.cmd file and apply the rotations with the v360 filter. The output video is stabilized.
For details and source code please have a look at chapter 2.78 in my book: http://www.astro-electronic.de/FFmpeg_Book.pdf
If anyone wants to implement this in FFmpeg, please feel free to do it. Better upload DNG files that do not decode with FFmpeg.
In this message http://ffmpeg.org/pipermail/ffmpeg-user/2020-August/049681.html you find a link to many DNG images which FFmpeg can't decode correctly. There is no error message, but the result is much too dark with low saturation. I did convert a RAW image from a Canon 6D to DNG with Adobe DNG Converter V12.4. FFmpeg is unable to decode this DNG image. See this message for details: http://ffmpeg.org/pipermail/ffmpeg-user/2020-August/049738.html You can download the DNG image here (I will delete it from my webspace in a few days): www.astro-electronic.de/IMG_3459.dng I did also try a RAW image from a Canon 5D-MK4 with the same negative result. A friend gave me a DNG image that was written by his Pentax K5 camera. Same negative result. Summary: I did try DNG images from 4 different sources and in 4 of 4 cases FFmpeg failed. Michael
On Thu, Oct 01, 2020 at 11:40:57PM +0200, Michael Koch wrote:
Am 01.10.2020 um 22:11 schrieb Paul B Mahol:
On Tue, Sep 29, 2020 at 10:54:54PM +0200, Michael Koch wrote:
Hello all,
I've programmed a C# workaround for stabilization of 360° videos. The procedure is as follows:
1. FFmpeg: From each frame of the equirectangular input video, extract two small images which are 90° apart in the input video. I call them A and B images.
2. C# code: Analyze the x and y image shift from subsequent A and B images. Calculate how the equirectangular frames must be rotated (yaw, pitch, roll) to compensate the image shifts. This part wasn't easy. Two rotation matrices and one matrix multiplication are required. Write the results to a *.cmd file.
3. FFmpeg: Read the *.cmd file and apply the rotations with the v360 filter. The output video is stabilized.
For details and source code please have a look at chapter 2.78 in my book: http://www.astro-electronic.de/FFmpeg_Book.pdf
If anyone wants to implement this in FFmpeg, please feel free to do it. Better upload DNG files that do not decode with FFmpeg.
In this message http://ffmpeg.org/pipermail/ffmpeg-user/2020-August/049681.html you find a link to many DNG images which FFmpeg can't decode correctly. There is no error message, but the result is much too dark with low saturation.
Use correct player like mpv, which does not ignore color_trc.
I did convert a RAW image from a Canon 6D to DNG with Adobe DNG Converter V12.4. FFmpeg is unable to decode this DNG image. See this message for details: http://ffmpeg.org/pipermail/ffmpeg-user/2020-August/049738.html You can download the DNG image here (I will delete it from my webspace in a few days): www.astro-electronic.de/IMG_3459.dng
I did also try a RAW image from a Canon 5D-MK4 with the same negative result.
A friend gave me a DNG image that was written by his Pentax K5 camera. Same negative result.
Summary: I did try DNG images from 4 different sources and in 4 of 4 cases
In my testcases, 30 out of 30 DNGs decoded just fine.
FFmpeg failed.
Michael
_______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-user
To unsubscribe, visit link above, or email ffmpeg-user-request@ffmpeg.org with subject "unsubscribe".
Am 02.10.2020 um 10:10 schrieb Paul B Mahol:
On Thu, Oct 01, 2020 at 11:40:57PM +0200, Michael Koch wrote:
Am 01.10.2020 um 22:11 schrieb Paul B Mahol:
On Tue, Sep 29, 2020 at 10:54:54PM +0200, Michael Koch wrote:
Hello all,
I've programmed a C# workaround for stabilization of 360° videos. The procedure is as follows:
1. FFmpeg: From each frame of the equirectangular input video, extract two small images which are 90° apart in the input video. I call them A and B images.
2. C# code: Analyze the x and y image shift from subsequent A and B images. Calculate how the equirectangular frames must be rotated (yaw, pitch, roll) to compensate the image shifts. This part wasn't easy. Two rotation matrices and one matrix multiplication are required. Write the results to a *.cmd file.
3. FFmpeg: Read the *.cmd file and apply the rotations with the v360 filter. The output video is stabilized.
For details and source code please have a look at chapter 2.78 in my book: http://www.astro-electronic.de/FFmpeg_Book.pdf
If anyone wants to implement this in FFmpeg, please feel free to do it. Better upload DNG files that do not decode with FFmpeg. In this message http://ffmpeg.org/pipermail/ffmpeg-user/2020-August/049681.html you find a link to many DNG images which FFmpeg can't decode correctly. There is no error message, but the result is much too dark with low saturation. Use correct player like mpv, which does not ignore color_trc.
A player? The output is a jpg image and it's too dark. ffmpeg -i input.dng output.jpg
I did convert a RAW image from a Canon 6D to DNG with Adobe DNG Converter V12.4. FFmpeg is unable to decode this DNG image. See this message for details: http://ffmpeg.org/pipermail/ffmpeg-user/2020-August/049738.html You can download the DNG image here (I will delete it from my webspace in a few days): www.astro-electronic.de/IMG_3459.dng
I did also try a RAW image from a Canon 5D-MK4 with the same negative result.
A friend gave me a DNG image that was written by his Pentax K5 camera. Same negative result.
Summary: I did try DNG images from 4 different sources and in 4 of 4 cases In my testcases, 30 out of 30 DNGs decoded just fine.
Did any of your 30 DNGs come from Adobe DNG converter V12.4, or from a Pentax K5 camera? Michael
On Fri, Oct 02, 2020 at 07:59:12PM +0200, Michael Koch wrote:
Am 02.10.2020 um 10:10 schrieb Paul B Mahol:
On Thu, Oct 01, 2020 at 11:40:57PM +0200, Michael Koch wrote:
Am 01.10.2020 um 22:11 schrieb Paul B Mahol:
On Tue, Sep 29, 2020 at 10:54:54PM +0200, Michael Koch wrote:
Hello all,
I've programmed a C# workaround for stabilization of 360° videos. The procedure is as follows:
1. FFmpeg: From each frame of the equirectangular input video, extract two small images which are 90° apart in the input video. I call them A and B images.
2. C# code: Analyze the x and y image shift from subsequent A and B images. Calculate how the equirectangular frames must be rotated (yaw, pitch, roll) to compensate the image shifts. This part wasn't easy. Two rotation matrices and one matrix multiplication are required. Write the results to a *.cmd file.
3. FFmpeg: Read the *.cmd file and apply the rotations with the v360 filter. The output video is stabilized.
For details and source code please have a look at chapter 2.78 in my book: http://www.astro-electronic.de/FFmpeg_Book.pdf
If anyone wants to implement this in FFmpeg, please feel free to do it. Better upload DNG files that do not decode with FFmpeg. In this message http://ffmpeg.org/pipermail/ffmpeg-user/2020-August/049681.html you find a link to many DNG images which FFmpeg can't decode correctly. There is no error message, but the result is much too dark with low saturation. Use correct player like mpv, which does not ignore color_trc.
A player? The output is a jpg image and it's too dark.
Yes, player, ffmpeg is not correct in conversion. Because swscale is old and ignores color_trc metadata that is required for correct display. Use zscale instead for converting pixel formats.
ffmpeg -i input.dng output.jpg
I did convert a RAW image from a Canon 6D to DNG with Adobe DNG Converter V12.4. FFmpeg is unable to decode this DNG image. See this message for details: http://ffmpeg.org/pipermail/ffmpeg-user/2020-August/049738.html You can download the DNG image here (I will delete it from my webspace in a few days): www.astro-electronic.de/IMG_3459.dng
I did also try a RAW image from a Canon 5D-MK4 with the same negative result.
A friend gave me a DNG image that was written by his Pentax K5 camera. Same negative result.
Summary: I did try DNG images from 4 different sources and in 4 of 4 cases In my testcases, 30 out of 30 DNGs decoded just fine.
Did any of your 30 DNGs come from Adobe DNG converter V12.4, or from a Pentax K5 camera?
No, as that converter tends to create invalid DNG files. But thanks for uploading only one DNG file, I added support for it.
Michael
_______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-user
To unsubscribe, visit link above, or email ffmpeg-user-request@ffmpeg.org with subject "unsubscribe".
Am 02.10.2020 um 22:01 schrieb Paul B Mahol:
Use correct player like mpv, which does not ignore color_trc. A player? The output is a jpg image and it's too dark. Yes, player, ffmpeg is not correct in conversion. Because swscale is old and ignores color_trc metadata that is required for correct display.
Use zscale instead for converting pixel formats.
Can you please give an example how to convert a DNG image to a JPG image with FFmpeg? Michael
Use correct player like mpv, which does not ignore color_trc. A player? The output is a jpg image and it's too dark. Yes, player, ffmpeg is not correct in conversion. Because swscale is old and ignores color_trc metadata that is required for correct display.
Use zscale instead for converting pixel formats.
Can you please give an example how to convert a DNG image to a JPG image with FFmpeg?
I did try ffmpeg -i sample_r00000.dng -vf zscale -y out.jpg but it gives an error message. The input image is from https://drive.google.com/drive/folders/1u5m7aVAFPpEsL4YtDVsfSpfcve9PFG4m and IrfanView has no problem to open this image. Michael C:\Users\astro\Desktop\dng>c:\ffmpeg\ffmpeg -i sample_r00000.dng -vf zscale -y out.jpg ffmpeg version 2020-09-20-git-ef29e5bf42-full_build-www.gyan.dev Copyright (c) 2000-2020 the FFmpeg developers built with gcc 10.2.0 (Rev3, Built by MSYS2 project) configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libglslang --enable-vulkan --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libwavpack --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint libavutil 56. 59.100 / 56. 59.100 libavcodec 58.106.100 / 58.106.100 libavformat 58. 58.100 / 58. 58.100 libavdevice 58. 11.102 / 58. 11.102 libavfilter 7. 87.100 / 7. 87.100 libswscale 5. 8.100 / 5. 8.100 libswresample 3. 8.100 / 3. 8.100 libpostproc 55. 8.100 / 55. 8.100 [tiff @ 0000025f5dbdf680] Assuming black level pattern values are identical [mjpeg @ 0000025f5dbe4ec0] mjpeg_decode_dc: bad vlc: 0:0 (0000025f5dbe5588) Last message repeated 1 times [tiff_pipe @ 0000025f5dbdd640] Stream #0: not enough frames to estimate rate; consider increasing probesize Input #0, tiff_pipe, from 'sample_r00000.dng': Duration: N/A, bitrate: N/A Stream #0:0: Video: tiff, bayer_rggb16le, 6016x3200, 25 tbr, 25 tbn, 25 tbc Stream mapping: Stream #0:0 -> #0:0 (tiff (native) -> mjpeg (native)) Press [q] to stop, [?] for help [tiff @ 0000025f5dbefe40] Assuming black level pattern values are identical [mjpeg @ 0000025f5dc15180] mjpeg_decode_dc: bad vlc: 0:0 (0000025f5dc38248) Last message repeated 1 times code 3074: no path between colorspaces Error while filtering: Generic error in an external library Failed to inject frame into filter network: Generic error in an external library Error while processing the decoded data for stream #0:0 Conversion failed!
On Sat, Oct 03, 2020 at 08:27:20AM +0200, Michael Koch wrote:
Use correct player like mpv, which does not ignore color_trc. A player? The output is a jpg image and it's too dark. Yes, player, ffmpeg is not correct in conversion. Because swscale is old and ignores color_trc metadata that is required for correct display.
Use zscale instead for converting pixel formats.
Can you please give an example how to convert a DNG image to a JPG image with FFmpeg?
I did try ffmpeg -i sample_r00000.dng -vf zscale -y out.jpg but it gives an error message. The input image is from https://drive.google.com/drive/folders/1u5m7aVAFPpEsL4YtDVsfSpfcve9PFG4m and IrfanView has no problem to open this image.
Then join IrfanView mailing list. Every single DNG file you posted is invalid in some part.
Michael
C:\Users\astro\Desktop\dng>c:\ffmpeg\ffmpeg -i sample_r00000.dng -vf zscale -y out.jpg ffmpeg version 2020-09-20-git-ef29e5bf42-full_build-www.gyan.dev Copyright (c) 2000-2020 the FFmpeg developers built with gcc 10.2.0 (Rev3, Built by MSYS2 project) configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libglslang --enable-vulkan --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libwavpack --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint libavutil 56. 59.100 / 56. 59.100 libavcodec 58.106.100 / 58.106.100 libavformat 58. 58.100 / 58. 58.100 libavdevice 58. 11.102 / 58. 11.102 libavfilter 7. 87.100 / 7. 87.100 libswscale 5. 8.100 / 5. 8.100 libswresample 3. 8.100 / 3. 8.100 libpostproc 55. 8.100 / 55. 8.100 [tiff @ 0000025f5dbdf680] Assuming black level pattern values are identical [mjpeg @ 0000025f5dbe4ec0] mjpeg_decode_dc: bad vlc: 0:0 (0000025f5dbe5588) Last message repeated 1 times [tiff_pipe @ 0000025f5dbdd640] Stream #0: not enough frames to estimate rate; consider increasing probesize Input #0, tiff_pipe, from 'sample_r00000.dng': Duration: N/A, bitrate: N/A Stream #0:0: Video: tiff, bayer_rggb16le, 6016x3200, 25 tbr, 25 tbn, 25 tbc Stream mapping: Stream #0:0 -> #0:0 (tiff (native) -> mjpeg (native)) Press [q] to stop, [?] for help [tiff @ 0000025f5dbefe40] Assuming black level pattern values are identical [mjpeg @ 0000025f5dc15180] mjpeg_decode_dc: bad vlc: 0:0 (0000025f5dc38248) Last message repeated 1 times code 3074: no path between colorspaces Error while filtering: Generic error in an external library Failed to inject frame into filter network: Generic error in an external library Error while processing the decoded data for stream #0:0 Conversion failed!
_______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-user
To unsubscribe, visit link above, or email ffmpeg-user-request@ffmpeg.org with subject "unsubscribe".
Am 03.10.2020 um 13:25 schrieb Paul B Mahol:
On Sat, Oct 03, 2020 at 08:27:20AM +0200, Michael Koch wrote:
Use correct player like mpv, which does not ignore color_trc. A player? The output is a jpg image and it's too dark. Yes, player, ffmpeg is not correct in conversion. Because swscale is old and ignores color_trc metadata that is required for correct display.
Use zscale instead for converting pixel formats. Can you please give an example how to convert a DNG image to a JPG image with FFmpeg? I did try ffmpeg -i sample_r00000.dng -vf zscale -y out.jpg but it gives an error message. The input image is from https://drive.google.com/drive/folders/1u5m7aVAFPpEsL4YtDVsfSpfcve9PFG4m and IrfanView has no problem to open this image. Then join IrfanView mailing list.
Why should I? IrfanView has no problem.
Every single DNG file you posted is invalid in some part.
Adobe is the creator of the DNG format and it sounds quite unlikely that their own DNG Converter produces invalid DNG files. I tested that the files can be opened with IrfanView, Windows Media Player, Fitswork, 3D LUT Creator and GIMP with Darktable plugin. I saw that you fixed it (thanks for that!) and will test it soon. The images from the above link are a different thing, as I don't know how they were created. I agree that they might be invalid. Michael
Am 29.09.2020 um 22:54 schrieb Michael Koch:
Hello all,
I've programmed a C# workaround for stabilization of 360° videos. The procedure is as follows:
1. FFmpeg: From each frame of the equirectangular input video, extract two small images which are 90° apart in the input video. I call them A and B images.
2. C# code: Analyze the x and y image shift from subsequent A and B images. Calculate how the equirectangular frames must be rotated (yaw, pitch, roll) to compensate the image shifts. This part wasn't easy. Two rotation matrices and one matrix multiplication are required. Write the results to a *.cmd file.
3. FFmpeg: Read the *.cmd file and apply the rotations with the v360 filter. The output video is stabilized.
For details and source code please have a look at chapter 2.78 in my book: http://www.astro-electronic.de/FFmpeg_Book.pdf
If anyone wants to implement this in FFmpeg, please feel free to do it.
I've written and tested an improved version for 360° stabilization, it's in chapter 2.79. Michael
On 10/11/20, Michael Koch <astroelectronic@t-online.de> wrote:
Am 29.09.2020 um 22:54 schrieb Michael Koch:
Hello all,
I've programmed a C# workaround for stabilization of 360° videos. The procedure is as follows:
1. FFmpeg: From each frame of the equirectangular input video, extract two small images which are 90° apart in the input video. I call them A and B images.
2. C# code: Analyze the x and y image shift from subsequent A and B images. Calculate how the equirectangular frames must be rotated (yaw, pitch, roll) to compensate the image shifts. This part wasn't easy. Two rotation matrices and one matrix multiplication are required. Write the results to a *.cmd file.
3. FFmpeg: Read the *.cmd file and apply the rotations with the v360 filter. The output video is stabilized.
For details and source code please have a look at chapter 2.78 in my book: http://www.astro-electronic.de/FFmpeg_Book.pdf
If anyone wants to implement this in FFmpeg, please feel free to do it.
I've written and tested an improved version for 360° stabilization, it's in chapter 2.79.
Your book is full or factual errors, Notice to anyone: do not use this thing for anything serious.
Am 03.11.2022 um 09:35 schrieb Paul B Mahol:
On 10/11/20, Michael Koch <astroelectronic@t-online.de> wrote:
Am 29.09.2020 um 22:54 schrieb Michael Koch:
Hello all,
I've programmed a C# workaround for stabilization of 360° videos. The procedure is as follows:
1. FFmpeg: From each frame of the equirectangular input video, extract two small images which are 90° apart in the input video. I call them A and B images.
2. C# code: Analyze the x and y image shift from subsequent A and B images. Calculate how the equirectangular frames must be rotated (yaw, pitch, roll) to compensate the image shifts. This part wasn't easy. Two rotation matrices and one matrix multiplication are required. Write the results to a *.cmd file.
3. FFmpeg: Read the *.cmd file and apply the rotations with the v360 filter. The output video is stabilized.
For details and source code please have a look at chapter 2.78 in my book: http://www.astro-electronic.de/FFmpeg_Book.pdf
If anyone wants to implement this in FFmpeg, please feel free to do it. I've written and tested an improved version for 360° stabilization, it's in chapter 2.79. Your book is full or factual errors,
Notice to anyone: do not use this thing for anything serious.
Thank you for so much promotion for my book. Sure, it's likely that in more than 900 pages there are a few errors. We all make errors. If anyone finds an error, please let me know. You find my e-mail address on the first page. http://www.astro-electronic.de/FFmpeg_Book.pdf Michael
On 11/3/22, Michael Koch <astroelectronic@t-online.de> wrote:
Am 03.11.2022 um 09:35 schrieb Paul B Mahol:
On 10/11/20, Michael Koch <astroelectronic@t-online.de> wrote:
Am 29.09.2020 um 22:54 schrieb Michael Koch:
Hello all,
I've programmed a C# workaround for stabilization of 360° videos. The procedure is as follows:
1. FFmpeg: From each frame of the equirectangular input video, extract two small images which are 90° apart in the input video. I call them A and B images.
2. C# code: Analyze the x and y image shift from subsequent A and B images. Calculate how the equirectangular frames must be rotated (yaw, pitch, roll) to compensate the image shifts. This part wasn't easy. Two rotation matrices and one matrix multiplication are required. Write the results to a *.cmd file.
3. FFmpeg: Read the *.cmd file and apply the rotations with the v360 filter. The output video is stabilized.
For details and source code please have a look at chapter 2.78 in my book: http://www.astro-electronic.de/FFmpeg_Book.pdf
If anyone wants to implement this in FFmpeg, please feel free to do it. I've written and tested an improved version for 360° stabilization, it's in chapter 2.79. Your book is full or factual errors,
Notice to anyone: do not use this thing for anything serious.
Thank you for so much promotion for my book. Sure, it's likely that in more than 900 pages there are a few errors. We all make errors. If anyone finds an error, please let me know. You find my e-mail address on the first page.
Some things in sea of myriad others: Why you claim that FFmpeg does not have SER format support while in fact it have support for it? Why you claim that datascope does not support >8 bit formats where in fact it does support it? Why you claim that drawbox does not support RGB formats while in fact it does support them? I think this is enough, I could continue this forever. I could bet there is at least one and sometimes even much more factual errors on every single page of this book that mentions FFmpeg directly or indirectly.
Am 03.11.2022 um 10:10 schrieb Paul B Mahol:
On 11/3/22, Michael Koch <astroelectronic@t-online.de> wrote:
Am 03.11.2022 um 09:35 schrieb Paul B Mahol:
On 10/11/20, Michael Koch <astroelectronic@t-online.de> wrote:
Am 29.09.2020 um 22:54 schrieb Michael Koch:
Hello all,
I've programmed a C# workaround for stabilization of 360° videos. The procedure is as follows:
1. FFmpeg: From each frame of the equirectangular input video, extract two small images which are 90° apart in the input video. I call them A and B images.
2. C# code: Analyze the x and y image shift from subsequent A and B images. Calculate how the equirectangular frames must be rotated (yaw, pitch, roll) to compensate the image shifts. This part wasn't easy. Two rotation matrices and one matrix multiplication are required. Write the results to a *.cmd file.
3. FFmpeg: Read the *.cmd file and apply the rotations with the v360 filter. The output video is stabilized.
For details and source code please have a look at chapter 2.78 in my book: http://www.astro-electronic.de/FFmpeg_Book.pdf
If anyone wants to implement this in FFmpeg, please feel free to do it. I've written and tested an improved version for 360° stabilization, it's in chapter 2.79. Your book is full or factual errors,
Notice to anyone: do not use this thing for anything serious. Thank you for so much promotion for my book. Sure, it's likely that in more than 900 pages there are a few errors. We all make errors. If anyone finds an error, please let me know. You find my e-mail address on the first page.
http://www.astro-loltronic.de/FFmpeg_Book.pdf Some things in sea of myriad others: Why you claim that FFmpeg does not have SER format support while in fact it have support for it?
You are right, it is supported now. But my first test to convert a 16-bit grayscale SER file to MP4 failed. I will provide a short sample file later. At the moment I only have SER files that are too big for uploading.
Why you claim that datascope does not support >8 bit formats where in fact it does support it? Why you claim that drawbox does not support RGB formats while in fact it does support them?
Perhaps these features weren't supported at the time when I wrote that, but they are supported now. All three issues are already corrected in the book and an updated version is online. Michael
Am 03.11.2022 um 10:10 schrieb Paul B Mahol:
Your book is full or factual errors,
Notice to anyone: do not use this thing for anything serious. Thank you for so much promotion for my book. Sure, it's likely that in more than 900 pages there are a few errors. We all make errors. If anyone finds an error, please let me know. You find my e-mail address on the first page.
I did not write what you quoted above. You did edit the link. It's very bad behaviour to edit what someone else has written, and make it look as if I wrote it. Michael
Am 03.11.2022 um 10:10 schrieb Paul B Mahol:
Some things in sea of myriad others: [...] Why you claim that datascope does not support >8 bit formats where in fact it does support it?
In some cases datascope works with 16-bit data, but not in all cases. Here is an example where it doesn't work: ffmpeg -f lavfi -i color=black:s=32x32,format=gray10le -lavfi geq=lum='X+32*Y',format=gray10le -frames 1 -y test.png ffmpeg -i test.png -vf format=rgb48,showinfo,datascope=s=1280x1152:mode=color2:format=hex -y out.png Michael
On 11/19/22, Michael Koch <astroelectronic@t-online.de> wrote:
Am 03.11.2022 um 10:10 schrieb Paul B Mahol:
Some things in sea of myriad others: [...] Why you claim that datascope does not support >8 bit formats where in fact it does support it?
In some cases datascope works with 16-bit data, but not in all cases. Here is an example where it doesn't work:
ffmpeg -f lavfi -i color=black:s=32x32,format=gray10le -lavfi geq=lum='X+32*Y',format=gray10le -frames 1 -y test.png
ffmpeg -i test.png -vf format=rgb48,showinfo,datascope=s=1280x1152:mode=color2:format=hex -y out.png
Works here just fine, nice trolling.
On 11/19/22, Paul B Mahol <onemda@gmail.com> wrote:
On 11/19/22, Michael Koch <astroelectronic@t-online.de> wrote:
Am 03.11.2022 um 10:10 schrieb Paul B Mahol:
Some things in sea of myriad others: [...] Why you claim that datascope does not support >8 bit formats where in fact it does support it?
In some cases datascope works with 16-bit data, but not in all cases. Here is an example where it doesn't work:
ffmpeg -f lavfi -i color=black:s=32x32,format=gray10le -lavfi geq=lum='X+32*Y',format=gray10le -frames 1 -y test.png
ffmpeg -i test.png -vf format=rgb48,showinfo,datascope=s=1280x1152:mode=color2:format=hex -y out.png
Works here just fine, nice trolling.
And if you want 16bit numbers, this is libavfilter pixel format negotiation bug which does not pick proper high bit depth format nothing to do with datascope at all.
On 2022-11-19 07:34, Michael Koch wrote:
Am 03.11.2022 um 10:10 schrieb Paul B Mahol:
Some things in sea of myriad others: [...] Why you claim that datascope does not support >8 bit formats where in fact it does support it?
In some cases datascope works with 16-bit data, but not in all cases. Here is an example where it doesn't work:
ffmpeg -f lavfi -i color=black:s=32x32,format=gray10le -lavfi geq=lum='X+32*Y',format=gray10le -frames 1 -y test.png
ffmpeg -i test.png -vf format=rgb48,showinfo,datascope=s=1280x1152:mode=color2:format=hex -y out.png
Perhaps clarify what you observe as "doesn't work", and what behaviour you expect? Both those commands run for me without error, and I can view both test.png and out.png without problem. out.png has a cascade of 32 2-digit hex numbers on the left half, and solid black on the right half. The hex numbers run from 00 to 7F, in white on a black to grey gradient background, and from 80 to FF, in black on a grey to white background. Tested on ffmpeg version 4.4.2 as distributed via MacPorts, on macOS 12.6 Monterey. And I don't know anything about how datascope is supposed to work; <http://ffmpeg.org/ffmpeg-all.html#datascope> does not say much about it.
Am 20.11.2022 um 02:39 schrieb list+ffmpeg-user@jdlh.com:
On 2022-11-19 07:34, Michael Koch wrote:
Am 03.11.2022 um 10:10 schrieb Paul B Mahol:
Some things in sea of myriad others: [...] Why you claim that datascope does not support >8 bit formats where in fact it does support it?
In some cases datascope works with 16-bit data, but not in all cases. Here is an example where it doesn't work:
ffmpeg -f lavfi -i color=black:s=32x32,format=gray10le -lavfi geq=lum='X+32*Y',format=gray10le -frames 1 -y test.png
ffmpeg -i test.png -vf format=rgb48,showinfo,datascope=s=1280x1152:mode=color2:format=hex -y out.png
Perhaps clarify what you observe as "doesn't work", and what behaviour you expect?
Both those commands run for me without error, and I can view both test.png and out.png without problem. out.png has a cascade of 32 2-digit hex numbers on the left half, and solid black on the right half. The hex numbers run from 00 to 7F, in white on a black to grey gradient background, and from 80 to FF, in black on a grey to white background.
I would expect 4-digit hex numbers, because the rgb48 pixel format is 16-bit per channel. For example, it works fine if "rgb48" is replaced by "gray16". Michael
On 2022-11-20 00:15, Michael Koch wrote:
Am 20.11.2022 um 02:39 schrieb list+ffmpeg-user@jdlh.com:
On 2022-11-19 07:34, Michael Koch wrote:
Am 03.11.2022 um 10:10 schrieb Paul B Mahol:
Some things in sea of myriad others: [...] Why you claim that datascope does not support >8 bit formats where in fact it does support it?
In some cases datascope works with 16-bit data, but not in all cases. Here is an example where it doesn't work:
ffmpeg -f lavfi -i color=black:s=32x32,format=gray10le -lavfi geq=lum='X+32*Y',format=gray10le -frames 1 -y test.png
ffmpeg -i test.png -vf format=rgb48,showinfo,datascope=s=1280x1152:mode=color2:format=hex -y out.png
Perhaps clarify what you observe as "doesn't work", and what behaviour you expect?
Both those commands run for me without error, and I can view both test.png and out.png without problem. out.png has a cascade of 32 2-digit hex numbers on the left half, and solid black on the right half. The hex numbers run from 00 to 7F, in white on a black to grey gradient background, and from 80 to FF, in black on a grey to white background.
I would expect 4-digit hex numbers, because the rgb48 pixel format is 16-bit per channel. For example, it works fine if "rgb48" is replaced by "gray16".
The phrase "works fine" appeals to some notion of what is "correct" behaviour. It seems that you have your own idea of "correct" behaviour for this filter. But it seems more helpful for communication on this list to use FFmpeg's idea of "correct" behaviour. And the best source we have for FFmpeg's idea of "correct" is the filter documentation <https://ffmpeg.org/ffmpeg-all.html#datascope>: "Video data analysis filter. This filter shows hexadecimal pixel values of part of video." I think the description in the documentation is incomplete and unclear. I wish FFmpeg had a better description. But the actual behaviour does not conflict with this incomplete description. The description does not promise that the datascope filter shows the full-precision, untruncated pixel values. It might be (I did not check) that the 8-bit values which datascope displays for an rgb48 input image are the correct upper 8 bits of 16-bit pixel values. So, you said, "In some cases datascope works with 16-bit data, but not in all cases." If you had instead said, "In some cases datascope gives useful results with 16-bit data, but not in all cases", then I would be completely with you. It is clear that truncated 8 bit values for an rbg48 input are not as helpful as full-precision, untruncated 16-bit pixel values. But the sad reality is the FFmpeg only does not always document its intended behaviour clearly, and does not seem to have a goal of always providing the most helpful behaviour. The culture here understands "doesn't support" or "doesn't work" to mean, "ffmpeg terminates prematurely with an error" or "ffmpeg fails to generate output". If you use those phrases when your objection is actually, "runs to completion and generates output, but output is not as helpful as it could be", then your message gets diluted by misunderstanding. Best regards, —Jim DeLaHunt
Am 22.11.2022 um 02:15 schrieb Jim DeLaHunt:
On 2022-11-20 00:15, Michael Koch wrote:
Am 20.11.2022 um 02:39 schrieb list+ffmpeg-user@jdlh.com:
On 2022-11-19 07:34, Michael Koch wrote:
Am 03.11.2022 um 10:10 schrieb Paul B Mahol:
Some things in sea of myriad others: [...] Why you claim that datascope does not support >8 bit formats where in fact it does support it?
In some cases datascope works with 16-bit data, but not in all cases. Here is an example where it doesn't work:
ffmpeg -f lavfi -i color=black:s=32x32,format=gray10le -lavfi geq=lum='X+32*Y',format=gray10le -frames 1 -y test.png
ffmpeg -i test.png -vf format=rgb48,showinfo,datascope=s=1280x1152:mode=color2:format=hex -y out.png
Perhaps clarify what you observe as "doesn't work", and what behaviour you expect?
Both those commands run for me without error, and I can view both test.png and out.png without problem. out.png has a cascade of 32 2-digit hex numbers on the left half, and solid black on the right half. The hex numbers run from 00 to 7F, in white on a black to grey gradient background, and from 80 to FF, in black on a grey to white background.
I would expect 4-digit hex numbers, because the rgb48 pixel format is 16-bit per channel. For example, it works fine if "rgb48" is replaced by "gray16".
The phrase "works fine" appeals to some notion of what is "correct" behaviour. It seems that you have your own idea of "correct" behaviour for this filter. But it seems more helpful for communication on this list to use FFmpeg's idea of "correct" behaviour. And the best source we have for FFmpeg's idea of "correct" is the filter documentation <https://ffmpeg.org/ffmpeg-all.html#datascope>: "Video data analysis filter. This filter shows hexadecimal pixel values of part of video."
I think the description in the documentation is incomplete and unclear. I wish FFmpeg had a better description. But the actual behaviour does not conflict with this incomplete description. The description does not promise that the datascope filter shows the full-precision, untruncated pixel values. It might be (I did not check) that the 8-bit values which datascope displays for an rgb48 input image are the correct upper 8 bits of 16-bit pixel values.
So, you said, "In some cases datascope works with 16-bit data, but not in all cases." If you had instead said, "In some cases datascope gives useful results with 16-bit data, but not in all cases", then I would be completely with you. It is clear that truncated 8 bit values for an rbg48 input are not as helpful as full-precision, untruncated 16-bit pixel values.
But the sad reality is the FFmpeg only does not always document its intended behaviour clearly, and does not seem to have a goal of always providing the most helpful behaviour. The culture here understands "doesn't support" or "doesn't work" to mean, "ffmpeg terminates prematurely with an error" or "ffmpeg fails to generate output". If you use those phrases when your objection is actually, "runs to completion and generates output, but output is not as helpful as it could be", then your message gets diluted by misunderstanding.
Sorry for describing the issue not clear enough. In ticket #10057 it should be clear. Michael
participants (4)
-
Jim DeLaHunt -
list+ffmpeg-user＠jdlh.com
-
Michael Koch -
Paul B Mahol