[FFmpeg-devel] [PATCH] libavformat: add mbedTLS based TLS
Dave Gregory
dave.gregory at manything.com
Mon May 14 20:04:00 EEST 2018
Hi all,
I am an engineer working at Manything (https://manything.com). We
develop software that allows users to view their security cameras over
the Internet. Thomas has been working with us to develop this mbedTLS
integration.
Thanks for your feedback on his patch; we have run some tests that I
hope will answer some of your questions. We've not been able to
produce all the detail you've requested, but hopefully what we have is
a compelling case. With some advice, I hope we could produce any other
evidence you need to support acceptance of the patch. This is our
first contribution back to FFmpeg, so we welcome guidance on how to do
the right things in the right way.
First, a bit of context on our motivation. We write software that runs on
IP cameras to connect them to our cloud service. We started off by
using GnuTLS to secure our connections but there's very limited disk
space on the cameras and the relatively large binary footprint caused
some problems. Looking around for an alternative library we discovered
mbedTLS, a project with a good open-source heritage, suitable license
terms and a major commercial backer (ARM Holdings). We were easily
able to make use of its integration with cURL but it wasn't yet
supported by the FFmpeg project, so we found Thomas via the project's
consulting page and embarked on the integration. mbedTLS is an
excellent fit for our needs; I hope this is an indicator of its potential value
to other FFmpeg users.
Below you will find our method and results. We look forward to your
feedback.
Many thanks,
Dave
--
FFmpeg TLS libraries: disk and memory footprint comparison report
Goal: Compare the memory usage and disk footprint of the new mbedTLS
integration against the three existing platform-agnostic TLS libraries
supported by FFmpeg.
Approach summary (full detail in appendix A):
- Build FFmpeg against four TLS libraries, measure output library sizes.
- schannel (Windows) and securetransport (macOS) are excluded from
the analysis because it would be very difficult to compare results
fairly across different operating systems (and impossible on a Linux
test rig).
- Stream well-known free video file (Big Buck Bunny) to remote rtsps
server in realtime,
collect memory statistics with valgrind/massif.
- Interrogate massif outputs to assess relative memory consumption of
each library.
Note: The project's actual devices are mostly ARM-based but we have
found valgrind hard to run on ARM. The stats below have therefore been
created on an x86_64 box to provide memory measurements easily and
repeatably.
Results summary (full detail in appendix B):
- GnuTLS 3.4.17
- Peak ssl/tls/crypto-related memory allocation: 4.36MB
- TLS library file size on disk, including dependencies: 5.5MB
- LibreSSL 2.7.2 (libtls)
- Results
- Peak ssl/tls/crypto-related memory allocation: 115KB
- TLS library file size on disk, including dependencies: 3.27MB
- mbedTLS 2.8.0
- Results
- Peak ssl/tls/crypto-related memory allocation: 40KB
- TLS library file size on disk, including dependencies: 828KB
- OpenSSL 1.0.2n
- Results
- Peak ssl/tls/crypto-related memory allocation: 151KB
- TLS library file size on disk, including dependencies: 2.7MB
All supporting data can be downloaded from
https://assetcdn.manything.com/downloads/data/ffmpeg_tls_massif_data.tar.xz.
Conclusion: For the task of outbound interleaved RTSPS streaming,
mbedTLS uses about 35% of the memory of LibreSSL/libtls (its nearest
competitor) with only 25% of the disk footprint.
---
APPENDIX A: Test detail
Test system:
- Linux vagrant-ubuntu-trusty-64 3.13.0-87-generic #133-Ubuntu (VM)
- Running in virtualbox on MacBook Pro (Retina, 13-inch, Early 2015)
- Ubuntu EGLIBC 2.19-0ubuntu6.11
Test process:
- Compile dependencies, with configuration flags listed, and install
into custom prefix (referred to as ${DEPS_OUTDIR})
- Unless otherwise specified, steps are:
configure --prefix=${DEPS_OUTDIR} (any extra flags specified above)
make
make install
- Measure size of relevant libs
e.g. ls -lh ${DEPS_OUTDIR}/lib/ | grep
'ssl\|crypto\|tls\|nettle\|hogweed\|gmp\|gpg\|gcrypt'
- Compile ffmpeg against dependencies, with additional configuration
flags listed above.
configure --prefix=${DEPS_OUTDIR} --disable-stripping
--disable-debug --enable-shared --disable-everything --enable-version3
--enable-parser=h264,hevc,aac
--enable-decoder=h264,hevc,aac,pcm_alaw,pcm_mulaw
--enable-protocol=tcp,udp,file,tls,tls_gnutls --enable-demuxer=rtsp
--enable-muxer=rtsp,mp4,mpegts,pcm_alaw,pcm_mulaw --disable-ffplay
--disable-ffprobe --disable-doc --disable-avdevice
- Stream known free video file to remote rtsps server in realtime,
wrapped in valgrind/massif
valgrind --tool=massif --threshold=0.1 ${DEPS_OUTDIR}/bin/ffmpeg
-re -i BigBuckBunny_320x180.mp4 -c:v copy -c:a copy -f rtsp
-rtsp_transport tcp rtsps://USER:PASS@REMOTEHOST:443/STREAMNAME
- Redact sensitive information (remote creds/url) from massif.out
- Rename massif.out.NNNNN and process through ms_print:
mv massif.out.NNNNN massif.out.LIBNAME
ms_print --threshold=0.1 massif.out.LIBNAME > ms_print.massif.out.LIBNAME
- Generate visualizations for each massif.out with massif-visualizer
- Perform detailed memory analysis on each snapshot:
- Filter the heap graph (ms_print.out) to include only top-level items.:
grep -e '^->' -e '^---' -e '^\s\+[0-9n]\+ '
ms_print.massif.out.LIBNAME >
ms_print.massif.out.LIBNAME.toplevelfilter
- For each snapshot, sum values for crypto/tls libraries to get
total crypto/tls heap usage for that snap.
- Identify steady-state value from a snap near the middle of the run.
- Identify peak value, if different from steady-state.
---
APPENDIX B: Full results with library configurations
- GnuTLS 3.4.17
- TLS library compile-time configuration
configure --prefix=${DEPS_OUTDIR}
--with-libz-prefix=${DEPS_OUTDIR} --enable-shared --without-p11-kit
--with-included-libtasn1 --disable-doc
make
make install
- Dependencies with configure flags:
- zlib 1.2.11 (configure --prefix=${DEPS_OUTDIR})
- gmp 6.1.2 (configure --prefix=${DEPS_OUTDIR})
- nettle 3.1 (configure --prefix=${DEPS_OUTDIR} --enable-shared
--disable-assembler --disable-openssl)
- gpg-error 1.26 (configure --prefix=${DEPS_OUTDIR})
- gcrypt 1.7.5 (configure --prefix=${DEPS_OUTDIR}
--with-libgpg-error-prefix=${DEPS_OUTDIR})
- Additional FFmpeg configure flags (main set below):
--enable-gnutls --enable-protocol=tcp,udp,file,tls,tls_gnutls
- Results
- Key tls/crypto-related memory allocation from detailed snapshots:
Peak: 4.36MB from snap #5
Steady-state: 4.1MB from snap #36
- Values for other snapshots, and detail for peak:
#4: 3.6MB
#5: 4.36MB
5 117,151,532 6,229,360 5,747,248
482,112 0
->61.00% (3,800,000B) 0x6C75D95: _asn1_add_single_node
(in /home/build/2.3.0-rtsps.gnutls.2/lib/libgnutls.so.30.7.0)
->16.81% (1,047,104B) 0x5DEDB16: av_realloc_f (in
/home/build/2.3.0-rtsps.gnutls.2/lib/libavutil.so.56.15.100)
->04.40% (274,341B) 0x6BBEC3D: _gnutls_fread_file (in
/home/build/2.3.0-rtsps.gnutls.2/lib/libgnutls.so.30.7.0)
->03.19% (198,992B) 0x6B8F3CD: _gnutls_base64_decode (in
/home/build/2.3.0-rtsps.gnutls.2/lib/libgnutls.so.30.7.0)
->02.30% (143,357B) 0x6C74517: _asn1_set_value (in
/home/build/2.3.0-rtsps.gnutls.2/lib/libgnutls.so.30.7.0)
->01.62% (100,729B) 0x5DEDA0B: av_malloc (in
/home/build/2.3.0-rtsps.gnutls.2/lib/libavutil.so.56.15.100)
->01.44% (89,984B) 0x6C740BD: _asn1_add_static_node (in
/home/build/2.3.0-rtsps.gnutls.2/lib/libgnutls.so.30.7.0)
->00.47% (29,479B) 0x6C745A8: _asn1_set_value_lv (in
/home/build/2.3.0-rtsps.gnutls.2/lib/libgnutls.so.30.7.0)
->00.37% (22,953B) in 43 places, all below massif's
threshold (00.10%)
->00.31% (19,376B) 0x6BFD92F: gnutls_x509_crt_init (in
/home/build/2.3.0-rtsps.gnutls.2/lib/libgnutls.so.30.7.0)
->00.20% (12,741B) 0x6B9E022: gnutls_realloc_fast (in
/home/build/2.3.0-rtsps.gnutls.2/lib/libgnutls.so.30.7.0)
->00.13% (8,192B) 0x6C033B5: gnutls_x509_crt_list_import2
(in /home/build/2.3.0-rtsps.gnutls.2/lib/libgnutls.so.30.7.0)
#6: 4.1MB
[...]
#36: 4.1MB
36 770,323,285 5,988,232 5,503,656
484,576 0
->63.45% (3,799,240B) 0x6C75D95: _asn1_add_single_node
(in /home/build/2.3.0-rtsps.gnutls.2/lib/libgnutls.so.30.7.0)
->17.49% (1,047,104B) 0x5DEDB16: av_realloc_f (in
/home/build/2.3.0-rtsps.gnutls.2/lib/libavutil.so.56.15.100)
->03.32% (198,992B) 0x6B8F3CD: _gnutls_base64_decode (in
/home/build/2.3.0-rtsps.gnutls.2/lib/libgnutls.so.30.7.0)
->02.39% (143,357B) 0x6C74517: _asn1_set_value (in
/home/build/2.3.0-rtsps.gnutls.2/lib/libgnutls.so.30.7.0)
->02.02% (121,133B) 0x5DEDA0B: av_malloc (in
/home/build/2.3.0-rtsps.gnutls.2/lib/libavutil.so.56.15.100)
->01.50% (89,984B) 0x6C740BD: _asn1_add_static_node (in
/home/build/2.3.0-rtsps.gnutls.2/lib/libgnutls.so.30.7.0)
->00.49% (29,479B) 0x6C745A8: _asn1_set_value_lv (in
/home/build/2.3.0-rtsps.gnutls.2/lib/libgnutls.so.30.7.0)
->00.44% (26,481B) in 67 places, all below massif's
threshold (00.10%)
->00.37% (22,414B) 0x6B9E022: gnutls_realloc_fast (in
/home/build/2.3.0-rtsps.gnutls.2/lib/libgnutls.so.30.7.0)
->00.32% (19,376B) 0x6BFD92F: gnutls_x509_crt_init (in
/home/build/2.3.0-rtsps.gnutls.2/lib/libgnutls.so.30.7.0)
->00.10% (6,096B) 0x6C0A67E: gnutls_x509_trust_list_init
(in /home/build/2.3.0-rtsps.gnutls.2/lib/libgnutls.so.30.7.0)
#55: 4.1MB
- File size on disk, including dependencies: 5.5MB
-rwxr-xr-x 1 build build 1.4M Apr 27 10:36 libgcrypt.so.20.1.5
-rwxr-xr-x 1 build build 675K Apr 27 10:34 libgmp.so.10.3.2
-rwxr-xr-x 1 build build 1.7M Apr 27 10:41 libgnutls.so.30.7.0
-rwxr-xr-x 1 build build 195K Apr 27 10:41 libgnutlsxx.so.28.1.0
-rwxr-xr-x 1 build build 108K Apr 27 10:35 libgpg-error.so.0.21.0
-rw-r--r-- 1 build build 625K Apr 27 10:37 libhogweed.so.4.0
-rw-r--r-- 1 build build 757K Apr 27 10:37 libnettle.so.6.0
- LibreSSL 2.7.2
- TLS library compile-time configuration
configure --prefix=${DEPS_OUTDIR}
make
make install
- Dependencies with configurations (if any):
- zlib 1.2.11 (configure --prefix=${DEPS_OUTDIR})
- Additional FFmpeg configure flags:
--enable-libtls --enable-protocol=tcp,udp,file,tls,tls_openssl
- Results
- Key tls/crypto-related memory allocations from detailed snapshots:
Peak / steady-state: 115KB from snap #40
- Values for other snapshots, and detail for peak:
#2: <1KB
#8: 115KB
[...]
#40: 115KB
40 720,412,615 1,374,584 1,323,570
51,014 0
->76.18% (1,047,104B) 0x5DEDB16: av_realloc_f (in
/home/build/2.3.0-rtsps.libtls.2/lib/libavutil.so.56.15.100)
->09.87% (135,655B) 0x5DEDA0B: av_malloc (in
/home/build/2.3.0-rtsps.libtls.2/lib/libavutil.so.56.15.100)
->03.11% (42,792B) 0x7C72EDF: lh_insert (in
/home/build/2.3.0-rtsps.libtls.2/lib/libcrypto.so.43.0.1)
->01.31% (17,951B) in 105 places, all below massif's
threshold (00.10%)
->01.22% (16,712B) 0x78D43E5: ssl3_setup_read_buffer (in
/home/build/2.3.0-rtsps.libtls.2/lib/libssl.so.45.0.1)
->01.20% (16,560B) 0x78D4533: ssl3_setup_write_buffer (in
/home/build/2.3.0-rtsps.libtls.2/lib/libssl.so.45.0.1)
->01.04% (14,264B) 0x7CE1E17: reallocarray (in
/home/build/2.3.0-rtsps.libtls.2/lib/libcrypto.so.43.0.1)
->00.46% (6,288B) 0x7C7AA22: OBJ_NAME_add (in
/home/build/2.3.0-rtsps.libtls.2/lib/libcrypto.so.43.0.1)
->00.44% (6,010B) 0x6B4E4FD: tls_conninfo_cert_pem (in
/home/build/2.3.0-rtsps.libtls.2/lib/libtls.so.17.0.1)
->00.35% (4,744B) 0x400BAF2: _dl_new_object (dl-object.c:75)
->00.23% (3,219B) 0x7BC1CFE: asn1_enc_save (in
/home/build/2.3.0-rtsps.libtls.2/lib/libcrypto.so.43.0.1)
->00.17% (2,400B) 0x7BA5BBA: ASN1_OBJECT_new (in
/home/build/2.3.0-rtsps.libtls.2/lib/libcrypto.so.43.0.1)
->00.17% (2,388B) 0x7CE1E6F: recallocarray (in
/home/build/2.3.0-rtsps.libtls.2/lib/libcrypto.so.43.0.1)
->00.17% (2,328B) 0x7BBE5CD: asn1_item_ex_combine_new (in
/home/build/2.3.0-rtsps.libtls.2/lib/libcrypto.so.43.0.1)
->00.15% (2,090B) 0x7BA14E3: c2i_ASN1_BIT_STRING (in
/home/build/2.3.0-rtsps.libtls.2/lib/libcrypto.so.43.0.1)
->00.12% (1,656B) 0x7BADAC6: ASN1_STRING_type_new (in
/home/build/2.3.0-rtsps.libtls.2/lib/libcrypto.so.43.0.1)
->00.10% (1,409B) 0x7BAD95F: ASN1_STRING_set (in
/home/build/2.3.0-rtsps.libtls.2/lib/libcrypto.so.43.0.1)
#58: 115KB
#68: 115KB
#78: 115KB
- File size on disk, including dependencies: 3.27MB
-rw-r--r-- 1 build build 2.7M Apr 27 12:31 libcrypto.so.43.0.1
-rw-r--r-- 1 build build 475K Apr 27 12:31 libssl.so.45.0.1
-rw-r--r-- 1 build build 96K Apr 27 12:31 libtls.so.17.0.1
- mbedTLS 2.8.0
- TLS library compile-time configuration
cmake -DUSE_SHARED_MBEDTLS_LIBRARY=On
-DCMAKE_INSTALL_PREFIX=${DEPS_OUTDIR} .
sed -i "s|DESTDIR=/usr/local|DESTDIR=${DEPS_OUTDIR}|g"
${ROOTDIR}/deps/${MANUFACTURER}/${PLATFORM}/work/mbedtls/Makefile
make install ${DEPS_OUTDIR}
- Dependencies with configurations:
- zlib 1.2.11 (configure --prefix=${DEPS_OUTDIR})
- Additional FFmpeg configure flags:
--enable-mbedtls --enable-protocol=tcp,udp,file,tls,tls_mbedtls
- Results
- Key tls/crypto-related memory allocation from detailed snapshots:
Peak / Steady-state: 40KB from snap #32
- Values for other snapshots, and detail for peak:
#2: 40KB
#17: 40KB
#24: 40KB
#32: 40KB
32 4,735,121,659 1,238,712 1,228,242
10,470 0
->84.53% (1,047,104B) 0x5DEEB16: av_realloc_f (in
/home/build/2.3.0-rtsps.1/lib/libavutil.so.56.15.100)
->10.20% (126,329B) 0x5DEEA0B: av_malloc (in
/home/build/2.3.0-rtsps.1/lib/libavutil.so.56.15.100)
->01.35% (16,717B) 0x6B68FE3: mbedtls_ssl_setup (in
/home/build/2.3.0-rtsps.1/lib/libmbedtls.so.2.8.0)
->01.35% (16,717B) 0x6B69012: mbedtls_ssl_setup (in
/home/build/2.3.0-rtsps.1/lib/libmbedtls.so.2.8.0)
->00.76% (9,364B) in 48 places, all below massif's
threshold (00.10%)
->00.38% (4,744B) 0x400BAF2: _dl_new_object (dl-object.c:75)
->00.35% (4,315B) 0x78DB3B1: x509_crt_parse_der_core (in
/home/build/2.3.0-rtsps.1/lib/libmbedx509.so.2.8.0)
->00.13% (1,608B) 0x7AFF1DE: mbedtls_mpi_grow (in
/home/build/2.3.0-rtsps.1/lib/libmbedcrypto.so.2.8.0)
->00.11% (1,344B) 0x78D7430: mbedtls_x509_get_name (in
/home/build/2.3.0-rtsps.1/lib/libmbedx509.so.2.8.0)
#36: 40KB
#45: 40KB
- File size on disk, including dependencies: 828KB
-rwxr-xr-x 1 build build 479K Apr 25 08:27 libmbedcrypto.so.2.8.0
-rwxr-xr-x 1 build build 240K Apr 25 08:27 libmbedtls.so.2.8.0
-rwxr-xr-x 1 build build 109K Apr 25 08:27 libmbedx509.so.2.8.0
- OpenSSL 1.0.2n
- TLS library compile-time configuration
./Configure linux-generic32 shared --prefix=${DEPS_OUTDIR}
--openssldir=${DEPS_OUTDIR}/openssl
make
make install
- Dependencies with configurations:
- zlib 1.2.11 (configure --prefix=${DEPS_OUTDIR})
- Additional FFmpeg configure flags:
--enable-openssl --enable-protocol=tcp,udp,file,tls,tls_openssl
- Results
- Key tls/crypto-related memory allocation from detailed snapshots:
Peak / Steady-state: 151KB from snap #30
- Values for other snapshots, and detail for peak:
#1: 82KB
#8: 151KB
[...]
#30: 151KB
30 3,264,891,835 1,405,992 1,342,597
63,395 0
->74.47% (1,047,104B) 0x5DEDB16: av_realloc_f (in
/home/build/2.3.0-rtsps.2/lib/libavutil.so.56.15.100)
->09.57% (134,567B) 0x6E1B036: CRYPTO_malloc (in
/home/build/2.3.0-rtsps.2/lib/libcrypto.so.1.0.0)
->09.31% (130,917B) 0x5DEDA0B: av_malloc (in
/home/build/2.3.0-rtsps.2/lib/libavutil.so.56.15.100)
->01.39% (19,584B) 0x6E1B127: CRYPTO_realloc (in
/home/build/2.3.0-rtsps.2/lib/libcrypto.so.1.0.0)
->00.40% (5,681B) in 30 places, all below massif's
threshold (00.10%)
->00.34% (4,744B) 0x400BAF2: _dl_new_object (dl-object.c:75)
#51: 151KB
- File size on disk, including dependencies: 2.7MB
-r-xr-xr-x 1 build build 2.2M Apr 25 16:00 libcrypto.so.1.0.0
-r-xr-xr-x 1 build build 506K Apr 25 16:00 libssl.so.1.0.0
More information about the ffmpeg-devel
mailing list