[FFmpeg-devel] [PATCH v6 1/4] doc: Explain what "context" means
Andrew Sayers
ffmpeg-devel at pileofstuff.org
Tue Jun 4 17:47:21 EEST 2024
Derived from explanations kindly provided by Stefano Sabatini and others:
https://ffmpeg.org/pipermail/ffmpeg-devel/2024-April/325903.html
---
doc/context.md | 430 +++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 430 insertions(+)
create mode 100644 doc/context.md
diff --git a/doc/context.md b/doc/context.md
new file mode 100644
index 0000000000..bd8cb58696
--- /dev/null
+++ b/doc/context.md
@@ -0,0 +1,430 @@
+ at page Context Introduction to contexts
+
+ at tableofcontents
+
+FFmpeg uses the term “context” to refer to an idiom
+you have probably used before:
+
+```c
+// C structs often share context between functions:
+
+FILE *my_file; // my_file stores information about a filehandle
+
+printf(my_file, "hello "); // my_file provides context to this function,
+printf(my_file, "world!"); // and also to this function
+```
+
+```python
+# Python classes provide context for the methods they contain:
+
+class MyClass:
+ def print(self,message):
+ if self.prev_message != message:
+ self.prev_message = message
+ print(message)
+```
+
+<!-- marked "c" because Doxygen doesn't support JS highlighting: -->
+```c
+// Many JavaScript callbacks accept an optional context argument:
+
+const my_object = {};
+
+my_array.forEach(function_1, my_object);
+my_array.forEach(function_2, my_object);
+```
+
+Be careful comparing FFmpeg contexts to things you're already familiar with -
+FFmpeg may sometimes happen to reuse words you recognise, but mean something
+completely different. For example, the AVClass struct has nothing to do with
+[object-oriented classes](https://en.wikipedia.org/wiki/Class_(computer_programming)).
+
+If you've used contexts in other C projects, you may want to read
+ at ref Context_comparison before the rest of the document.
+
+ at section Context_general “Context” as a general concept
+
+ at par
+A context is any data structure used by several functions
+(or several instances of the same function) that all operate on the same entity.
+
+In the broadest sense, “context” is just a way to think about code.
+You can even use it to think about code written by people who have never
+heard the term, or who would disagree with you about what it means.
+Consider the following snippet:
+
+```c
+struct DualWriter {
+ int fd1, fd2;
+};
+
+ssize_t write_to_two_files(
+ struct DualWriter *my_writer,
+ uint8_t *buf,
+ int buf_size
+) {
+
+ ssize_t bytes_written_1 = write(my_writer->fd1, buf, buf_size);
+ ssize_t bytes_written_2 = write(my_writer->fd2, buf, buf_size);
+
+ if ( bytes_written_1 != bytes_written_2 ) {
+ // ... handle this edge case ...
+ }
+
+ return bytes_written_1;
+
+}
+
+int main() {
+
+ struct DualWriter my_writer;
+ my_writer.fd1 = open("file1", 0644, "wb");
+ my_writer.fd2 = open("file2", 0644, "wb");
+
+ write_to_two_files(&my_writer, "hello ", sizeof("hello "));
+ write_to_two_files(&my_writer, "world!", sizeof("world!"));
+
+ close( my_writer.fd1 );
+ close( my_writer.fd2 );
+
+}
+```
+
+The term “context” doesn't appear anywhere in the snippet. But `DualWriter`
+is passed to several instances of `write_to_two_files()` that operate on
+the same entity, so it fits the definition of a context.
+
+When reading code that isn't explicitly described in terms of contexts,
+remember that your interpretation may differ from other people's.
+For example, FFmpeg's avio_alloc_context() accepts a set of callback functions
+and an `opaque` argument - even though this function guarantees to *return*
+a context, it does not require `opaque` to *provide* context for the callback
+functions. So you could choose to pass a struct like `DualWriter` as the
+`opaque` argument, or you could pass callbacks that use `stdin` and `stdout`
+and just pass a `NULL` argument for `opaque`.
+
+When reading code that *is* explicitly described in terms of contexts,
+remember that the term's meaning is guaranteed by *the project's community*,
+not *the language it's written in*. That means guarantees may be more flexible
+and change more over time. For example, programming languages that use
+[encapsulation](https://en.wikipedia.org/wiki/Encapsulation_(computer_programming))
+will simply refuse to compile code that violates its rules about access,
+while communities can put up with special cases if they improve code quality.
+
+The next section will discuss what specific conventions FFmpeg developers mean
+when they describe parts of their code as using “contexts”.
+
+ at section Context_ffmpeg FFmpeg contexts
+
+This section discusses specific context-related conventions used in FFmpeg.
+Some of these are used in other projects, others are unique to this project.
+
+ at subsection Context_indicating Indicating context: “Context”, “ctx” etc.
+
+```c
+// Context struct names usually end with `Context`:
+struct AVSomeContext {
+ ...
+};
+
+// Functions are usually named after their context,
+// context parameters usually come first and are often called `ctx`:
+void av_some_function(AVSomeContext *ctx, ...);
+```
+
+FFmpeg struct names usually signal whether they are contexts (e.g. AVBSFContext
+or AVCodecContext). Exceptions to this rule include AVMD5, which is only
+identified as a context by @ref libavutil/md5.c "the functions that call it".
+
+Function names usually signal the context they're associated with (e.g.
+av_md5_alloc() or avcodec_alloc_context3()). Exceptions to this rule include
+ at ref avformat.h "AVFormatContext's functions", many of which begin with
+just `av_`.
+
+Functions usually signal their context parameter by putting it first and
+naming it some variant of `ctx`. Exceptions include av_bsf_alloc(), which puts
+its context argument second to emphasise it's an out variable.
+
+Some functions fit awkwardly within FFmpeg's context idiom, so they send mixed
+signals. For example, av_ambient_viewing_environment_create_side_data() creates
+an AVAmbientViewingEnvironment context, then adds it to the side-data of an
+AVFrame context. So its name hints at one context, its parameter hints at
+another, and its documentation is silent on the issue. You might prefer to
+think of such functions as not having a context, or as “receiving” one context
+and “producing” another.
+
+ at subsection Context_data_hiding Data hiding: private contexts
+
+```c
+// Context structs often hide private context:
+struct AVSomeContext {
+ void *priv_data; // sometimes just called "internal"
+};
+```
+
+Contexts present a public interface, so changing a context's members forces
+everyone that uses the library to at least recompile their program,
+if not rewrite it to remain compatible. Many contexts reduce this problem
+by including a private context with a type that is not exposed in the public
+interface. Hiding information this way ensures it can be modified without
+affecting downstream software.
+
+Private contexts often store variables users aren't supposed to see
+(similar to an [OOP](https://en.wikipedia.org/wiki/Object-oriented_programming)
+private block), but can be used for more than just access control. They can
+also store information shared between some but not all instances of a context
+(e.g. codec-specific functionality), and @ref Context_avoptions
+"AVOptions-enabled structs" can provide user configuration options through
+the @ref avoptions "AVOptions API".
+
+ at subsection Context_lifetime Manage lifetime: creation, use, destruction
+
+```c
+void my_function(...) {
+
+ // Context structs are allocated then initialized with associated functions:
+
+ AVSomeContext *ctx = av_some_context_alloc(...);
+
+ // ... configure ctx ...
+
+ av_some_context_init(ctx, ...);
+
+ // ... use ctx ...
+
+ // Context structs are closed then freed with associated functions:
+
+ av_some_context_close(ctx);
+ av_some_context_free(ctx);
+
+}
+```
+
+FFmpeg contexts go through the following stages of life:
+
+1. allocation (often a function that ends with `_alloc`)
+ * a range of memory is allocated for use by the structure
+ * memory is allocated on boundaries that improve caching
+ * memory is reset to zeroes, some internal structures may be initialized
+2. configuration (implemented by setting values directly on the context)
+ * no function for this - calling code populates the structure directly
+ * memory is populated with useful values
+ * simple contexts can skip this stage
+3. initialization (often a function that ends with `_init`)
+ * setup actions are performed based on the configuration (e.g. opening files)
+5. normal usage
+ * most functions are called in this stage
+ * documentation implies some members are now read-only (or not used at all)
+ * some contexts allow re-initialization
+6. closing (often a function that ends with `_close()`)
+ * teardown actions are performed (e.g. closing files)
+7. deallocation (often a function that ends with `_free()`)
+ * memory is returned to the pool of available memory
+
+This can mislead object-oriented programmers, who expect something more like:
+
+1. allocation (usually a `new` keyword)
+ * a range of memory is allocated for use by the structure
+ * memory *may* be reset (e.g. for security reasons)
+2. initialization (usually a constructor)
+ * memory is populated with useful values
+ * related setup actions are performed based on arguments (e.g. opening files)
+3. normal usage
+ * most functions are called in this stage
+ * compiler enforces that some members are read-only (or private)
+ * no going back to the previous stage
+4. finalization (usually a destructor)
+ * teardown actions are performed (e.g. closing files)
+5. deallocation (usually a `delete` keyword)
+ * memory is returned to the pool of available memory
+
+The remainder of this section discusses FFmpeg's differences from OOP, to help
+object-oriented programmers avoid misconceptions. You can safely skip this
+section if you aren't familiar with the OOP lifetime described above.
+
+FFmpeg's allocation stage is broadly similar to the OOP stage of the same name.
+Both set aside some memory for use by a new entity, but FFmpeg's stage can also
+do some higher-level operations. For example, @ref Context_avoptions
+"AVOptions-enabled structs" set their AVClass member during allocation.
+
+FFmpeg's configuration stage involves setting any variables you want before
+you start using the context. Complicated FFmpeg structures like AVCodecContext
+tend to have many members you *could* set, but in practice most programs set
+few if any of them. The freeform configuration stage works better than bundling
+these into the initialization stage, which would lead to functions with
+impractically many parameters, and would mean each new option was an
+incompatible change to the API. One way to understand the problem is to read
+ at ref Context_avoptions "the AVOptions section below" and think how a constructor
+would handle those options.
+
+FFmpeg's initialization stage involves calling a function that sets the context
+up based on your configuration.
+
+FFmpeg's first three stages do the same job as OOP's first two stages.
+This can mislead object-oriented developers, who expect to do less work in the
+allocation stage, and more work in the initialization stage. To simplify this,
+most FFmpeg contexts provide a combined allocator and initializer function.
+For historical reasons, suffixes like `_alloc`, `_init`, `_alloc_context` and
+even `_open` can indicate the function does any combination of allocation and
+initialization.
+
+FFmpeg's "closing" stage is broadly similar to OOP's "finalization" stage,
+but some contexts allow re-initialization after finalization. For example,
+SwrContext lets you call swr_close() then swr_init() to reuse a context.
+Be aware that some FFmpeg functions happen to use the word "finalize" in a way
+that has nothing to do with the OOP stage (e.g. av_bsf_list_finalize()).
+
+FFmpeg's "deallocation" stage is broadly similar to OOP, but can perform some
+higher-level functions (similar to the allocation stage).
+
+Closing functions usually end with "_close", while deallocation
+functions usually end with "_free". Very few contexts need the flexibility of
+separate "closing" and "deallocation" stages, so many "_free" functions
+implicitly close the context first.
+
+ at subsection Context_avoptions Configuration options: AVOptions-enabled structs
+
+The @ref avoptions "AVOptions API" is a framework to configure user-facing
+options, e.g. on the command-line or in GUI configuration forms.
+
+To understand FFmpeg's configuration requirements, run `ffmpeg -h full` on the
+command-line, then ask yourself how you would implement all those options
+with the C standard [`getopt` function](https://en.wikipedia.org/wiki/Getopt).
+You can also ask the same question for other approaches - for example, how would
+you maintain a GUI with 15,000+ configuration options?
+
+Most solutions assume you can just put all options in a single code block,
+which is unworkable at FFmpeg's scale. Instead, we split configuration
+across many *AVOptions-enabled structs*, which use the @ref avoptions
+"AVOptions API" to inspect and configure options, including in private contexts.
+
+AVOptions-accessible members of a context should be accessed through the
+ at ref avoptions "AVOptions API" whenever possible, even if they're not hidden
+in a private context. That ensures values are validated as they're set, and
+means you won't have to do as much work if a future version of FFmpeg changes
+the allowed values.
+
+Although not strictly required, it is best to only modify options during
+the configuration stage. Initialized structs may be accessed by internal
+FFmpeg threads, and modifying them can cause weird intermittent bugs.
+
+ at subsection Context_logging Logging: AVClass context structures
+
+FFmpeg's @ref lavu_log "logging facility" needs to be simple to use,
+but flexible enough to let people debug problems. And much like options,
+it needs to work the same across a wide variety of unrelated structs.
+
+FFmpeg structs that support the logging framework are called *@ref AVClass
+context structures*. The name @ref AVClass was chosen early in FFmpeg's
+development, but in practice it only came to store information about
+logging, and about options.
+
+ at section Context_further Further information about contexts
+
+So far, this document has provided a theoretical guide to FFmpeg contexts.
+This final section provides some alternative approaches to the topic,
+which may help round out your understanding.
+
+ at subsection Context_example Learning by example: context for a codec
+
+It can help to learn contexts by doing a deep dive into a specific struct.
+This section will discuss AVCodecContext - an AVOptions-enabled struct
+that contains information about encoding or decoding one stream of data
+(e.g. the video in a movie).
+
+The name "AVCodecContext" tells us this is a context. Many of
+ at ref libavcodec/avcodec.h "its functions" start with an `avctx` parameter,
+indicating this parameter provides context for that function.
+
+AVCodecContext::internal contains the private context. For example,
+codec-specific information might be stored here.
+
+AVCodecContext is allocated with avcodec_alloc_context3(), initialized with
+avcodec_open2(), and freed with avcodec_free_context(). Most of its members
+are configured with the @ref avoptions "AVOptions API", but for example you
+can set AVCodecContext::draw_horiz_band() if your program happens to need it.
+
+AVCodecContext provides an abstract interface to many different *codecs*.
+Options supported by many codecs (e.g. "bitrate") are kept in AVCodecContext
+and exposed with AVOptions. Options that are specific to one codec are
+stored in the private context, and also exposed with AVOptions.
+
+AVCodecContext::av_class contains logging metadata to ensure all codec-related
+error messages look the same, plus implementation details about options.
+
+To support a specific codec, AVCodecContext's private context is set to
+an encoder-specific data type. For example, the video codec
+[H.264](https://en.wikipedia.org/wiki/Advanced_Video_Coding) is supported via
+[the x264 library](https://www.videolan.org/developers/x264.html), and
+implemented in X264Context. Although included in the documentation, X264Context
+is not part of the public API. That means FFmpeg's @ref ffmpeg_versioning
+"strict rules about changing public structs" aren't as important here, so a
+version of FFmpeg could modify X264Context or replace it with another type
+altogether. An adverse legal ruling or security problem could even force us to
+switch to a completely different library without a major version bump.
+
+The design of AVCodecContext provides several important guarantees:
+
+- lets you use the same interface for any codec
+- supports common encoder options like "bitrate" without duplicating code
+- supports encoder-specific options like "profile" without bulking out the public interface
+- exposes both types of options to users, with help text and detection of missing options
+- provides uniform logging output
+- hides implementation details (e.g. its encoding buffer)
+
+ at subsection Context_comparison Learning by comparison: FFmpeg vs. Curl contexts
+
+It can help to learn contexts by comparing how different projects tackle
+similar problems. This section will compare @ref AVMD5 "FFmpeg's MD5 context"
+with [curl 8.8.0's equivalent](https://github.com/curl/curl/blob/curl-8_8_0/lib/md5.c#L48).
+
+The [MD5 algorithm](https://en.wikipedia.org/wiki/MD5) produces
+a fixed-length digest from arbitrary-length data. It does this by calculating
+the digest for a prefix of the data, then loading the next part and adding it
+to the previous digest, and so on.
+
+```c
+// FFmpeg's MD5 context looks like this:
+typedef struct AVMD5 {
+ uint64_t len;
+ uint8_t block[64];
+ uint32_t ABCD[4];
+} AVMD5;
+
+// Curl 8.8.0's MD5 context looks like this:
+struct MD5_context {
+ const struct MD5_params *md5_hash; /* Hash function definition */
+ void *md5_hashctx; /* Hash function context */
+};
+```
+
+Curl's struct name ends with `_context`, guaranteeing contexts are the correct
+interpretation. FFmpeg's struct does not explicitly say it's a context, but
+ at ref libavutil/md5.c "its functions do" so we can reasonably assume
+it's the intended interpretation.
+
+Curl's struct uses `void *md5_hashctx` to avoid guaranteeing
+implementation details in the public interface, whereas FFmpeg makes
+everything accessible. This disagreement about data hiding is a good example
+of how contexts can be used differently. Hiding the data means changing the
+layout in a future version of curl won't break downstream programs that used
+that data. But the MD5 algorithm has been stable for 30 years, and making the
+data public makes it easier for people to follow a bug in their own code.
+
+Curl's struct is declared as `struct <type> { ... }`, whereas FFmpeg uses
+`typedef struct <type> { ... } <type>`. These conventions are used with both
+context and non-context structs, so don't say anything about contexts as such.
+Specifically, FFmpeg's convention is a workaround for an issue with C grammar:
+
+```c
+void my_function(...) {
+ int my_var; // good
+ MD5_context my_curl_ctx; // error: C needs you to explicitly say "struct"
+ struct MD5_context my_curl_ctx; // good: added "struct"
+ AVMD5 my_ffmpeg_ctx; // good: typedef's avoid the need for "struct"
+}
+```
+
+Both MD5 implementations are long-tested, widely-used examples of contexts
+in the real world. They show how contexts can solve the same problem
+in different ways.
--
2.45.1
More information about the ffmpeg-devel
mailing list