[FFmpeg-devel] [RFC] FFmpeg Execution Graph Visualization

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> Stefano Sabatini
> Sent: Freitag, 21. März 2025 19:27
> To: FFmpeg development discussions and patches <ffmpeg-devel at ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [RFC] FFmpeg Execution Graph Visualization
> 
> On date Tuesday 2025-03-18 02:32:13 +0000, Soft Works wrote:
> > Hello everybody,
> >
> > working on the subject of writing out filtergraph information
> > obviously implies the goal of being able to visualize that data in
> > some way. While I do have something for long, it's tailored for
> > specific workflows and is hardly useful for most.
> >
> > Anyway, it shouldn't be required to use once another software for
> > visualize the output. This decimates the usability for any such
> > feature by far.  Few days ago, I was thinking about ways for making
> > this feature really useful for everybody without needing to jump
> > through any extra hoops. We have that range of writers (now "text
> > formatters"), and so I wondered whether there isn't some text format
> > that is meant to directly represent a graph for visualization
> [...]
> > Examples
> >
> > I've created a Gist with some examples of the output here:
> >
> > https://gist.github.com/softworkz/a196b2d0e9e2df49f766abd92f508551
> >
> > (also includes a zip with html file examples)
> > Questions
> >
> > I'm curious what you think about it!
> >
> > - What's good, what's bad, what should be changed/improved?
> >
> > - What about the displayed information, should something be added?
> >
> > - I'm folding out the buffer source/sink filters to simplify the view,
> is that ok?
> >   Should the TRIM filters be excluded as well?
> >
> > - Since it's no longer just about filter graphs - what do you think
> about the term
> >   "Ffmpeg Execution Graph"?
> >   (other ideas welcome)
> >
> > - Does anybody have some complex command lines for me to test?
> >   (no need to include media files, I can try to replicate
> >   something similar)
> >
> 
> Thanks for working on this, from the examples it looks pretty amazing.
> 
> What it's not clear to me is how this builds up on top of text
> formatters, since they are meant to render a tree structure in a
> generic way. From this you can have a description of a graph, but then
> you need specialized ad-hoc logic to convert it to a graph format.
> 
> What am I missing?


Hehe, that's been a bit tricky indeed, but more in terms of figuring out that (and how) it can fit into the writer/textformatter patterns, the result is rather simple and it requires only some moderate extensions to the writer/formatter APIs:


It is using a Mermaid Flowchart graph with only 3 kinds of elements:

- Shapes
  i.e. "boxes", can't be nested
- Subgraphs
  Container for shapes, can be nested
- Links
  connection from one shape to another


A simple graph with 2 shapes in a subgraph looks like this:

...
flowchart TD
 subgraph s1["Subgraph 1"]
        A("Shape A") --> B("Shape B")
  end
...

There's a link from A to B - but: it can also be written in a different way:

...
flowchart TD
 subgraph s1["Subgraph 1"]
        A("Shape A")
        B("Shape B")
  end

  A --> B
...

This does the same thing. The link definitions can be at a different place in the document - which is one of the key points to make it work.

Then there are three new section flags to denote subgraph, shapes and links and two fields in the section struct (source_id_key, dest_id_key). For link-producing sections they indicate the names of the keys to use for building a link. A section can also produce both - shapes and links when the flags are set accordingly.
I also use another (existing) mechanism, which is the data object that can be provided on section start, only that it's a (publicly) defined "SectionContext" struct, which is a somewhat alternative direction. I kept it for the moment as suggestion because I didn't know which ideas you have for the API to evolve, so that's just something for discussion.

Best,
sw