FFmpeg is the layer under almost everything else

What people actually used · Long

Most tools that claim to convert video are calling FFmpeg. It is the piece the ecosystem depends on and the piece nobody markets.

The invisible infrastructure

Open almost any video converter on any platform and there is a good chance the progress bar you are watching is fed by FFmpeg. The application has a logo; FFmpeg has a command line. The application has a purchase button and a five-star rating; FFmpeg has a mailing list and a bug tracker and, somewhere underneath the wrapping of thousands of commercial and open-source products, it is doing the actual work. This is not a criticism of the applications built on top of it. It is a description of how the video toolchain actually assembled itself over two decades: one project became load-bearing infrastructure for an entire industry, and stayed invisible because infrastructure rarely gets a brand.

FFmpeg was started in 2000 by Fabrice Bellard, a French programmer better known to some audiences for writing QEMU and for computing a then-record number of digits of pi. The name is a deliberate compound: FF from fast-forward, and MPEG for the standards body — the Moving Picture Experts Group — whose formats it first targeted. The project was released under the GNU Lesser General Public License, which meant any software could link against it without itself becoming open source, provided the terms were observed. That licensing choice lowered the barrier to adoption and is a significant part of why FFmpeg ended up inside so many commercial products.

The core of what FFmpeg does is decode a stream, optionally transform it, and re-encode it into another format. Stated that baldly it sounds simple; the scope of formats it covers is what makes it extraordinary. Over its lifetime the project has accumulated support for hundreds of codecs and dozens of containers — not by licensing them from their authors but by implementing them, reading specifications, reverse-engineering undocumented formats, and writing the codec logic from scratch. When a new container arrived, FFmpeg was usually among the first tools to support it. When a format was abandoned and documentation disappeared, FFmpeg frequently preserved working decode support that no commercial tool could any longer provide.

What it contains and why that matters

Inside FFmpeg, the relevant libraries are largely independent of the command-line tool itself. libavcodec handles encoding and decoding. libavformat handles the container layer — demuxing a file into streams and muxing streams back into a file. libavfilter provides a pipeline for transforming audio and video between decode and encode. These libraries are what other applications link against. HandBrake, which presents a preset-driven front end to the encode process, uses them. So does VLC. So does Kodi, Plex, and a significant fraction of the professional editing and transcoding tools that do not advertise the fact. The relationship is symbiotic in practice and invisible to users by design.

The codec coverage matters especially in the context of the formats this site traces. FFmpeg carries decode support for DivX ;-) 3.11 — the hacked Microsoft MPEG-4 build that put a feature film on a CD-R — because FFmpeg's developers reverse-engineered the Microsoft MPEG-4 variant it was built on and wrote a dedicated decoder for it. It supports XviD-encoded files through its MPEG-4 Part 2 decoder. It decodes H.264 with its own decoder and encodes it via libx264, the open reference encoder developed separately and integrated as an external library. It does the same for HEVC and, since AV1 became a ratified format, for AV1 through libaom and other external encoders. Each generation of codec that argued about what to discard and who pays to decode it eventually arrived in FFmpeg, usually before broad hardware support existed, which made FFmpeg the tool that let people evaluate new formats without waiting for manufacturers to ship support.

The container story is equally consequential. AVI was designed in 1992 for something far smaller than what people put inside it; FFmpeg's AVI support is complete enough that it handles the hacks and extensions that emerged when the format was pushed past its design limits. Matroska adopted FFmpeg as a key part of the tooling that made MKV practically usable. The MP4 container, which emerged from the MPEG process and absorbed QuickTime's structural logic, is fully supported in both directions. The container that held more than it was designed to and the container built for exactly what the first one could not do are both first-class citizens inside the same library.

The project and its governance

In 2011, a governance dispute in the FFmpeg project led a group of developers to fork the codebase into a project called Libav. The split was technical in some respects and interpersonal in others, and it produced a period of parallel development that confused packagers and distributions. Debian and Ubuntu shipped Libav rather than FFmpeg for several years. The divergence was eventually resolved in practice if not formally: by around 2015, Debian and Ubuntu had returned to FFmpeg as the maintained upstream, Libav development slowed, and the fork was eventually discontinued. The episode is a reminder that infrastructure projects of this scale run on contributor goodwill that is not guaranteed.

What the split did not change was FFmpeg's position as the reference implementation layer for the codec ecosystem. MPEG LA, which administers patent pools for H.264 and other formats, has no licensing relationship with FFmpeg — the project does not distribute encoded content or manufacture devices, and whether use of patented codec technology in an open-source implementation carries legal risk has never been tested in a ruling that would change the practical situation. The Xiph.Org Foundation's royalty-free formats — Vorbis, FLAC, Opus — are supported in FFmpeg without complication because they were designed to be. The patent-encumbered ones are supported because the project made an engineering decision to implement them, and that decision has never been legally unwound.

FFmpeg is not marketed. It has no revenue. Its continued development depends on contributors — employed by companies that use it, or working independently — choosing to maintain something that the video compression ecosystem would be materially diminished without. The tools with logos and purchase buttons depend on it. The streaming infrastructure depends on it. The archive of formats that would otherwise be unreadable depends on it. The piece nobody markets is, quietly, the piece that holds the rest together.