H.264 is still everywhere, and that is the achievement

The codec is the argument · Long

Ratified in 2003, H.264/AVC became the codec that everything had to support — and then kept that position long after its successors arrived.

The problem it was built to solve

By the late 1990s, the mismatch between video quality and available bandwidth had become the central engineering problem of digital media. MPEG-2, the format that powered DVD-Video, was efficient enough for a disc with gigabytes to spend, but it was wasteful at the lower bitrates that mattered for streaming, mobile delivery, and videoconferencing. The Moving Picture Experts Group had already produced MPEG-4 Part 2 — the specification that DivX and XviD encoded against — but the quality gains over MPEG-2 were real without being transformative. What the industry needed was a codec that could deliver broadcast-quality video at roughly half the bitrate of MPEG-2, across every delivery channel simultaneously.

The joint project that produced H.264 was unusual in scale. The ITU-T Video Coding Experts Group and the ISO/IEC Moving Picture Experts Group worked together under a Joint Video Team formed in 2001, pooling contributions from companies including Apple, Sony, Panasonic, Fraunhofer, and others. The formal name from the ITU side was H.264; from the MPEG side, MPEG-4 Part 10 Advanced Video Coding, hence AVC. Both names refer to the same bitstream. Ratification came in 2003, with subsequent amendments extending the profile structure over the following years.

The technical decisions made inside that joint effort are what gave H.264 its longevity. The codec introduced context-adaptive binary arithmetic coding alongside the older context-adaptive variable-length coding, giving encoders a choice of entropy method depending on the target hardware. It formalised a more flexible reference frame model, allowing an encoder to predict any frame from multiple previously decoded frames rather than just the nearest one — a significant gain in efficiency on complex motion. The deblocking filter was baked into the codec loop rather than applied as a post-process, which meant it contributed to subsequent predictions rather than only improving the visible frame. None of these ideas were entirely new; H.264 assembled them into a single, carefully specified system.

Profiles, levels, and the hardware bet

H.264 ships with a profile hierarchy — Baseline, Main, Extended, High, and further High variants — that allowed the same standard to serve a phone in 2004 and a Blu-ray disc in 2006. Baseline profile dropped the arithmetic coder and some frame types to keep decode complexity low; High profile added the full tool set and became the mandatory codec for Blu-ray Video, alongside VC-1 and MPEG-2. That range meant silicon vendors had a single specification to implement across wildly different product categories, and they did. By the late 2000s, hardware decode acceleration for H.264 was present in graphics chips, mobile SoCs, set-top boxes, and game consoles in a way no previous codec had achieved at the same pace.

The Levels within each profile set concrete caps on resolution, frame rate, and bitrate, expressed as a single number — Level 4.1, for instance, defined the ceiling that Blu-ray and many early streaming services encoded against. This gave device manufacturers an unambiguous compliance target and gave content producers a reliable floor: encode to a known level, and any compliant decoder handles it. The simplicity was deliberate and consequential.

Two-pass encoding, long-GOP structures, and careful keyframe placement were all well understood by the time H.264 arrived, and the format accommodated them without prescribing them — the standard specifies the bitstream a decoder must accept, not how an encoder should produce it. That distinction matters. Encoders from x264, the open-source implementation, to hardware chips in broadcast equipment all produce valid H.264 by different means, and a decoder need not know which produced the stream it is reading.

The licensing question

H.264's reach came with a cost structure that its successors were partly defined in reaction to. MPEG LA assembled a patent pool covering H.264 and administered it as a licensing program: any product that encoded or decoded H.264 commercially was, in principle, liable for royalties. The terms were adjusted over the years — internet video distributed free to end users was eventually exempted from per-unit royalty obligations — but the underlying framework remained a patent pool with a single administrator. It was the HEVC licensing fragmentation, where competing pools made compliance terms genuinely unclear, that accelerated the industry's move toward royalty-free alternatives. The Alliance for Open Media formed in 2015 explicitly to produce AV1 as a codec nobody had to license, and its founding membership read as a list of organisations that had grown tired of the H.264 and HEVC models.

Yet the reaction against licensing did not displace H.264. It displaced the expectation that H.264's successor would be similarly encumbered. H.264 itself continued to accumulate hardware support through the entire period that AV1 was being designed and deployed. By the time AV1 reached meaningful streaming use in the early 2020s, H.264 had nearly two decades of silicon behind it — not just in current devices but in the installed base of televisions, game consoles, and mobile handsets that had not been replaced. Streaming platforms targeting the widest possible reach still encode an H.264 rendition as a compatibility layer, not because it is the most efficient choice but because it is the one thing a decoder from 2010 and a decoder from 2024 have in common.

What endurance actually means

H.264's endurance is sometimes read as inertia, but that undersells the engineering bet made at the Joint Video Team. The profiles were designed to be implementable in hardware at the manufacturing costs of the time. The bitstream was specified tightly enough that interoperability held across a decade of independent implementations. The quality-per-bit was genuinely better than what preceded it, by a margin large enough that content producers and device makers both had immediate reason to move.

HEVC — H.265 — was ratified in 2013 and promised roughly equivalent quality at half the bitrate again. A decade later it remains less universally supported in hardware than H.264 was at a comparable age, in part because of the licensing complexity that slowed chip vendors' commitment. The contrast makes the H.264 outcome look less inevitable. A codec can be technically superior and still fail to achieve the hardware ubiquity that makes quality arguments moot. H.264 achieved ubiquity, and ubiquity is what the format-history books record long after the specific coding tools are forgotten.