Video Codecs Explained: Compression, Bitrate, Bit Depth and Why Newer Is Not Always Better

Uncompressed video is absurd. A single frame of 1080p with three colour components per pixel is a few megabytes on its own, and at twenty-five frames a second that is a rate no consumer storage or network was ever built to sustain.

Everything you have ever watched on a screen is therefore the output of a codec deciding what to throw away. Understanding roughly how it makes that decision explains most of the confusing things about video files — why one clip is ten times the size of another at the same resolution, why scrubbing jumps in chunks, and why a phone that plays a codec perfectly can produce a black screen for a different file in the same codec.

Compression is mostly prediction

The intuition that compression means "storing the picture more efficiently" is wrong in a way that hides how the whole thing works. A modern codec barely stores pictures at all. It stores predictions and corrections.

Four stages, in order.

1. Prediction. For each block of pixels, the encoder finds something it already has that resembles it. Two sources are available:

• Intra prediction uses neighbouring pixels within the same frame. A sky gradient is highly predictable from the pixels above it.
• Inter prediction uses a region from a nearby frame, together with a motion vector describing where that region moved to. If the camera pans across a static scene, almost every block in the new frame exists somewhere in the previous one.

2. Residual. The prediction is subtracted from reality. What is left — the residual — is usually close to zero almost everywhere, and data that is mostly zero compresses extremely well.

3. Transform and quantisation. The residual is transformed into frequency components and then quantised, which means dividing by a step size and rounding. Coarse detail survives; fine detail rounds to zero and is gone forever. This is the lossy step, and the quantisation step size is essentially what a "quality" slider controls.

4. Entropy coding. The remaining numbers are packed using a statistical coder that gives short codes to common values.

The consequence that matters practically: file size tracks unpredictability, not duration. A locked-off interview compresses to almost nothing because each frame is nearly the previous frame. Falling confetti, rain, fire, water and hand-held footage in a forest compress terribly, because nothing predicts anything and the residual is real data on every block.

This is also why re-encoding an already-compressed file is worse than encoding the original. The second encoder is now trying to predict an image that contains the first encoder's artefacts, and it faithfully spends bits reproducing them.

I, P and B frames, and why seeking is chunky

Frames come in three types, defined by what they are allowed to predict from.

• I-frames (intra-coded) reference nothing outside themselves. They are complete pictures and they are large.
• P-frames predict from earlier frames. Much smaller.
• B-frames predict from both earlier and later frames. Smaller still, and the reason encoded frame order differs from display order.

A GOP — group of pictures — is the span from one I-frame to the next. Because a P- or B-frame is meaningless without its references, a decoder can only start playback at an I-frame.

That single constraint explains scrubbing behaviour. When you drag the position bar, the player jumps to the nearest I-frame and decodes forward from there. Long GOPs give better compression and coarser, slower seeking; short GOPs give responsive seeking and larger files. Streaming formats use short GOPs because segments must be independently decodable; archival encodes use long ones.

It also explains a specific corruption pattern. If an I-frame is damaged, the error propagates through every frame that predicted from it until the next I-frame arrives, which is the smeared, melting look you occasionally get from a bad broadcast or a damaged file.

Bitrate is a budget

Bitrate is how many bits per second the encoder is allowed to spend. It is not a quality setting, because the same budget is generous or starved depending on what it has to cover: resolution, frame rate and the complexity of the footage.

The three common control modes:

Constant bitrate (CBR) holds the rate steady regardless of content. Necessary for broadcast and some streaming, wasteful for everything else, because simple scenes get bits they do not need and complex scenes get starved.

Variable bitrate (VBR) lets the rate rise and fall with complexity, typically targeting an average. Better quality per byte, less predictable file size.

Constant quality — the mode usually exposed as a quality factor rather than a bitrate — fixes the quantisation aggressiveness instead of the data rate, and lets the size land wherever it lands. This is generally the right choice for files you are keeping, because it spends bits where they are needed and stops spending them when they stop helping.

Two things follow that people often get wrong:

• Raising bitrate past a point buys nothing. Once the encoder is no longer discarding anything perceptible, additional bits reproduce noise.
• Resolution and bitrate are one decision. Downscaling 4K footage to 1080p and keeping the same bitrate roughly quadruples the bits available per pixel, which frequently looks better than the 4K version did. On a phone screen this is almost always the right trade.

Chroma subsampling

Human vision resolves brightness detail far more finely than colour detail — a consequence of how the retina is built, with luminance information carried at much higher spatial acuity than colour information. Codecs exploit this by storing colour at lower resolution than brightness.

The notation looks cryptic and means something simple:

4:2:0 discards three quarters of the colour samples and is almost invisible on photographic content. Where it shows is anywhere colour carries fine detail rather than brightness: saturated red text on a dark background, thin coloured lines, screen recordings of user interfaces, and chroma-key work where the edges of the key need colour precision.

If a screen recording of text looks strangely smeared and you cannot work out why, this is usually the reason, and the fix is to record in a format that keeps more colour information rather than to raise the bitrate.

Bit depth, banding, and the hardware trap

Bit depth is how many distinct levels each component can take. 8-bit gives 256 levels per channel; 10-bit gives 1024.

The visible symptom of insufficient depth is banding — a smooth gradient like a sunset or a studio backdrop rendering as visible steps rather than a continuous ramp, because there simply are not enough available levels between the two ends.

Ten-bit encoding helps even for material that will be shown on an 8-bit display, because the extra precision reduces the accumulation of rounding error during compression. This is why a good deal of modern content is distributed in 10-bit regardless of the display it is heading for.

And here is the trap. Bit depth is a separate profile with separate hardware support. A chipset can carry a fixed-function decoder for 8-bit H.265 and not for 10-bit H.265. The file reports the same codec either way, so the failure looks arbitrary: audio plays, the picture is black, and every tool tells you the codec is one your device supports. It supports the codec. It does not support that profile.

Profiles and levels

This generalises. Every modern codec defines:

• Profiles — which coding tools a bitstream is allowed to use. Higher profiles compress better and demand more of the decoder.
• Levels — limits on resolution, frame rate and bitrate that a decoder must handle.

A device advertising support for a codec supports specific profiles at specific levels. That is why a phone can decode a codec at 1080p and fail on the same codec at 4K60: it is a level limit, not a codec limit.

When a file will not play and everything says it should, the answer is nearly always in the profile and level fields rather than the codec name.