Video File Formats Explained: What Is Actually Inside an MP4, MKV or MOV

A container is a filing system that happens to be one file. It does not compress anything, it does not improve or degrade quality, and on its own it explains almost nothing about how a video will look. What it does is decide how tracks are stored, how they are indexed, and how much a player has to read before it can start.

Those three decisions turn out to explain a surprising amount of everyday behaviour: why a half-downloaded MP4 refuses to play at all, why the subtitles vanished when someone converted a file, and why some formats can be repackaged in seconds while others need an hour of processing.

What is actually in there

Open any container and you find the same categories of thing.

Tracks. At minimum a video track, usually at least one audio track, often several — commentary, another language, a downmix. Subtitle tracks and chapter markers are tracks too, in the containers that support them.

Timing. Each track carries a timescale and a set of timestamps saying when each chunk of data should be presented. This is what keeps audio and video in sync, and it is why a file can hold tracks with completely different internal rates and still play correctly.

An index. A map from playback position to byte offset. Without it a player cannot seek — dragging the scrub bar would mean reading forward through the file until it arrived.

Metadata. Title, language tags, track ordering, which audio track is the default, and whatever else the format allows.

Attachments, in the containers that permit them. Matroska will carry font files so that styled subtitles render as the author intended.

What is not in there is any compression logic. The video track holds data that a codec already compressed; the container just stores it, labels it and keeps it in order. That single fact is the reason you can move tracks between containers without touching quality.

The MP4 family

MP4 is standardised through ISO, and it descends fairly directly from Apple's QuickTime file format — the ISO base media file format was built on QuickTime's structure, and the family resemblance is still visible in the internals.

The structure is a tree of boxes (also called atoms in QuickTime terminology), each with a size and a four-character type. Two matter for practical purposes:

• mdat — the media data. The actual compressed video and audio, in bulk.
• moov — the movie box. Track definitions, timing and the sample tables that make up the index.

Here is the consequence that catches everybody. When a file is written by a normal encoder, the sizes and offsets that go into moov are not known until all the media has been written, so moov is placed at the end. A player therefore needs the end of the file before it can play the beginning.

That is why a download interrupted at ninety per cent frequently will not play at all, while an interrupted download of some other formats plays right up to where it stopped. Nothing is corrupt in the ordinary sense — the index simply is not there.

Files intended for progressive streaming are post-processed to move moov to the front, an operation usually labelled "faststart". It changes nothing about the content and everything about whether playback can begin before the file has fully arrived.

The relatives

MOV is the QuickTime container. Structurally the same family, and widely produced by cameras and editing software.

M4V is Apple's variant of MP4. Mechanically it is an MP4 with a different extension; some M4V files distributed by Apple carry DRM, which is a licensing matter rather than a format one and means those specific files are meant to be played in the software that sold them.

3GP is the mobile profile defined by 3GPP, built on the same base format and constrained for the bandwidth and codecs of early mobile networks. It still turns up in files from older phones.

If you want to know which codecs are officially registered for use in this family, the MP4 Registration Authority maintains the list — which is a good illustration of how conservative the format is by design. MP4's compatibility comes from being deliberately narrow.

Matroska and WebM

Matroska takes the opposite position. It is an open container built on EBML, a binary structure loosely analogous to XML, and its design goal is to carry essentially anything.

Practical consequences of that permissiveness:

• Effectively unlimited tracks, so a file can hold several audio languages, several subtitle tracks and chapters at once.
• Styled subtitle formats with positioning and typography, plus the font files needed to render them.
• Codecs the format's authors never anticipated, because the container does not maintain a closed list.
• Chapters, tags and ordered editions.

This is why Matroska dominates archival and enthusiast use, and it is also the direct cause of its reputation for not working. If a container will carry anything, then any file you receive in it may contain something your device cannot decode — and the failure gets blamed on the extension. The container parsed fine. Something inside it did not.

Matroska's specification has been through formal standardisation at the IETF, in the working group set up to document it and the related lossless formats, which is a meaningful maturity signal for a format that began as a community project.

WebM is the constrained version: a subset of Matroska restricted to a small set of royalty-free codecs, built for use in web browsers. Everything true of Matroska's structure is true of WebM; the difference is the list of things allowed inside.

Transport streams

TS and M2TS come from a completely different design problem, and understanding that problem explains everything odd about them.

MPEG-2 Transport Stream, specified in ITU-T H.222.0, was designed for broadcast. A broadcast has no beginning. A receiver might be switched on at any moment, and the signal may arrive with errors. So the format is built as a continuous sequence of small fixed-size packets — 188 bytes each — with the information needed to start decoding repeated periodically throughout the stream rather than stored once at the front.

That gives it properties no file-oriented container has:

• You can start playing from almost anywhere, because the setup information comes around again shortly.
• You can cut it anywhere and both halves remain playable.
• It tolerates corruption gracefully, losing the damaged packets rather than the file.

And it costs you overhead. Packet headers on every 188 bytes and repeated metadata make a transport stream noticeably larger than the same content in MP4.

M2TS is the variant used on Blu-ray discs and by many camcorders, which adds a timestamp to each packet. Files that came off a disc or a broadcast capture are usually one of these two, which is why they turn up on people's drives despite nobody choosing them deliberately.

The legacy shelf

FLV was Adobe's Flash Video container, and for a stretch of the 2000s it was how video on the web worked. Adobe ended support for Flash Player on 31 December 2020, and the container has no remaining reason to exist — but files created during that era still sit in people's archives, which is why players continue to open them.

AVI is older still: a Microsoft container from the early 1990s built on the RIFF structure, from the Video for Windows era. It predates modern expectations about variable frame rates, multiple audio configurations and subtitle tracks, and working around those limitations produced a long history of awkward extensions. It is worth naming here because it is the one common legacy container that our own player does not list among its supported formats — if you have an AVI archive, remuxing it into MKV is generally the sensible move regardless of what you play it with.