Subtitle formats explained: SRT, WebVTT and captions

Subtitles look like a solved problem right up until they are not. You drag a .srt onto a video player and the text appears, perfectly timed, and you never think about it again. Then a video platform rejects the file and asks for .vtt, or the Korean dialogue turns into rows of question marks, or the captions start three seconds early and get worse as the film goes on. Once you know what is actually in the file, all three of those stop being mysteries.

A subtitle file is just timed text

There is no picture in a subtitle file, no audio, and no link to the video beyond a shared clock. What it contains is a list of cues, and each cue is three things: a number, a start and end timestamp, and the lines of dialogue. The player reads the file into memory, watches its own playback position, and draws the text on screen whenever the clock falls inside a cue's window. That is the entire mechanism.

Which explains two things people find surprising. A two-hour film's subtitles weigh about 60 KB, because it is a text file. And a subtitle file will happily play against a completely different video — it has no idea which one it belongs to, and will show its lines regardless of whether they match what is being said.

SRT, the one that works everywhere

SubRip Text came out of a Windows program called SubRip in the late 1990s, which pulled subtitles off DVDs. It never had a formal specification and it still does not, which somehow did not stop it becoming the format every player on earth supports. A cue looks like this:

1
00:01:14,880 --> 00:01:17,320
I told you not to open that door.

2
00:01:18,000 --> 00:01:20,450
- Someone had to.
- Not you.

The rules are short. Cues are numbered sequentially from 1. Timestamps run HH:MM:SS,mmm with a comma before the milliseconds, which is a European convention that has confused programmers for twenty-five years. An arrow separates start from end. A blank line ends the cue. Files are UTF-8 by convention rather than by rule, which causes most of the trouble described further down.

There is no styling in SRT and no positioning. Text lands wherever the player decides to put it, which is nearly always centred at the bottom. Many players tolerate a handful of HTML-ish tags — italics, bold, underline, a font colour — but that is player generosity, not specification. SRT survives because it is trivial to parse and you can fix one in Notepad without any tooling at all.

WebVTT, the one browsers wanted

WebVTT was written for the web, specifically for the HTML5 <track> element. A page hands the browser a video and a caption file and the browser handles rendering, toggling and the caption menu:

<video controls>
  <source src="film.mp4" type="video/mp4">
  <track src="film.vtt" kind="subtitles" srclang="en" label="English" default>
</video>

The file itself looks close to SRT, with a few differences that matter:

WEBVTT

NOTE Translation reviewed by KH, March

warning
00:01:14.880 --> 00:01:17.320 line:85% align:center
I told you not to open that door.

00:01:18.000 --> 00:01:20.450
- <c.speaker2>Someone had to.</c>
- Not you.

The first line must be the word WEBVTT. That is not decoration — a browser is allowed to reject the whole file if it is missing, and usually does, which is the single most common reason a hand-made .vtt shows no captions at all. Milliseconds use a period, not a comma. The hours field is optional, so 01:14.880 is valid. Cue numbers are optional too, and the line before a timestamp can be any identifier you like, useful for referencing a cue from script or CSS.

On top of that WebVTT adds things SRT never had. Cue settings after the timestamp control vertical position, horizontal alignment, box width and vertical writing mode for Japanese. Class spans let you tag speakers and style them from CSS with the ::cue selector. NOTE lines are comments. REGION blocks support roll-up captions of the kind used in live broadcast. And a track marked kind="chapters" turns the same syntax into a chapter list.

Browsers standardised on it because it was designed against their constraints: text-based, styleable with CSS they already had, subject to normal cross-origin rules, and defined to never throw — a malformed cue gets skipped rather than taking the page down with it. SRT could not meet any of those, mainly because there was no specification to hold it to.

Converting SRT to VTT is mostly punctuation

Given all that, the conversion is less dramatic than it sounds. Add the WEBVTT header and a blank line at the top, swap the commas in timestamps for periods, and you have a valid VTT file. Cue numbers can stay exactly as they are. Everything else already lines up. That is what SRT to VTT does, along with normalising the text to UTF-8 on the way through, which quietly fixes the encoding problem before it reaches the browser. It runs in your browser, so the file is not uploaded anywhere.

Sometimes you do not want a subtitle file at all — you want the words. The main converter can strip a subtitle file down to plain dialogue text, dropping the indexes and timestamps and leaving the script. Handy for a transcript, for handing a translator something readable, or for checking how much dialogue there actually is.

The other formats worth knowing

ASS and SSA (Advanced SubStation Alpha) come out of the anime fansub world and are far more capable than either format above. A file has a styles section with named styles, fonts, outlines, shadows and margins, plus override tags inside the dialogue for karaoke timing, absolute positioning, rotation and animated transitions. This is how you get lyrics that fill in syllable by syllable, or a translation note pinned to a sign in the background. The cost is fragility: an ASS file rendered on a machine without the fonts it names will look wrong, sometimes badly so.

SUB with IDX (VobSub) is a different species. DVD subtitles were never text — they are bitmap images with a small colour palette, overlaid on the frame. The .sub holds the pictures and the .idx holds the timings. You cannot edit a typo in one without an image editor, and you cannot search or translate it without running OCR first. Blu-ray does the same thing in a format called PGS, usually seen as .sup files. If a subtitle track ripped from a disc refuses to convert to SRT, this is why.

Broadcast captions are different again. EIA-608 and CEA-708 ride inside the video signal itself rather than in a separate file, a holdover from analogue television where captions lived on line 21 of the picture. Broadcast workflows exchange them as SCC or MCC files, and streaming services generally use TTML, an XML format, for distribution masters.

Soft subtitles versus burned in

Soft subtitles stay a separate track, either as a sidecar file next to the video or muxed inside an MKV or MP4. Viewers can turn them off, change the size, pick a language, and a search engine can read them. Burned-in subtitles — hardsubs — are drawn into the pixels during encoding. They are part of the picture forever.

Each has its place. Burn in when the video is going somewhere that strips subtitle tracks and autoplays muted, which is most social feeds. Keep them soft everywhere else, because a burned-in caption cannot be translated, cannot be turned off when it covers something, goes blocky at low bitrate, and is invisible to both screen readers and crawlers.

The three things that actually go wrong

Mojibake. The subtitle was saved in a legacy encoding — Windows-1252, CP949 for Korean, Big5 for Chinese — and the player assumes UTF-8. Accented letters become pairs of symbols, and non-Latin scripts become solid nonsense. Re-saving the file as UTF-8 fixes it. Watch for the opposite trap too: a UTF-8 byte order mark at the start of an SRT makes the first cue number unparseable in some older players, so cue 1 silently vanishes.

Frame rate drift. Subtitles timed against a 23.976 fps release, played against a 25 fps version, run about 4% fast. They look fine for the first minute and end up more than five minutes adrift by the closing credits. No amount of nudging the offset will fix this — the timestamps have to be rescaled.

A constant offset. The release you have was cut differently: an extra studio logo at the front, or an extended edition. Every cue is wrong by the same amount. Any player can fix this live; in VLC the G and H keys shift subtitles by 50 ms a press.

Telling the last two apart takes ten seconds. Check the sync at the start of the film and again near the end. Same error both times means an offset. Growing error means frame rate.

Why captions are worth the work

The obvious reason is access: viewers who are deaf or hard of hearing cannot use your video without them, and in a fair number of jurisdictions that is a legal obligation rather than a courtesy. The less obvious reason is that most people watching video on a phone in public have the sound off, and captions are what keeps them watching. Language learners read along. People in open-plan offices read along.

There is a search benefit as well. A VTT track is plain text a crawler can read, so the content of your video becomes indexable in a way the audio never is. Upload accurate captions to a video platform and they take precedence over the automatic transcription, which improves both the on-screen result and what search picks up. A caption file is the cheapest transcript you will ever produce, and you get it as a side effect of work you had reason to do anyway.