Everything you need to know about HTML5 video and audio

By Simon Pieters

Introduction

The latest version of Opera supports the HTML5 video and audio elements. But how do you use them? Introduction to HTML5 video is a great general introduction but doesn't go deep into the details. Accessible HTML5 Video with JavaScripted captions shows how captions can be implemented until the spec gains proper support for captions; and (re-)Introducing <video> has some information on Opera's implementation. I recommend reading all three!

This article aims to provide all the nitty-gritty details of HTML5 media, the DOM API, events, and so forth, so you can implement your own HTML5 player with fallback for older browsers.

Editor's note: This article was originally published on the Opera Core Concerns blog, but we liked it so much that we convinced Simon to let us publish it here as well.

What's supported?

Opera supports everything in the HTML5 video spec with the following exceptions:

• The preload attribute is not supported. (autobuffer was changed to preload in the spec; Opera has autobuffer in the DOM but it doesn't do anything.)
• The buffered, seekable and played IDL attributes always return empty TimeRanges objects.
• playbackRate and defaultPlaybackRate don't affect playback speed or direction.

Currently Opera supports the WebM container format with the VP8 and Vorbis codecs, the Ogg container format with the Theora and Vorbis codecs, and the WAVE container format and PCM codec.

Let's get something playing

So, how do we get a video to play in HTML? First you need an actual video in the right format. Opera currently supports Ogg/WebM, which is also supported by Firefox and Chrome.

If you have a video that you want to play but it's not in Ogg/WebM, you need to convert it. You can use Miro or another program of choice to do this.

So now you have a video lying around on your server (or your local disk), and you want to play it in HTML. Use the following markup:

<video src="video.ogv" controls>
 video not supported
</video>

The controls attribute instructs the browser to provide its own controls. If you want to write your own controls with JavaScript, you just leave out the controls attribute. The browser's controls can still be enabled by the user from the context menu in Opera, and when scripting is disabled, Opera's controls are present regardless of the controls attribute.

The text "video not supported" will be shown if the browser doesn't support the video element; you could replace this with a link to the video file itself, or maybe an object element to display an alternative version with a plugin, eg Flash.

Depending on how your server is configured, the video might or might not actually play. Current Opera requires that your video file is served as video/ogg (or audio/ogg, or application/ogg, or audio/wav...) for it to play. So if it doesn't play, your server might not know about the ogv file extension and serve the video as text/plain, which Opera refuses to play. Here's how to fix this for Apache servers; add the following lines to your .htaccess file:

AddType video/ogg .ogv
AddType audio/ogg .oga

That sets the right type for Ogg audio as well. While you're at it you could add video/mp4 for mp4 extensions.

The audio element works much the same as the video element, except it doesn't show any video, and some features that only makes sense for videos are missing.

<audio src="audio.oga" controls>
 audio not supported
</audio>

You can also create video and audio elements with script. For a video to render anything, you also need to insert it in the document. An audio element doesn't need to be in the document to play sound, but it does if you want to show the browser's controls.

Here's how to create a video element and insert it as the last child of body:

var video = document.createElement('video');
video.src = 'video.ogv';
video.controls = true;
document.body.appendChild(video);

For audio, you can do the same thing:

var audio = document.createElement('audio');
audio.src = 'audio.oga';
audio.controls = true;
document.body.appendChild(audio);

There's also a convenient Audio() constructor, which is equivalent to creating an audio element with createElement, setting its src attribute to the constructor's first argument, if there is one, and setting the preload attribute to the value auto.

var audio = new Audio();
audio.src = 'audio.oga';

var audio = new Audio('audio.oga');

For the rest of this article, I'll mostly only show examples for video, although most apply to audio as well.

But it doesn't work in Safari!

Safari doesn't support Ogg/WebM out of the box — it instead supports the H.264 codec. There are a few options available to get round the conflicting codec support issue we are currently faced with:

• Encode your video twice — once as Ogg/WebM and once as MPEG-4.
• Tell Safari users to install the Xiph QuickTime Component. This will make Ogg work in Safari.
• Replace the video element with the Cortado Java applet when you detect that Ogg/WebM isn't supported.

To convert your video to MPEG-4/H.264/AAC, you can use HandBrake or some other program — this is also detailed in Dive Into HTML5.

Now you have two video files, you should expose both in the code so browsers can play whichever one they understand. To do this, you can use the source element as follows:

<video controls>
 <source src="video.ogv">
 <source src="video.mp4">
 video not supported
</video>

Now the browser will first try to load and play video.ogv, and if it can't play it, it will try the next source element. If you want to save precious bandwidth, you can tell the browser the MIME type of each video so it doesn't need to download it to tell whether it can play it:

<video controls>
 <source src="video.ogv" type="video/ogg">
 <source src="video.mp4" type="video/mp4">
 video not supported
</video>

However, these MIME types only tell you which container format is being used (Ogg or MPEG-4, in the above case) — it doesn't say anything about which video and audio codecs are being used. A container format is similar to a ZIP archive containing several other files; to know that you support the individual files, you'd need information about the individual files, not just the archive format. For video, we use the codec's MIME parameter for this purpose:

<video controls>
 <source src="video.ogv" type='video/ogg; codecs="theora, vorbis"'>
 <source src="video.mp4" type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"'>
 video not supported
</video>

Note that the codecs parameter uses double quotes, which means we have to use single quotes for the attribute value.

The codec strings for Theora and Vorbis are straightforward. The codec strings for H.264 and AAC are more complicated; this is because there are several profiles for H.264 and AAC. The above represent the Baseline profile for H.264 and the Low-complexity profile for AAC. Those are the profiles used by YouTube and supported on the iPhone. Higher profiles require more CPU to decode but are better compressed so take less bandwidth.

If you don't want to encode your video twice, you can show a message for Safari users. You could have a link visible for everyone, or you could detect that Ogg isn't supported and only then show a message. The second point requires detection, so let's go through how to do that.

Detecting support

There are several levels of support. First, the video element might not be supported at all. This is the case for Opera 10.10 and below and IE8 and below. For this case, you can just put content inside the video element and it will be rendered (in the above examples, the content is just "video not supported"). No need to do anything further for this case.

Second, the video element might be supported but the codecs you want to use might not be. Safari doesn't support Ogg/WebM, while Opera and Firefox don't support MPEG-4/H.264/AAC. To detect this, you can either use the canPlayType() method on a media element, or you could have an onerror event listener; if a video fails to play because the codec is not supported, an error event is fired.

The canPlayType() method takes a string argument in the form of a MIME type. The method returns one of three strings:

The empty string

The container format or one of the codecs are not supported.

"maybe"

The container format is probably supported, but don't know about the codecs.

"probably"

The container format and the codecs are probably supported.

Note there's no "yes" — a MIME type doesn't contain enough information for a browser to know for sure whether it can play a given video. For instance, the video might have too high a bitrate so that the browser is unable to decode it.

The MIME type is of the form video/ogg or video/mp4; codecs="..." — just like in the server configuration and the type attribute on source.

var video = document.getElementsByTagName('video')[0];

// Opera 10.50 gives "maybe"
alert(video.canPlayType('video/ogg'));

// Opera 10.50 gives "probably"
alert(video.canPlayType('video/ogg; codecs="theora, vorbis"'));

// Opera 10.50 gives ""
alert(video.canPlayType('video/mp4'));
alert(video.canPlayType('video/mp4; codecs="avc1.42E01E, mp4a.40.2"'));

If you have an Ogg video and want to detect support, you could do it as follows:

var video = document.getElementsByTagName('video')[0];
if (video.canPlayType) { // <video> is supported!
  if (video.canPlayType('video/ogg; codecs="theora, vorbis"')) {
    // it can play (maybe)!
  } else {
    // the container format or codecs aren't supported
    // let's fall back
    fallback(video);
  }
}

Note: HTML5 earlier said to return the string "no" instead of the empty string, which would make the above code never fall back (since the string "no" is truthy in JavaScript, while the empty string is falsy). If you want to support old video-supporting browsers that implemented "no", then you would have to check for that string explicitly, or check for "maybe" and "probably" instead.

The fallback function would take out the video and source elements from the DOM, but keep the other children of the video. This function could be implemented like this:

function fallback(video) {
  while (video.firstChild) {
    if (video.firstChild instanceof HTMLSourceElement) {
      video.removeChild(video.firstChild);
    } else {
      video.parentNode.insertBefore(video.firstChild, video);
    }
  }
  video.parentNode.removeChild(video);
}

The other way to detect lack of codec support is to listen for the error event on the video:

<video src="video.ogv" controls onerror="fallback(this)">
 video not supported
</video>

This will still make browsers that don't support Ogg download part of the video; we can fix that by using the source element and using onerror on the source element instead:

<video controls>
 <source src="video.ogv" type='video/ogg; codecs="theora, vorbis"'
         onerror="fallback(this.parentNode)">
 video not supported
</video>

At this point you can add a link to the Xiph QuickTime Component page for Safari users:

<video controls>
 <source src="video.ogv" type='video/ogg; codecs="theora, vorbis"'
         onerror="fallback(this.parentNode)">
 video not supported; if you're using Safari, try installing
 <a href="http://www.xiph.org/quicktime/">XiphQT</a>

</video>

If you have several source elements, the onerror handler would go on the last source element:

<video controls>
 <source src="video.ogv" type='video/ogg; codecs="theora, vorbis"'>
 <source src="video.mp4" type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"'
         onerror="fallback(this.parentNode)">
 video not supported
</video>

An error is fired on each failing source element, and since they're tried in order, you know all of them have failed if the last one gets an error event.

Falling back to plugins

If you want to try a plugin instead of showing a message when a video fails, you could use the Cortado Java applet for Ogg, or you could use Flash for MP4, since Flash supports playing MPEG-4/H.264/AAC.

If you just have an Ogg file, it could look something like this:

<video controls>
 <source src="video.ogv" type='video/ogg; codecs="theora, vorbis"'
         onerror="fallback(this.parentNode)">
 <object type="application/x-java-applet" width="480" height="288">
  <param name="archive" value="cortado-ovt-stripped-wm_r51500.jar">
  <param name="code" value="com.fluendo.player.Cortado.class">
  <param name="url" value="video.ogv">
  video and Java not supported
 </object>
</video>

If you just have an MP4 file, it could look something like this:

<video controls>
 <source src="video.mp4" type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"'
         onerror="fallback(this.parentNode)">
 <object data="videoplayer.swf">
  <param name="flashvars" value="video.mp4">
  video and Flash not supported
 </object>
</video>

If you have both an Ogg and an MP4 file, you could try falling back to both by nesting the object elements inside each other. You could also build up the fallback DOM with dynamically when you detect lack of support, which avoids having huge markup boilerplate for each video. The html5media project does this using the Flowplayer Flash video player.

At this point, you should have video working in all popular browsers — including Opera 10.10 and IE, assuming Java or Flash are installed and enabled.

What's up with all that downloading?

Opera, Chrome and Safari will automatically download the whole video file even if it hasn't started to play yet. Firefox 3.6 only loads enough to render a frame and determine duration, unless the autobuffer attribute is present. Note that the spec changed from autobuffer to preload, which hasn't been implemented anywhere yet. Opera plans to change to the Firefox behavior of only loading enough to render a frame and determine duration by default, unless the preload attribute says otherwise.

The preload attribute has the following states:

Attribute absent

The browser is allowed to download as little or much as it wants. When Opera implements preload, this will probably be equivalent to metadata

preload="none"

The author hints that nothing should be downloaded.

preload="metadata"

The author hints that enough data should be downloaded to show a frame and to determine duration.

preload=""

preload="auto"

The author hints that the browser should download as much as it sees fit to give a good user experience, possibly downloading the whole video.

If you want to simulate preload="none" in today's browsers, you can omit the src attribute and the source elements, and only add them when the user clicks a button.

<video controls>
 video not supported
</video>
<input type="button" value="Load video"
       onclick="document.getElementsByTagName('video')[0].src = 'video.ogv';">

To populate a video with source elements dynamically, it could be done as follows:

<video controls>
 video not supported
</video>
<script>
function loadVideo() {
  var video = document.getElementsByTagName('video')[0];
  video.insertBefore(createSource('video.ogv', 'video/ogg; codecs="theora, vorbis"'), video.firstChild);
  video.insertBefore(createSource('video.mp4', 'video/mp4; codecs="avc1.42E01E, mp4a.40.2"'), video.firstChild.nextSibling);
}
function createSource(src, type) {
  var source = document.createElement('source');
  source.src = src;
  source.type = type;
  return source;
}
</script>
<input type="button" value="Load video" onclick="loadVideo()">

This will show a blank video element until the user clicks the "load video" button. If you want to show an image instead of nothing then you can use the poster attribute.

<video controls poster="videoframe.jpg">

Alternatively you could use an img element and replace it with a video element dynamically when the user clicks on it or on a button.

What else have you got?

There are some more attributes for media elements that I haven't mentioned yet. I'll list them all here for completeness.

The video element:

src

URL for the video.

poster

URL to an image to use as the poster frame until the video starts playing.

preload

Hint to the browser how much it should download before the video starts playing.

autoplay

Boolean attribute that hints that the browser should start playing the video automatically.

loop

Boolean attribute indicating whether the video should loop.

controls

Boolean attribute indicating whether the browser should show its controls.

width

Width of the video box, in CSS pixels.

height

Height of the video box, in CSS pixels.

The audio element has the same attributes, excepting poster, width and height.

The autoplay attribute would probably be used on pages where the primary content is a single video — for instance, videos on YouTube start playing automatically. Users can in theory disable autoplaying videos with a preference in the browser, although I'm not aware of any such preference in existing browsers. If there weren't an attribute to do this, people would probably make videos start playing automatically with script anyway, which makes it harder for the user to disable them.

The loop attribute, if present, indicates that when the video has ended, the browser should seek back to the beginning if the direction of playback is forwards. (The video doesn't loop when playing backwards.)

The autoplay, loop and controls attributes are so-called boolean attributes. Such attributes represent the "off" state when absent, and the "on" state when present, regardless of the value. This is the same as e.g. the disabled attribute on input. In HTML5, these attributes can be written in three ways:

<video loop>

<video loop="">

<video loop="loop">

In the first case, the attribute value is the empty string. They all mean the same thing. It could also be written in all-uppercase or with mixed case.

In JavaScript, boolean IDL attributes return true if the attribute is present, and false if absent. Setting a boolean IDL attribute to true means the corresponding attribute is set with the same value as the attribute name, and setting it to false means the corresponding attribute is removed. Thus, the following are equivalent:

video.loop = true;

video.setAttribute('loop', 'loop');

The width and height attributes set the dimensions for the video element, in CSS pixels. You should not use any unit for these attributes; just like with the img element. Also — in the same way as the img element — if you only set one of width and height, the other dimension is automatically adjusted appropriately so that the video retains its aspect ratio. However — unlike the img element — if you set width and height to something that doesn't match the aspect ratio of the video, the video is not stretched to fill the box. Instead, the video retains the correct aspect ratio and is letterboxed inside the video element. The video will be rendered as large as possible inside the video element while retaining the aspect ratio.

You can read out the video's intrinsic width and height by using the videoWidth and videoHeight IDL attributes:

<video src="video.ogv" width="300" height="300"
       onloadedmetadata="alert(this.videoWidth + 'x' + this.videoHeight);">
 video not supported
</video>

If you want to use a percentage width or other units, you can set the dimensions with CSS. If you want, you can change the dimensions on :hover and/or :focus, and use the -o-transition CSS property to smoothly transition between the two.

video { width:100px; -o-transition:0.5s width }
video:hover, video:focus { width:400px }

The source element has three attributes:

src

URL for the video.

type

MIME type for the video.

media

A Media Query indicating for which medium the video applies.

The media attribute takes a Media Query, just like the media attribute on the style element. For instance, you could specify media="handheld" to indicate that the video is appropriate for handheld devices. Or you could specify media="all and (min-device-height:720px)" to indicate that the video is appropriate for screens with 720 lines of pixels or bigger.

I want to roll my own controls

If you're satisfied with the browser's native controls, then you can stop reading now. If you want the controls to have a different design, or you want a button for captions, or playback speed, and so forth, then read on.

In the early days, the DOM API for video in HTML5 was very simple; you could play() a video, and you could pause() a video, and that was more or less it. Today the API is a lot bigger, and there are lots of events, so that you can implement sophisticated controls in JavaScript.

When using your own controls, you omit the controls attribute.

<video src="video.ogv">
 video not supported
</video>

You can then use buttons that do something when clicked:

<script>
var video = document.getElementsByTagName('video')[0];
</script>
<input type="button" value="Play" onclick="video.play()">
<input type="button" value="Pause" onclick="video.pause()">

You can then style those buttons in some fancy way, for instance:

input[type=button] {
 background:papayawhip;
 color:black;
 height:3em;
 border:double;
 border-radius:0.5em;
 box-shadow:0 0.2em 0.5em black;
}

If you want to use a single button for both play and pause, you need to listen for the play and pause events. The user can play and pause from the context menu, so if you just naively toggle between play and pause for every click on your button, it would become out of sync.

There are three ways to set an event listener in HTML. The first is to use a normal attribute:

<video onplay="exampleFunction()">

The second is using an IDL attribute with JavaScript:

video.onplay = exampleFunction;

The third is to use the DOM Events addEventListener method:

video.addEventListener('play', exampleFunction, false);

Let's replace the two buttons with a single play/pause button:

<input type="button" value="Play" id="playpause" onclick="video.play()">

Initially we assume that the video is paused, since that's the initial state. Even when the autoplay attribute is used, the video is still in the paused state until some video data has been loaded and the browser decides to start playing.

Now, we need to change the button when the video is played or paused:

var playpause = document.getElementById('playpause');
video.onpause = function(e) {
  playpause.value = 'Play';
  playpause.onclick = function(e) { video.play(); }
}
video.onplay = funtion(e) {
  playpause.value = 'Pause';
  playpause.onclick = function(e) { video.pause(); }
}

Alternatively, we could come up with a more elegant solution and take look at the paused IDL attribute when updating the button's label and deciding what to do when clicking the button:

<input type="button" value="Play" id="playpause" onclick="playOrPause()">

video.onpause = video.onplay = function(e) {
  playpause.value = video.paused ? 'Play' : 'Pause';
}
function playOrPause() {
  if (video.paused) {
    video.play();
  } else {
    video.pause();
  }
}

Note that when the video has ended, the paused state of the video is still unpaused. Thus, if you want to show the "play" button when the video has ended, you need to change it when getting the ended event.

video.onended = function(e) {
  playpause.value = 'Play';
}

The playOrPause function would be modified by also checking the value of the ended IDL attribute:

function playOrPause() {
  if (video.ended || video.paused) {
    video.play();
  } else {
    video.pause();
  }
}

Can you play anything yet?

Initially a video element won't load anything — it's empty. This is represented by the networkState IDL attribute having the value 0, which is represented by a constant on the video element called NETWORK_EMPTY

var video = document.createElement('video');
alert(video.networkState == video.NETWORK_EMPTY); // true

The other values for networkState are 1, NETWORK_IDLE, which means the browser has chosen a video to use but isn't downloading anything; 2, NETWORK_LOADING, which means the browser is trying to download data; and 3, NETWORK_NO_SOURCE, which means no source has been successfully loaded yet.

When we set the src attribute, or insert a source element as a child of the video, the element automatically starts the loading process, and will try to use the src if it was set or find a suitable source element after the current script has finished running. When the load starts, a loadstart event is fired.

The processing then depends on whether you used the src attribute or whether you used source elements. Let's go through the process for the src attribute first.

The currentSrc IDL attribute is set to the resolved value of src. Then the browser tries to download and decode the referenced video. If this fails, then an error event is fired, and the video's error IDL attribute is set to a MediaError object whose code IDL attribute is set to value 4, i.e. MEDIA_ERR_SRC_NOT_SUPPORTED, and networkState is set to NETWORK_NO_SOURCE.

var video = document.createElement('video');
video.src = 'a-video-that-is-unsupported';
video.onerror = function(e) {
  alert(video.error.code == video.error.MEDIA_ERR_SRC_NOT_SUPPORTED); // true
  alert(video.networkState == video.NETWORK_NO_SOURCE); // true
}

The other values of MediaError's code IDL attribute are 1, MEDIA_ERR_ABORTED, which means the user aborted the download (which will cause a abort event to be fired); 2, MEDIA_ERR_NETWORK, which means a network error occurred while the video was being downloaded; 3, MEDIA_ERR_DECODE, which means that a decoding error occured after successfully decoding part of the video.

When using source elements, the browser goes through the list of source element children of the video element, updating currentSrc for each new source element it visits. If the type attribute has a MIME type the browser thinks it can potentially play, and if the Media Query in the media attribute applies to the current medium, then tlt;/codepar/codeam name=he browser tries to download and decode that source. If the source cannot be used — determined either by looking at type and media, or by trying to download and decode the video — then an error event is fired on the current source element (not on the video element). The video's error IDL attribute is still null; there could be a later source that the browser is able to play. If this was the last source element, then networkState will be NETWORK_NO_SOURCE, but if you were to insert another source element with script, then the browser would try to play that one as well — the video element is still alive even though all source elements so far have failed.

If the load is successful, whether using the src attribute or using source elements, progress events will be fired as data is downloaded. When enough data has been loaded to determine the video's dimensions and duration, a loadedmetadata event is fired. When enough data has been loaded to render a frame, the loadeddata event is fired. When enugh data has been loaded to be able to play a little bit of the video, a canplay event is fired. When the browser determines that it can play through the whole video without stopping to download more data, a canplaythrough event is fired; this is also the case when the video starts playing if it has a autoplay attribute.

Note: currently Opera doesn't try to determine when it has enough data to be able to play through, but instead just fires canplay and canplaythrough at the same time. This will probably be fixed in a future release.

If the browser decides to stop downloading data in order to save bandwidth, it will fire a suspend event. If the server stops giving data (without closing the connection) for some reason, the browser fires a stalled event after three seconds. If the browser is playing faster than the server is serving data, or when seeking causes the browser to wait for downloading data, a waiting event is fired.

Note: currently Opera doesn't suspend downloads, and doesn't fire the stalled event.

If you want to have your play button disabled until the video is able to play, you can enable it on the canplay event:

<input type="button" value="Play" id="playpause" onclick="playOrPause()" disabled>

video.oncanplay = function(e) {
  playpause.disabled = false;
}

The readyState IDL attribute indicates how much data the browser has loaded. At first, the value is 0, HAVE_NOTHING. When the loadedmetadata event is fired, readyState is 1, HAVE_METADATA. When loadeddata is fired, readyState is 2, HAVE_CURRENT_DATA. When canplay is fired, readyState is HAVE_FUTURE_DATA. When canplaythrough is fired, readyState is HAVE_ENOUGH_DATA. Note however that readyState can jump several steps in one go, for instance from HAVE_METADATA to HAVE_FUTURE_DATA, skipping HAVE_CURRENT_DATA, so if you inspect the readyState attribute in the canplay event handler, it might not be the value you expect because it has changed already before the event handler is run.

video.oncanplay = function(e) {
  alert(video.readyState); // might be 2, 3 or even 4
}

Skip forward, please

If you want a seekbar, you should use the currentTime and duration IDL attributes and the timeupdate event.

The currentTime IDL attribute returns the current time in seconds as a float value. The duration IDL attribute returns NaN if the duration is unknown (which it is initially), Infinite if the video is streaming, or the actual duration in seconds as a float value if the duration is known and finite. The timeupdate event is fired whenever the current position changes in some way, e.g. during normal playback or because the user seeked in the video.

Let's add a seekbar:

<input type="range" step="any" id="seekbar">

This creates a slider control which we can update when we get the timeupdate event, and which we can make seek the video when changed. But first we need to set the seekbar's max attribute to the video's duration when it becomes known, which we do by listening to the durationchange event.

var seekbar = document.getElementById('seekbar');
function setupSeekbar() {
  seekbar.max = video.duration;
}
video.ondurationchange = setupSeekbar;

Now, we can make the video respond to changes to the seekbar, and make the seekbar change in response to the video's currentTime changing.

function seekVideo() {
  video.currentTime = seekbar.value;
}
function updateUI() {
  seekbar.value = video.currentTime;
}
seekbar.onchange = seekVideo;
video.ontimeupdate = updateUI;

This works fine for normal cases. However, the seekbar will be broken for streaming video. Why? Because the seekbar assumes that the video starts at time 0, and that the duration is not Infinity. For instance, consider a streaming video and the browser only caches the past 30 minutes worth of data. As the browser throws away data from the cache, you can't seek back to the thrown away data.

For this reason, there's an IDL attribute called startTime. For a normal video, it returns 0. For videos that have a timeline that isn't zero-based, it could be something different. For a streaming video, it's the earliest position the browser is able to seek back to.

For a non-streaming video whose timeline isn't zero-based, we can fix the above seekbar by setting the min attribute to the video's startTime, and setting max to startTime plus duration.

function setupSeekbar() {
  seekbar.min = video.startTime;
  seekbar.max = video.startTime + video.duration;
}

For a streaming video, duration is Infinity, so instead we need to set the max attribute to the latest time that has been buffered, and since startTime can change over time, we need to set the min attribute over time as well.

For getting the latest time that has been buffered, we need the buffered IDL attribute. It returns a TimeRanges object which has a length attribute, a start() method and an end() method. In normal cases, there will only be one range — the browser starts downloading from time 0, and the downloaded range extends to however much is currently available. However, if the user seeks forward, the browser can stop the current download and start a new request for a later part of the video. In this case, there would be two ranges of buffered data.

The TimeRanges object's length IDL attribute returns how many ranges there are. The start() method takes an argument index, where 0 represents the index of the first range, 1 represents the index of the second range, and so forth. It returns the start time of the range with the given index. The end() method similarly returns the end time of the range with the given index.

So to find out the latest position of buffered data, we read the end time of the last range in buffered:

var lastBuffered = video.buffered.end(video.buffered.length-1);

We can then use this to update the seekbar:

function updateUI() {
  var lastBuffered = video.buffered.end(video.buffered.length-1);
  seekbar.min = video.startTime;
  seekbar.max = lastBuffered;
  seekbar.value = video.currentTime;
}

The played and seekable IDL attributes also return TimeRanges objects. The played IDL attribute returns the ranges that have been played, and the seekable IDL attribute returns which ranges the browser is able to seek to.

These attributes can also be used to show fancy colored bars indicating which parts of the video has been downloaded, played, and are seekable. For the buffered attribute, you could update the bar for every progress event, which is fired when some media data has been downloaded. There's no event currently when media data is being discarded from the cache.

Note: currently Opera always returns empty TimeRanges objects for buffered, played and seekable. buffered and seekable are planned to be implemented, but we don't see clear use cases for played, and it's easy to keep track of what's been played with JavaScript, so for now we don't plan to implement played.

Seeking can be slow sometimes — especially if the time you're trying to seek to hasn't been downloaded yet. If you want to show a spinning icon or something while the browser is busy seeking, you can listen to the seeking and seeked events. seeking is fired when a seek starts, and seeked is fired when a seek is completed. There's also a seeking IDL attribute which returns true while the browser is seeking.

HTTP byte range requests

While on the subject of downloaded data, seeking and duration, let's talk about HTTP byte range requests. HTTP supports a way for clients to request a range of bytes of a file, and for the server to respond with sending only the requested bytes. This is great for seeking in video, because you don't need to wait for the whole video to have downloaded before you can seek to the end. Opera supports this, and lets you seek to any part of a video even though it hasn't been downloaded yet, assuming the server supports byte range requests.

HTTP byte range requests are not only needed to be able to seek quickly, it's also needed to determine the duration for Ogg files. The Ogg format doesn't include any metadata about the duration of the video, so to know the duration, the browser has to seek to the end. Opera does this. Depending on the video, it can result in a few extra requests. There have been some discussions about adding duration metadata to Ogg files, as well as stating the duration metadata as an HTTP header. When this happens, the extra requests will not be necessary.

Note that if your server doesn't support byte range requests, Opera will assume that the video is streaming, i.e. duration will be Infinity.

Adjust the volume

Media players usually have a mute button and a volume control. HTML5 provides the volume and muted IDL attributes, as well as the volumechange event. The event is fired whenever the value of volume or muted is changed.

To implement a mute button, we flip the value of muted, and we update the button's label when the volume changes:

<input type="button" value="Unmuted" id="mutebutton" onclick="muteOrUnmute()">

var mutebutton = document.getElementById('mutebutton');
video.onvolumechange = function(e) {
  mutebutton.value = video.muted ? 'Muted' : 'Unmuted';
}
function muteOrUnmute() {
  video.muted = !video.muted;
}

For the volume control, we can use a slider control just like the seekbar:

<input type="range" max="1" step="any" id="volumecontrol" onchange="updateVolume()">

var volumecontrol = document.getElementById('volumecontrol');
video.onvolumechange = function(e) {
  mutebutton.value = video.muted ? 'Muted' : 'Unmuted';
  volumecontrol.value = video.volume;
}
function updateVolume() {
  video.volume = volumecontrol.value;
}

Let's look at another movie

If you want to show several clips one after another, or otherwise dynamically change the source of a video element, it's possible without having to throw away the whole video element and creating a new one.

If you're using the src attribute, then it's super-simple: just set src to the new value; the current video will be aborted, and the new video will be loaded in.

<video src="video.ogv" controls>
 video not supported
</video>
<input type="button" value="Load another video"
       onclick="document.getElementsByTagName('video')[0].src = 'video2.ogv';">

However, if you're using source elements, then you need to call load() manually when you're done changing the source elements.

<video controls>
 <source src="video.ogv" type='video/ogg; codecs="theora, vorbis"'>
 <source src="video.mp4" type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"'>
 video not supported
</video>
<script>
function loadAnotherVideo() {
  var video = document.getElementsByTagName('video')[0];
  var sources = video.getElementsByTagName('source');
  sources[0].src = 'video2.ogv';
  sources[1].src = 'video2.mp4';
  video.load(); // need this for the new video to load
}
</script>
<input type="button" value="Load another video"
       onclick="loadAnotherVideo()">

If you want to load in another video when the current one has ended, you can listen for the ended event. Remember though that you'll get another ended event when the second video has ended, so if you just want two videos to play, you need to clear the event listener after it has run once. If you use an onended attribute or IDL attribute, then you can set the IDL attribute to null.

video.onended = function(e) {
  video.onended = null;
  video.src = 'video2.ogv';
}

If you use addEventListener(), then you remove the listener with removeEventListener():

video.addEventListener('ended', function(e) {
  video.removeEventListener('ended', arguments.callee, false);
  video.src = 'video2.ogv';
}, false);

When you call load(), or when you set src, or when a video fatally fails to load, an emptied event is fired, which allows you to reset your UI.

Fast forward, slow motion and rewind

The playbackRate IDL attribute sets the speed and direction of video playback. The default value is 1, meaning normal speed. Higher numbers mean fast forward. Lower numbers mean slow motion. Negative numbers mean playing backwards. The number can be any float value. When the playback rate is changed, a ratechange event is fired.

<input type="range" min="-3" max="3" value="1" id="ratecontrol"
       onchange="changePlaybackRate()">
<script>
var ratecontrol = document.getElementById('ratecontrol');
video.onratechange = function(e) {
  ratecontrol.value = video.playbackRate;
}
function changePlaybackRate() {
  video.playbackRate = ratecontrol.value;
}
</script>

The defaultPlaybackRate IDL attribute sets the default speed (and direction) of video playback. This could be useful if your video is incorrectly encoded so that its intrinsic speed is too slow or too fast. playbackRate is relative to defaultPlaybackRate. The default value of defaultPlaybackRate is 1. The ratechange event is also fired when defaultPlaybackRate changes value.

Note: currently Opera does not support playbackRate or defaultPlaybackRate.

How to keep things synchronized

There's currently no good API for synchronizing things with the timeline of a video, for instance captions or infoboxes. The spec has had "cue ranges" for this purpose earlier (which even earlier were "cue points"); it is expected that something similar will be added in the future, including support for declarative captions.

However, for now, you will have to either use a timer and read currentTime, or listen for timeupdate and read currentTime. timeupdate is fired at 15 to 250 ms intervals while the video is playing, unless the previous event handler for timeupdate is still running, in which case the browser should skip firing another event. Opera currently always fires it at 250 ms intervals while the video is playing, while Firefox currently fires it once per rendered frame. The idea is to allow the event to be fired at greater intervals if the system load increases, which could save battery life on a handheld device or keep things responsive in a heavy application. The bottom line is that you should not rely on the interval being the same over time or between browsers or devices.

Let's say you want to show a div element between the times 3s and 7s of the video; you could do it like this:

<div hidden data-starttime=3 data-endtime=7 id=hello>Hello world!</div>
<script>
var video = document.getElementsByTagName('video')[0];
var hello = document.getElementById('hello');
var hellostart = hello.getAttribute('data-starttime');
var helloend = hello.getAttribute('data-endtime');
video.ontimeupdate = function(e) {
 var hasHidden = hello.hasAttribute('hidden');
 if (video.currentTime > hellostart && video.currentTime < helloend) {
   if (hasHidden)
     hello.removeAttribute('hidden');
 } else {
   if (!hasHidden)
     hello.setAttribute('hidden', '');
 }
}
</script>

The hidden attribute indicates that the element is not relevant and should be hidden. This is not supported in browsers yet, so you have to hide it with CSS:

*[hidden] { display:none }

The data-starttime and data-endtime attributes are custom data-* attributes that HTML5 allows to be placed on any element. It's great for including data that you want to read with script, instead of abusing the class or title atributes. HTML5 also has a convenience API for data-* attributes, but it's not supported in browsers yet, so we have to use getAttribute a little longer.

The above would look like this using a timer instead:

<div hidden data-starttime=3 data-endtime=7 id=hello>Hello world!</div>
<script>
var video = document.getElementsByTagName('video')[0];
var hello = document.getElementById('hello');
var hellostart = hello.getAttribute('data-starttime');
var helloend = hello.getAttribute('data-endtime');
setInterval(function() {
 var hasHidden = hello.hasAttribute('hidden');
 if (video.currentTime > hellostart && video.currentTime < helloend) {
   if (hasHidden)
     hello.removeAttribute('hidden');
 } else {
   if (!hasHidden)
     hello.setAttribute('hidden', '');
 }
}, 100);
</script>

This will run every 100 ms. Whether you should use setInterval or timeupdate depends on what you're doing and whether you're ok with the interval changing. Note that the setInterval example above also runs when the video is not playing, which the timeupdate example doesn't. It's possible to clear the interval with clearInterval when the video stops playing and setting it again when it starts playing, though.

If you want to synchronize something with the time playback starts, or after a seek, you should listen for playing and seeked — not play or seeking. The former indicate when playback has actually started and a seek has finished, respectively, while the latter indicate that playback or seeking has just been requested, but could take some time before it actually occurs.

Video on a canvas

The canvas element is like a dynamic img element, which you can draw on with JavaScript. I'm not going into detail how canvas works in general (HTML5 canvas - the basics is a recommended introduction), but I'll mention that it's possible to draw a video element on a canvas using the 2d context's drawImage() method (which also accepts img and canvas elements, and in Opera, svg elements). It will draw the current frame of the video onto the canvas. This allows you to do transformations of a video and to read pixel data from a video, which could be used to detect faces or movement in JavaScript.

<video src="video.ogv" controls>
 video not supported
</video>
<canvas id="canvas">
 canvas not supported
</canvas>
<script>
var ctx = document.getElementById('canvas').getContext('2d');
var video = document.getElementsByTagName('video')[0];
video.onloadeddata = function(e) {
  ctx.canvas.width = video.videoWidth;
  ctx.canvas.height = video.videoHeight;
  ctx.drawImage(video, 0, 0);
}
</script>

You can also define a pattern of a video element with createPattern().

Summary

Phew, that was quite an article, no? On the plus side, I think I managed to cover the whole video and audio DOM API and all related events. Now I'm just waiting for you guys to do something creative with all of this. I haven't included any demos in this post — sorry — but I want you to create the demos and kick-ass sites and applications. There's lots of potential here of what can be done. Personally, I need to get back to working on QA...

If the examples in this article don't work in some browsers, then either I've made a typo or some other mistake, or there's a bug in the browser. Check the error console, the HTML5 <video> spec, and if you think there's a bug, file it in the relevant browser vendor's bug tracker. For Opera, you can use our Bug Report Wizard — include "video" or "audio" or "media" in the summary. Thanks!

Read more...

• Building a custom HTML5 video player with CSS3 and jQuery
• An HTML5 <audio> radio player

This article is licensed under a Creative Commons BSD License license.