Introduction to XMLHttpRequest Level 2

By Tiffany Brown

Introduction

XMLHttpRequest allows developers to make HTTP and HTTPS requests and modify the current page without reloading it. Submitting forms and retrieving additional content are two common uses.

Early forms of XMLHttpRequest limited requests to text, HTML and XML. Sending variables and values required syntax — URL-encoded strings — that were messy to read and write.

Early XHR was also subject to a same-origin policy, that made cross-domain requests more difficult. You couldn't, for instance, share data between http://foo.example/ and http://bar.example/ without an intermediary such as Flash-enabled XHR or a proxy server. Sharing data between subdomains (e.g. http://foo.example and http://www.foo.example) required setting a document.domain in scripts on both origins. Doing so, however carried security risks.

Uploading files with earlier implementations of XHR? No can do. Instead we had to rely on workarounds such as SWFUpload, which required a plugin. Or we had to use a hidden iframe, which lacked client-side progress events.

We can do better than that, and we have. This article looks at improvements to XMLHttpRequest, and the state of support in Opera 12.

What's new in XMLHttpRequest

With changes to the XMLHttpRequest specification and improved browser support, you can now:

Setting and handling timeouts

Sometimes requests are slow to complete. This may be due to high latency in the network or a slow server response. Slow requests will make your application appear unresponsive, which is not good for the user experience.

XHR now provides a way for handling this problem: request timeouts. Using the timeout attribute, we can specify how many milliseconds to wait before the application does something else. In the example that follows, we've set a three second (3000 millisecond) timeout:

function makeRequest() {
  var url = 'data.json';
  var onLoadHandler = function(event){
     // Parse the JSON and build a list.
  }
  var onTimeOutHandler = function(event){
    var content = document.getElementById('content'),
      p = document.createElement('p'),
      msg = document.createTextNode('Just a little bit longer!');
      p.appendChild(msg);
      content.appendChild(p);

      // Restarts the request.
      event.target.open('GET',url);

      // Optionally, set a longer timeout to override the original.
      event.target.timeout = 6000;
      event.target.send();
  }
  var xhr = new XMLHttpRequest();
  xhr.open('GET',url);
  xhr.timeout = 3000;
  xhr.onload = onLoadHandler;
  xhr.ontimeout = onTimeOutHandler;
  xhr.send();
}

window.addEventListener('DOMContentLoaded', makeRequest, false);

If more than three seconds pass before response data is received, we'll notify the user that the request is taking too long. Then we'll initiate a new request with a longer timeout limit (view an XHR timeouts demo). Resetting the timeout limit within the timeout event handler isn't strictly necessary. We've done so for this URL to avoid a loop since its response will always exceed the initial timeout value.

To date, Chrome and Safari do not support XHR timeouts. Opera, Firefox, and Internet Explorer 10 do. Internet Explorer 8 and 9 also support timeouts on the XDomainRequest object.

Requesting data from another domain

One limitation of early XHR was the same-origin policy. Both the requesting document and the requested document had to originate from the same scheme, host, and port. A request from http://www.foo.example to http://www.foo.example:5050 — a cross-port request — would cause a security exception (except in older versions of Internet Explorer, which allowed cross-port requests).

Now XMLHttpRequest supports cross-origin requests, provided cross-origin resource sharing (CORS) is enabled.

Internet Explorer 8 and 9 do not support cross-domain XMLHttpRequest, though IE10 does. Instead, Internet Explorer 8 and 9 use the XDomainRequest object, which works similarly.

Cross-origin requests look just like same-origin requests, but use a full URL instead of a relative one:

var xhr = new XMLHttpRequest();
var onLoadHandler = function(event) {
  /* do something with the response */
}
xhr.open('GET','http://other.server/and/path/to/script');
xhr.onload = onLoadHandler;
xhr.send();

The critical difference is that the target URL must permit access from the requesting origin by sending an Access-Control-Allow-Origin response header.

An overview of cross-origin resource sharing

For an in-depth look at CORS, read DOM access control using cross-origin resource sharing. Here we'll just cover two headers: the Origin request header, and the Access-Control-Allow-Origin response header.

The origin header

When making a cross-origin XHR request, Opera and other browsers will automatically include an Origin header — see below for an example:

GET /data.xml HTTP/1.1
User-Agent: Opera/9.80 (Macintosh; Intel Mac OS X 10.6.8; U; en) Presto/2.10.289 Version/12.00
Host: datahost.example
Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/webp, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1
Accept-Language: en,en-US
Accept-Encoding: gzip, deflate
Referer: http://requestingserver.example/path/to/askingdocument.html
Connection: Keep-Alive
Origin: http://requestingserver.example

Origin typically contains the scheme, host name, and port of the requesting document. It is not an author request header, meaning it can’t be set or modified using the setRequestHeader() method; user agents will ignore it if you try. Its entire purpose is to inform the target server about the origins of this request. Bear in mind that there is no trailing slash.

The Access-Control-Allow-Origin response header

The Access-Control-Allow-Origin header is sent by the target server in response to a cross-origin request. It tells the user agent whether access should be granted to the requesting origin. DOM operations involving a cross-origin XHR request will not be completed unless the requested URL allows it. An example follows:

HTTP/1.1 200 OK
Date: Fri, 27 May 2011 21:27:14 GMT
Server: Apache/2
Last-Modified: Fri, 27 May 2011 19:29:00 GMT
Accept-Ranges: bytes
Content-Length: 1830
Keep-Alive: timeout=15, max=97
Connection: Keep-Alive
Content-Type:  application/xml; charset=UTF-8
Access-Control-Allow-Origin: *

In this case, we’re using a wild card (*) to allow access from any origin. This is fine if you are offering a public-facing API. For most other uses, you'll want to set a more specific origin value.

Sending user credentials with cross-domain requests

There may be occasions when you will want to send cookie data along with your cross-domain request. That’s where the withCredentials attribute comes in handy. It is a boolean attribute that alerts the browser that it should send user credentials along with the request. By default, the credentials flag is false. In the code below, let’s assume that our request is going from http://foo.example to http://other.server:

var xhr = new XMLHttpRequest();
var onLoadHandler = function(event) {
  doSomething(event.target.responseText);
}
xhr.open('GET','http://other.server/and/path/to/script');
xhr.withCredentials = true;
xhr.onload = onLoadHandler;
xhr.send();

In our XHR credentials demo, we are using a counter cookie to track the number of visits. If you examine the request and response data (you can do this with Dragonfly' Network panel), you will see that the browser is sending request cookies and receiving response cookies. Our server-side script will return text containing the new visit count, and update the value of the cookie.

Keep the following in mind when making requests with credentials:

  • withCredentials is only necessary for cross-origin requests.
  • The Access-Control-Allow-Origin header of the requested URI can not contain a wildcard (*).
  • The Access-Control-Allow-Credentials header of the requested URI must be set to true.
  • Only a subset of response headers will be available to getAllRequestHeaders(), unless the Access-Control-Expose-Headers header has been set.

Same-origin requests will ignore the credentials flag. A wildcard Access-Control-Allow-Origin header value will cause an exception. If the value of Access-Control-Allow-Credentials is false, cookies will still be sent and received, however they will not be available to the DOM.

Sending data as key-value pairs with FormData objects

In previous implementations, data sent via XHR had to be submitted as a string, either using URL-encoding, or JSON (with JSON.stringify()). The example below uses URL-encoding:

var msg = 'field1=foo&field2=bar';
var xhr = new XMLHttpRequest();
xhr.open('POST','/processing_script');
xhr.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
xhr.send(msg);

Now, we can use the FormData object, and its ordered, key-value pairs. The syntax offers three benefits:

  • Scripts are more readable
  • Data is sent in key-value pairs, as with regular HTML forms
  • FormData objects are sent with multipart/form-data encoding, making it possible to use XHR for sending binary data.

If you've ever worked with URLVariables in ActionScript 3.0, FormData will feel familiar. First create a FormData object, then add data using the append() method. The append() method requires two parameters: key and value. Each FormData key becomes a variable name available to your server-side script. An example follows:

var xhr = new XMLHttpRequest();
var dataToSend = new FormData(); // create a new FormData object

xhr.open('POST','/processing_script');

dataToSend.append('name','Joseph Q. Public'); // add data to the object
dataToSend.append('age','52');
dataToSend.append('hobby','knitting');

xhr.send(dataToSend); // send the object

We’ve passed the FormData object as the argument of the send() method: xhr.send(dataToSend). We did not set a Content-Type header on our XMLHttpRequest object. Let's take a look at the request headers sent by Opera:

POST /processing_script HTTP/1.1
User-Agent: Opera/9.80 (Macintosh; Intel Mac OS X 10.6.8) Presto/2.10.289 Version/12.00
Host: datahost.example
Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/webp, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1
Accept-Language: en,en-US
Accept-Encoding: gzip, deflate
Expect: 100-continue
Referer: http://datahost.example/upload/
Connection: Keep-Alive
Content-Length: 4281507
Content-Type: multipart/form-data; boundary=----------J2GMKTyAkjRjNgFzKv3VBJ

Opera has added the Content-Type header for us because we are using a FormData object. Other browsers do the same.

Using FormData with an HTML form

You can also send the values from a form with FormData, by passing the form to the FormData object as shown below (view an XHR FormData demo).

var submitHandler = function(event) {
  var dataToSend = new FormData(event.target), xhr = new XMLHttpRequest();
  xhr.open('POST','/processing_script');
  xhr.send(dataToSend);
}

var form = document.getElementById('myform');

form.addEventListener('submit',submitHandler,false);

FormData is still untrusted data. Treat input from a FormData object as you would any other kind of form submission.

Monitoring data transfers with progress events

XMLHttpRequest now provides progress event attributes that allow us to monitor data transfers. Previously, we would listen to the readystatechange event, as in the example below:

var xhr = new XMLHttpRequest();
var onReadyStateHandler = function(event) {
  if( event.target.readyState == 4 && event.target.status == 200){
    /* handle the response */
  }
}
xhr.open('GET','/path_to_data');
xhr.onreadystatechange = onReadyStateHandler;
xhr.send();

Though it works well for alerting us that all of our data has downloaded, readystatechange doesn’t tell us anything about how much data has been received. For backward compatibility, it remains a part of the specification. The ProgressEvent interface, however, is far more robust. It adds seven events that are available to both the XMLHttpRequest and the XMLHttpRequestUpload objects.

The different XMLHttpRequest Progress Events are as follows:

attribute type Explanation
onloadstart loadstart When the request starts.
onprogress progress While loading and sending data.
onabort abort When the request has been aborted, either by invoking the abort() method or navigating away from the page.
onerror error When the request has failed.
onload load When the request has successfully completed.
ontimeout timeout When the author specified timeout has passed before the request could complete.
onloadend loadend When the request has completed, regardless of whether or not it was successful.

ProgressEvent inherits from the DOM, Level 2 EventTarget interface so we can either use event attributes such as onload, or the addEventListener method in our code. In the examples above, we've used event attributes. In our next example, we’ll use addEventListener.

Monitoring uploads

All XMLHttpRequest-based file uploads create an XMLHttpRequestUpload object, which we can reference with the upload attribute of XMLHttpRequest. To monitor upload progress, we’ll need to listen for events on the XMLHttpRequestUpload object.

In the code below, we’re listening for the progress, load and error events:

var onProgressHandler = function(event) {
  if(event.lengthComputable) {
    var howmuch = (event.loaded / event.total) * 100;
        document.querySelector('progress').value = Math.ceil(howmuch);
  } else {
    console.log("Can't determine the size of the file.");
  }
}

var onLoadHandler = function() {
  displayLoadedMessage();
}

var onErrorHandler = function() {
  displayErrorMesssage();
}

xhr.upload.addEventListener('progress', onProgressHandler, false);
xhr.upload.addEventListener('load', onLoadHandler, false);
xhr.upload.addEventListener('error', onErrorHandler, false);

Pay special attention to the lengthComputable, loaded and total properties used in the onProgressHandler function. Each of these are properties of the progress event object. The lengthComputable property reveals whether or not the browser can detect the input file size, while loaded and total reveal how many bytes have been uploaded and the total size of the file. You can view an XHR progress events demo.

These events only monitor the browser’s progress in sending data to or receiving data from the server. When uploading, you may experience a lag between when the load event is fired and when the server sends a response. How long of a lag will depend on the size of the file, the server’s resources, and network speeds.

In the example above, we’re setting event listeners on the XMLHttpRequestUpload object. To monitor file downloads, add event listeners to the XMLHttpRequest object instead.

Enforcing a response MIME type

MIME-type mismatches are pretty common on the web. Sometimes XML data will have a Content-type: text/html response header, which will cause the value of xhr.responseXML to be null.

To ensure that the browser handles such responses in the way we’d like, we can use the overrideMimeType() method. In the example below, data.xml returns the following response headers:

Date: Sat, 04 Jun 2011 03:11:31 GMT
Server: Apache/2.2.17
Access-Control-Allow-Origin: *
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=UTF-8

That’s the wrong content type for an XML document. So let’s guarantee that the browser treats data.xml as XML, and populates the responseXML attribute:

var xhr = new XMLHttpRequest();
xhr.open('GET','data.xml');
xhr.overrideMimeType('application/xml');
xhr.send();
xhr.addEventListener('load', function(event) {
  console.log( event.target.responseXML );
}, false);

Now the value of xhr.responseXML is a Document object, which means we can parse the data as we would any other XML document. View an XHR override MIME type demo.

Enforcing a response type

It's also possible to tell the browser to handle a response as text, json, an arraybuffer, a blob or and document using the responseType property.

As with overrideMimeType, the responseType property must be set before the request is sent. In the example below, we are telling Opera to treat the response as a document, and write the firstChild to the console (view an enforcing response type demo):

var xhr = new XMLHttpRequest();
xhr.open('GET','data.xml');
xhr.responseType = 'document';
xhr.send();

xhr.addEventListener('load', function(e) {
  console.log( event.target.response.firstChild );
} false);

Though responseType allows developers to, say, handle image data as a byte array instead of a binary string, it does not work miracles. Changing document to json in the example above would cause our response property to be null because XML is not JSON. Similarly, invalid JSON data will also cause response to be null. When setting a responseType, you still need to ensure that your data is both valid and compatible with the specified type.

Note: As of publication, Opera does not support blob as a value, and only supports XML and not HTML for the document type. Chrome and Safari do not yet support json as a value.

Learn more

These XMLHttpRequest improvements are a leap forward for client-side interactivity. For more on XMLHttpRequest, CORS, and related APIs, see the following resources:

Note: Cover image — Ajax and Achilles Gaming — by Sharon Mollerus.

Tiffany B. Brown is a freelance web developer based in Los Angeles.


This article is licensed under a Creative Commons Attribution 3.0 Unported license.

Comments

  • photo

    Martin Kadlec

    Wednesday, August 29, 2012

    I've discovered reproducible crash with XHR2. Steps to reproduce:
    - Open: ../xhr2/xhr-responsetype.html
    - Press CTRL+U to display source code
    - Replace content of onLoadHandler to:
    function(event){
    var xhr = event.target,
    alert(xhr.response);
    }

    - Press "Apply Changes"
    - BAM! crash :(

    I sent a crash report (email: harou2 at gmail . com)
  • photo

    Chris Mills

    Wednesday, August 29, 2012

    Nice one Martin - we'll look at it asap.
  • photo

    Martin Kadlec

    Monday, September 3, 2012

    It seems there are more issues with the "document" response type in Opera. Firstly, html5test.com does not detect it. In my own tests it often returns just "null" even if I load valid document.
  • photo

    mechonomics

    Saturday, September 8, 2012

    Opera (12.02) seems to report values ~ 180% too high for progressEvent.loaded.

    Try this and see what I mean:

    var xhr = new XMLHttpRequest();
    var url = YOUR_URL + Math.random();

    xhr.addEventListener("progress", function(evt) {
    console.log("Download progress event: " + evt.loaded + " Total: " + evt.total + " Percentage: " + evt.loaded/evt.total*100);
    });

    xhr.open("get", url);
    xhr.send();
  • photo

    Егор

    Sunday, September 9, 2012

    Does Opera support only XML in xhr.responseType = 'document' ?
    - Open: ../xhr2/xhr-responsetype.html
    - Run in console:
    xhr = new XMLHttpRequest();
    xhr.open("GET","http://devfiles.myopera.com/articles/9482/xhr-responsetype.html", true);
    xhr.responseType = "document";
    xhr.onload = function(e){ console.log(e.type, "loaded: " + e.loaded, "response: ", this.response) };
    xhr.send();

    and got
    > load, loaded: 3654, response: , null

    This is really oddly. Chrome and FF supported html files in xhr.response. And even worse Opera raise exception when I try to access to xhr.responseText
  • photo

    Tiffany Brown

    Tuesday, September 18, 2012

    Apologies that we've taken so long to respond to your question Erop. Yes, that is currently the case. If you are sending HTML, however, you don't really need to set the responseType property. That's how XHR has typically worked.

    I have, however, filed a bug report for that, since it could become a site-compatibility issue. Thank you for mentioning it.
  • photo

    mechonomics

    Wednesday, September 19, 2012

    So, any response as to why XMLHttpRequest progressevent loaded values are far too high? It's possible for a progress event loaded to be larger than a progress event total, which should never be the case.
  • photo

    Martin Kadlec

    Monday, September 24, 2012

    mechonomics: I believe it is caused by wrong value in Content-length header. I'm not sure if other browser has some kind of algorithm to prevent the problem.
  • photo

    mechonomics

    Wednesday, September 26, 2012

    Martin, I don't think that's the problem because progressevent.total still reports the correct value.
  • photo

    Bozzeye

    Saturday, October 6, 2012

    When I try to run the code in the article in an extension's injected script, I get this from Dragonfly:

    Unhandled Error: Undefined variable: XMLHttpRequest
  • photo

    mehuge

    Wednesday, February 6, 2013

    In your timeouts demo, the version with timeout support is arguably a better experience for the user because the data arrives in 5 seconds instead of 8.

    Aborting and retrying is undesirable because if data was arriving just taking a long time, then you have just thrown away your hard work, all just so you can guess the right length of timeout to use.

    What you need to do is keep increasing the timeout whilst data is arriving, keeping it just far enough ahead to allow for a slow connection whilst at the same time detecting a failure condition.

    http://code.google.com/p/xhr-progressive-timeouts/ does just that, it maintains a short timeout but allows long requests to complete without interrupting them, so no wasted processing.
  • photo

    rvmaksim

    Monday, March 18, 2013

    Good day!
    Have a problem with HTTPS and upload.onprogress event. When we upload file throught https event progress doesn't fire.

    You can confirm this replace in demo(../xhr2/xhr-progressevents.html) http on https:
    >>> var makeHttpsRequest = eval(makeRequest.toString().replace(/http/g,'https'));
    >>> document.querySelector('form').removeEventListener('submit', makeRequest, false);
    >>> document.querySelector('form').addEventListener('submit', makeHttpsRequest, false);

    Is there a solution to this problem?
    Thanks!
No new comments accepted.