An HTML5-style "Google Suggest"

By Anne van Kesteren

Update, January 10th 2011: The ability to use an external source for a <datalist> has been removed from the HTML5 spec to reduce complexity. It can still be implemented using XMLHttpRequest to fetch data and then adding <option> elements dynamically, but we may see it return as a native HTML5 feature at some point in the future.
Minor update on 26th April 2010.

Introduction

HTML5 is the next major revision of HTML (and XHTML) and is being jointly developed by the WHATWG and W3C HTML WG (as such, it is a work in progress, but this article will simply refer to it as HTML5.) I have already covered the basics of HTML forms in my previous article, improving your forms using HTML5, so this article will look at some more advanced aspects of HTML5 data input constructs, culminating in an example that shows how to build an auto-complete style text box using nothing more than a short server-side script and a few lines of markup.

The features discussed in this article were part of the Web Forms 2 specification, which is now part of W3C's HTML5 specification. The feature runs on Opera 9.5 or later. To test it, grab the latest version of Opera.

Combo boxes (`input list`)

Let's start off with some background information - first of all, let's look at how HTML5 deals with combo boxes:

<input list="languages" name="language">
<datalist id="languages">
  <option value="Norwegian"></option>
  <option value="Dutch"></option>
  <option value="English"></option>
</datalist>

In older browsers this will degrade to a simple text field. In newer user agents supporting HTML5 however you will be able to select one of the predefined values in addition to being able to type something in yourself. This functionality is very similar to that offered in e-mail clients and the address bar of the browser for instance. If you need this functionality but your suggested options need to be presented in older browsers as well you might want to consider the following markup (this has some additional context added as well):


  Enter your front-end specialty:
  <input list="languages" name="language">

<datalist id="languages">
  
    or select one from the list:
    <select name="language">
      
      
      
      
    </select>
  
</datalist>

Browsers supporting the list attribute / datalist element construct of HTML5 will not display the datalist element and its contents, instead carefully extracting the option element values out of it to populate the combo box. Older browsers will render the contents of the datalist element and allow the user to use either the input field or select an option.

External source for datalist

Another interesting feature is that you can let the suggestions be populated from an external XML file. The external XML file looks something like the following (it needs to be served with an application/xml media type):

<select xmlns="http://www.w3.org/1999/xhtml">

The contents of this select element will then replace the contents of any datalist that references it unless the select element in the external file has its type attribute set to incremental in which case the contents will be appended to, and not replace, existing content. You can tell the user agent to populate a datalist element with an external file called foo using something like this:

<input list="languages" name="language">
<datalist id="languages" data="foo"></datalist>

(select elements have also been extended with a data attribute in HTML5, in case you were wondering.)

The source code for the above examples can be found in the example files that accompany this article, found here. In addition, a live demo is available at http://html5.org/demos/dev.opera.com/article-examples.html.

A dynamic combo box

So far we have looked at combo boxes and a way to populate them using an external file. All we need now to emulate Google Suggest in HTML5 is an event to listen to the combo box and a small script on the server to dynamically generate the file that will populate the datalist element. Using conventional techniques you would need to create your own "dropdown menu" listing the various options, use XMLHttpRequest to fetch the external data, and write code to parse this data to populate the menu - a lot of work, I'm sure you'll agree.

So what event can we use? Web Forms 2 introduces an event called input, which is already supported by several browsers, including Opera. The event dispatches after the user inputs some text using the keyboard. For rapid keyboard input (entering multiple characters) a single event is dispatched. Integrating the event in our combo box makes the code a tad more complicated:

<input list="suggest" name="q"
       oninput="list.data = '?w=' + encodeURIComponent(value)">
<datalist id="suggest"></datalist>

As you can see the input event handler manipulates list.data. The list attribute on the input element refers to the datalist element by its id, so it knows what it is populating. All we need to do to load the data from the specified location is manipulate the data attribute - the location is ?w plus the user input, which is encoded into a URI compatible syntax using the encodeURIComponent function (part of the global object). So if the user enters foo the URI fetched will be ?w=foo (resolved relative to the URI of the page the script runs on). A server-side file receives this URI, searches a text file full of possible words for this text string, and then returns an XML file containing the words that contain the text string, to populate the datalist. This all happens nice and dynamically, so as soon as you change the search term inside the text input, the server-side file processes the new search term and sends an updated XML file, changing the contents of the datalist element accordingly.

I've provided a working example of this for you to try out yourselves - download the files from here, or check out the live demo at http://html5.org/demos/dev.opera.com/article-example-suggest.py.

The files for this example are:

A newline-delimited file of suggestions called suggest.txt, which will be parsed when the script searches for the user-entered search term
A Python script called article-example-suggest.py, which parses the textfile looking for the user-entered search term, and then returns the XML search results. This file also contains the input and datalist elements discussed above

The full python code file looks like this:

import os
qs = os.environ["QUERY_STRING"]

# The page as shown by default
main="""Content-Type:text/html;charset=UTF-8\n
<!doctype html>
<html>
<head>Demo</head>
<body>
  <p>
    
      Please enter a word:
      <input list="suggest" name="q"
             oninput="list.data = '?w=' + encodeURIComponent(value)">
    
    <datalist id="suggest"></datalist>
  </p>
</body>
</html>"""

if qs=="":
    print main
else:
    # If a query string was provided we need to provide an XML file with
    # options filtered using the user input
    import sys
    print "Content-type: application/xml"
    print "Cache-control: no-cache"
    print ""
    sys.stdout.write('<select xmlns="http://www.w3.org/1999/xhtml">')
    sys.stdout.write('  ' % qs[2:])
    for name in open('suggest.txt').readlines():
        if name.lower().find(qs[2:].lower())!=-1:
            sys.stdout.write('' % name)
    sys.stdout.write('</select>')

Summary

I hope you enjoyed these examples! (Many thanks to Johannes Hoff (Core developer at Opera) for writing the above Python script after I hinted in a presentation that Google Suggest was only a few lines of code using HTML5 - turned out to be true both on the client and server.) They are not ready for mass production yet, but they do give you another interesting taste of what is to come with HTML5.

Anne van Kesteren

Anne van Kesteren works for Opera Software from his home in the Netherlands. He also travels a fair bit to talk about standards. His goal is to keep the Web an open place and fix all the bugs in Web standards.

This article is licensed under a Creative Commons Attribution, Non Commercial - Share Alike 2.5 license.