Keyboard accessibility in web applications: part 2
- Previous article — Keyboard accessibility in web applications: part 1
- Next article — Keyboard accessibility in web applications: part 3
Table of contents
Introduction
In the first part of the article we added keyboard shortcuts to a simple paint application. Now we will implement a way to move the pointer inside the drawing canvas using keyboard shortcuts.
At this point, the web application cannot be used entirely via the keyboard; it still requires a pointing device for the actual drawing. Users who really need keyboard-based drawing will most likely have mouse keys activated in their operating system of choice (check this page out for details of the Windows implementation, for example).
For educational and experimentation purposes, we can implement keyboard-based drawing inside our paint tool. Compared to a system-wide mouse keys implementation, this has the advantage of being tied directly to our web application, opening a range of possible improvements applicable only to our use-case.
To implement mouse keys in the paint application we have to add support for extending the application functionality from external code. We must make sure any extended functionality can be accessed from the keyboard shortcuts configuration file.
To make it easier to follow along with the code walkthrough
presented below, download the
full code example and follow along with it as you read the
article. The code discussed in this article is all found inside the app4
directory.
Extending the paint application
Let's begin by adding support for removable actions to the paint
application. Here is a snippet from the updated Painter
class:
function Painter () {
// ...
/**
* Holds all the removable functionality from the paint application.
*
* @type Object
* @see Painter#actionAdd Add a new action.
* @see Painter#actionRemove Remove an action.
*/
this.actions = {};
// ...
this.actionAdd = function (id, func, overwrite) {
if (typeof id != 'string' || typeof func != 'function' || (this.actions[id] && !overwrite)) {
return false;
}
this.actions[id] = new func(_self);
return this.actions[id] ? true : false;
};
// ...
this.actionRemove = function (id) {
if (!id || !_self.actions[id]) {
return false;
}
if (typeof _self.actions[id].actionRemove == 'function') {
_self.actions[id].actionRemove();
}
delete _self.actions[id];
return true;
};
// ...
};
The new code allows us to add new actions, which are in fact functions
that return an object holding any properties and methods you want. Any new
object is added to the Painter
actions with the given ID. When
the action is removed, the actionRemove()
method is called, if
the action implements it. This allows you to execute any custom
"uninstall" procedures you like.
There is not much we can do with that, for now. Let's tie things together
by allowing the keyboard shortcuts configuration to call methods from
installed actions. This can be done in the ev_keyhandler()
function:
// ...
var gkey = PainterConfig.keys[ev.kid_];
if (!gkey) {
return;
}
ev.kobj_ = gkey;
// Execute the associated action.
if (gkey.action) {
var action = _self.actions[gkey.action];
if (action) {
var func = action[ev.type];
if (typeof func == 'function') {
func(ev);
}
}
}
// ...
This update allows us to include an action
property for each
configured keyboard shortcut, just like we have the tool
property
for tool activation. When the user presses the key combination, the
method having the same name as the event type will be invoked, from the
action object.
With the above code in place, we can now do something really simple like this:
PainterInstance.actionAdd('test', function (app) {
// ...
this.keydown = function (ev) { ... };
this.keypress = function (ev) { ... };
// ...
});
Into the PainterConfig object we can add a keyboard shortcut that invokes the test action:
var PainterConfig = {
keys: {
T: { action: 'test', prop1: 'val1' }
}
};
From this code you can see that the key T is associated with
the action with an ID of test
. Now we've implemented the change on the
ev_keyhandler()
function, the keydown()
,
keypress()
and the keyup()
methods from the test
action are invoked when their respective events are fired. Additionally, by
adding the kobj_
property to the DOM Event object, any method
from the test action can access the keyboard shortcut configuration and any
of its properties. This allows us to pass custom parameters to the action
method.
Mouse keys implementation
To implement mouse keys support, we need to always store the last mouse location in an object, to allow keyboard-based movements to continue from the same place.
During typical use, the event flow involves several
mousemove
events to the desired position, then
mousedown
to start drawing, then several mousemove
events until the destination is reached, and finally a mouseup
event for ending the drawing operation. We need to store the state of the
mouse button - if it is down or not. The mouse button is considered to be
down/active between a mousedown
and a mouseup
event. This is needed by a mouse keys implementation in order to determine
and synthetically alter the state of the mouse button and generate any
required mouse events when the user presses a key.
Here are the updates needed in the ev_canvas()
function:
function Painter () {
// ...
/**
* Holds the last recorded mouse coordinates and the button state (if it's
* down or not).
*
* @private
* @type Object
*/
this.mouse = {x: 0, y: 0, buttonDown: false};
// ...
function ev_canvas (ev) {
// ...
/*
* If the mouse is down already, skip the event.
* This is needed to allow the user to go out of the drawing canvas, release
* the mouse button, then come back and click to end the drawing operation.
* Additionally, this is needed to allow extensions like mouse keys to
* perform their actions during a drawing operation, even when a real mouse
* is used. For example, allow the user to start drawing with the keyboard
* (press 0) then use the real mouse to move and click to end the drawing
* operation.
*/
if (_self.mouse.buttonDown && ev.type == 'mousedown') {
return false;
}
// Don't overwrite any existing x_ / y_ property.
// These properties might be added by other functions.
if (typeof ev.x_ == 'undefined') {
if (typeof ev.layerX != 'undefined') { // Firefox
ev.x_ = ev.layerX;
ev.y_ = ev.layerY;
} else if (typeof ev.offsetX != 'undefined') { // Opera
ev.x_ = ev.offsetX;
ev.y_ = ev.offsetY;
}
// Update the current mouse position only for mouse events.
// Other events do not provide accurate mouse coordinates.
switch (ev.type) {
case 'mousedown':
case 'mousemove':
case 'mouseup':
_self.mouse.x = ev.x_;
_self.mouse.y = ev.y_;
}
}
if (ev.type == 'mousedown') {
_self.mouse.buttonDown = true;
}
// Call the event handler of the active tool.
var func = _self.tool[ev.type];
if (typeof func != 'function') {
return false;
}
res = func(ev);
if (ev.type == 'mouseup') {
_self.mouse.buttonDown = false;
}
};
// ...
};
Here we've added a new property object to the Painter
class
- the mouse
property - to hold information about the mouse
coordinates and the button state. The updated ev_canvas()
function tries to determine the mouse coordinates if the DOM Event object
does not already have the two x_
and y_
properties.
Both properties will be added by the mouse keys implementation. The new
ev_canvas()
function updates the mouse coordinates and the
button state, depending on each event type.
The above completes the updates necessary to the main code - check out the fourth index.js update.
In the current tools, we have set an inner variable named started
to true on mousedown
, to track if the mouse button is being
clicked, so their mousemove
event handler can perform the
drawing operation. We can drop that now, and reuse the new
mouse.buttonDown property from the application object. Check out
the minor changes in the fourth tools.js
update.
The Painter
API provides us with sufficient pointer
information, and an extension mechanism for adding custom keyboard shortcuts
- we can now start implementing mouse keys support. Let's create a new mousekeys.js file, which is added into the index.html
file (to see all of these in action, fire up index.html
from the app4 folder in the code download). Here is the basic code structure for
mouse keys:
/**
* @class The MouseKeys action.
*
* @param {Painter} app Reference to the main paint application object.
*/
function PaintMouseKeys (app) {
var canvas = app.buffer.canvas,
mouse = app.mouse;
/**
* Holds the current mouse movement speed in pixels.
*
* @private
* @type Number
*/
var speed = 1;
/**
* Holds the current mouse movement acceleration, taken from the
* configuration.
*
* @private
* @type Number
* @see PainterConfig.mousekeys_accel The mouse keys acceleration setting.
*/
var accel = PainterConfig.mousekeys_accel;
if (!canvas || !canvas.parentNode) {
return false;
}
/**
* Holds a reference to the DOM element representing the pointer on top of the
* canvas element.
*
* @private
* @type Element
*/
var pointer = document.createElement('div');
if (!pointer) {
return false;
}
pointer.id = 'mousekeysPointer';
pointer.style.display = 'none';
canvas.parentNode.appendChild(pointer);
function pointerMousemove (ev) { ... };
function dispatch (type, ev) { ... };
this.keydown = function (ev) { ... };
this.keypress = function (ev) { ... };
this.actionRemove = function () { ... };
canvas.addEventListener('mousemove', pointerMousemove, false);
};
The PaintMouseKeys
class object holds the main methods:
keydown()
, keypress()
and
actionRemove()
. The two private variables, speed
and
accel
, hold information about the speed and the acceleration of
the mouse pointer. The longer the user holds down a key, the faster the
pointer will move. While a key is being held down the speed
value
increases based on the accel
value.
The pointer
DOM element is added in the parent node of the
<canvas>
element because it is needed for providing the
user with a visual indicator of where the pointer is moving. Since this
script cannot move the real mouse pointer, which might not even exist, we
need our own virtual pointer. The pointerMousemove()
updates the
position of the pointer
element when the mouse moves:
/**
* Track the virtual pointer coordinates, by updating the position of the
* pointer
element. This allows the keyboard users to see where
* they moved the virtual pointer.
*
* @param {Event} ev The DOM Event object.
*/
function pointerMousemove (ev) {
if (typeof ev.x_ == 'undefined' || !ev.kobj_ || !ev.kobj_.action || ev.kobj_.action != 'mousekeys') {
if (pointer.style.display == 'block') {
pointer.style.display = 'none';
}
return;
}
pointer.style.top = ev.y_ + 'px';
pointer.style.left = ev.x_ + 'px';
};
The function above will only update the pointer location if the
mousemove
event contains the x_
and y_
coordinates, and only if the event was generated by the mousekeys
action.
If the event is not a synthetic one, the pointer
element is
hidden. We only want it visible when the keyboard is used.
You need to decide how you want the users to interact with the drawing canvas using their keyboard. I decided for this demo to use the number keys. Therefore, key 2 should move the cursor to the south (towards the bottom of the image), key 4 to the west (left), key 6 to the east (right), and key 8 should move the pointer towards the north (top of the image). The rest of the keys, 1, 3, 7 and 9, should move the pointer in diagonal directions (eg 1 for south-west). If the user also holds down the Shift key in addition to a direction, the mouse pointer moves faster.
Having defined what we want for the pointer movement (the
mousemove
event) we need a key to generate the
mousedown
and the mouseup
events. For this demo,
when the user presses the 0 key the synthetic
mousedown
event is dispatched. Pressing it again would generate
the mouseup
event. As such, this key allows the user to toggle
the "mouse is down" state to true or false, by alternating the two events.
When the mouse is down, any mousemove
event will perform the
drawing operation associated with the active tool. The drawing tools do not
need any code change since the mouse events are all fired synthetically.
Configuration file update
Another important technical point is that we do not want to modify the
script when we change the keyboard shortcuts. Here is the
modification for the PainterConfig
object:
var PainterConfig = {
/**
* The mouse keys movement acceleration.
*
* @type Number
* @see PaintMouseKeys The MouseKeys extension.
*/
mousekeys_accel: 0.1,
/**
* Keyboard shortcuts associated to drawing tools and other actions.
*
* @type Object
* @see PaintTools The object holding all the drawing tools.
*/
keys: {
0: { action: 'mousekeys', param: 'Toggle' },
1: { action: 'mousekeys', param: 'SouthWest' },
2: { action: 'mousekeys', param: 'South' },
3: { action: 'mousekeys', param: 'SouthEast' },
4: { action: 'mousekeys', param: 'West' },
6: { action: 'mousekeys', param: 'East' },
7: { action: 'mousekeys', param: 'NorthWest' },
8: { action: 'mousekeys', param: 'North' },
9: { action: 'mousekeys', param: 'NorthEast' },
L: { tool: 'line' },
P: { tool: 'pencil' },
R: { tool: 'rect' }
}
};
We have added the virtual pointer acceleration we want, and the new
keyboard shortcuts for the mousekeys
action. The param
property
specifies the action what to do. This is where adding the kobj_
property to the DOM Event object in the ev_keyhandler()
becomes
useful. The keydown()
and keypress()
methods from
the mouse keys implementation can use the param
property to
determine the mouse movement direction associated to the keyboard shortcut.
As such, we do not include any of the keyboard shortcuts at all in the
implementation, only in the configuration file.
When the Shift key is held down, the system generates different keys for the numbers on the number pad. We need to take them into consideration in the keyboard shortcuts configuration:
lib.extend(PainterConfig.keys, {
'Shift Insert': PainterConfig.keys['0'],
'Shift End': PainterConfig.keys['1'],
'Shift Down': PainterConfig.keys['2'],
'Shift PageDown': PainterConfig.keys['3'],
'Shift Left': PainterConfig.keys['4'],
'Shift Right': PainterConfig.keys['6'],
'Shift Home': PainterConfig.keys['7'],
'Shift Up': PainterConfig.keys['8'],
'Shift PageUp': PainterConfig.keys['9']
});
We use the extend()
function to add the new keys, as
duplicates of the numbers. With this we have completed the update needed for
the configuration file.
The event handlers
Now we will continue with the MouseKeys action implementation. Here is
the code for the keydown()
event handler:
this.keydown = function (ev) {
speed = 1;
accel = PainterConfig.mousekeys_accel;
if (pointer.style.display == 'none') {
pointer.style.display = 'block';
pointer.style.top = mouse.y + 'px';
pointer.style.left = mouse.x + 'px';
}
if (!ev || !ev.kobj_ || ev.kobj_.param != 'Toggle') {
return false;
}
var type = mouse.buttonDown ? 'mouseup' : 'mousedown';
dispatch(type, ev);
return true;
};
The keydown()
method always resets the speed and the mouse
acceleration to the default values. This ensures that the speed gained from
the previous key press is not reused in another key press for mouse
movement. Otherwise, the mouse movement would become too fast quite
quickly.
The pointer
element is always made visible, such that the user
can see where the virtual pointer is located when he/she starts using the
keyboard for drawing.
The kobj_
property object is used to determine which action
needs to be performed. If the keyboard shortcut object holds the
param
property with the value Toggle
, we perform the
action of the key 0
, as discussed above. We dispatch
a mouseup
event if the mouse button is down, otherwise we
dispatch the mousedown
event. The mouse.buttonDown
boolean property is the one being updated by the ev_canvas()
function as previously described.
The implementation for the rest of the number keys used for dispatching
synthetic mousemove
events is held in the
keypress()
event handler in the mousekeys
action:
this.keypress = function (ev) {
if (!ev || !ev.kobj_ || !ev.kobj_.param) {
return false;
}
if (ev.shiftKey) {
speed += speed * accel * 3;
} else {
speed += speed * accel;
}
var w = canvas.width,
h = canvas.height,
x = mouse.x,
y = mouse.y,
step = Math.ceil(speed);
switch (ev.kobj_.param) {
case 'SouthWest':
x -= step;
y += step;
break;
case 'South':
y += step;
break;
case 'SouthEast':
x += step;
y += step;
break;
case 'West':
x -= step;
break;
case 'East':
x += step;
break;
case 'NorthWest':
x -= step;
y -= step;
break;
case 'North':
y -= step;
break;
case 'NorthEast':
x += step;
y -= step;
break;
default:
return false;
}
if (x < 0) {
x = 0;
} else if (x > w) {
x = w;
}
if (y < 0) {
y = 0;
} else if (y > h) {
y = h;
}
mouse.x = x;
mouse.y = y;
dispatch('mousemove', ev);
return true;
};
The implementation for the movement keys has been put in the
keypress()
event handler because this event is always repeated
while the user holds down a key. The keydown
event is
dispatched only once.
The code is rather simple: increase the movement speed based on the
acceleration (with a variation if the Shift key is down), and
update the mouse coordinates based on the param
property of the
keyboard shortcut object. At the end, the synthetic mousemove
event is dispatched. The mouse coordinates are those automatically updated
by the ev_canvas()
function.
Here is the code that performs the actual synthetic event dispatching:
function dispatch (type, ev) {
var ev_new = document.createEvent('MouseEvents');
ev_new.initMouseEvent(type,
ev.bubbles, ev.cancelable,
ev.view, 0,
0, 0,
0, 0,
ev.ctrlKey, ev.altKey,
ev.shiftKey, ev.metaKey,
0, ev.relatedTarget);
// Make sure the new coordinates are passed to the event handlers.
ev_new.x_ = mouse.x;
ev_new.y_ = mouse.y;
// Make sure the event handlers can check this is a synthetic event.
// This is needed by the pointerMousemove() function.
ev_new.keyCode_ = ev.keyCode_;
ev_new.key_ = ev.key_;
ev_new.kid_ = ev.kid_;
ev_new.kobj_ = ev.kobj_;
canvas.dispatchEvent(ev_new);
};
This function creates a new mouse event of the given type
. The
new mouse event will share several properties (eg the active key modifiers)
with the current keyboard event. Several new properties are added: the new
mouse coordinates and the keyboard-related properties. The new event is
dispatched to the buffer <canvas>
element.
When the real mouse is used, typically the following execution chain is activated:
- The mouse event is fired by the browser and the
ev_canvas()
event handler is invoked, which determines the mouse coordinates on the<canvas>
element and updates the mouse object to hold them and the button state - The event handler of the active tool is invoked to perform any drawing operations
When the mouse keys are used, typically the following execution chain is activated:
- The keyboard event is fired by the browser and handled by the
KeyboardEventListener
class instance from the minimal JavaScript library. - That code invokes the
ev_keyhandler()
function from the paint application, which in turn invokes the event handler within themousekeys
action. - The event handlers in the mouse keys implementation, as explained, also
dispatch a synthetic mouse event, which in turn is handled by the
ev_canvas()
function and by the active tool. - For synthetic
mousemove
events, thepointerMousemove()
function updates the location of the pointer element.
To wrap it all up, we include the actionRemove()
method and
perform the actual addition of the action in the Painter
instance:
function PaintMouseKeys (app) {
// ...
/**
* Handles action removal. This will remove the pointer DOM element and the
* canvas event listener.
*/
this.actionRemove = function () {
canvas.parentNode.removeChild(pointer);
canvas.removeEventListener('mousemove', pointerMousemove, false);
};
};
window.addEventListener('load', function () {
// Add the MouseKeys action to the Painter instance.
if (window.PainterInstance) {
PainterInstance.actionAdd('mousekeys', PaintMouseKeys);
}
}, false);
That's all! Go ahead and test the updated
application inside the app4
folder - and the documentation, found in app4/doc/
- in the code download.
In the next part
In the third and final part of the article we will look into the cross-browser compatibility layer we use for dealing with the browser differences in the DOM keyboard events handling.
If you want to learn more, stick around at DevOpera and also follow the upcoming major changes to PaintWeb - the open-source web application this article is based on.
- Previous article — Keyboard accessibility in web applications: part 1
- Next article — Keyboard accessibility in web applications: part 3
This article is licensed under a Creative Commons Attribution, Non Commercial - Share Alike 2.5 license.
Comments
The forum archive of this article is still available on My Opera.