Features

net.wars: Letter box

In case you thought the iPad was essentially a useless, if appealing, gadget, take heart: it now arguably has a reason to exist in the form of an app, iMean, designed to help autistic children communicate.

The back story: my friend Michael's son, Dan, is 14; his autism means he can't really speak and has motor control difficulties.

"He's somebody who at the age of 12 had a spoken vocabulary of 100 words," says Michael, "though he seemed to have a much greater recognition vocabulary and could understand most of what we said to him, though it was hard to be sure."

That year, 2008, the family went to Texas to consult Soma Mukhopadhyay, who over the space of four days was able to get Dan communicating through multiple-choice. At first, the choices were written on two pieces of paper and Dan would grab one. He rapidly moved on to using a pencil to point at large letters placed in alphabetical order on a piece of laminated cardboard, a process Michael compares to a series of multiple-choice questions with 26 possible answers.

"Before Soma there were no letters, only words. So what he came to realize was that all the words he knew and could recognize were all combinations of the same 26 letters," Michael says. "The letter board did for Dan what moveable type did for the Western world, but the difference is that before Gutenberg people could still write and Dan could not."

The need for a facilitator to keep Dan focused on the task of spelling out a sentence also raises the issue of ensuring that it's actually Dan who's communicating. Michael says, "I was always very concerned not to impose myself on Dan while helping him as much as possible." The iPad, therefore, offered the possibility of a more effective letter board that could incorporate predictive text and remember what's been said, and one whose other features might help Dan move on to more efficient – and more independent – communication. Dan's eyes jump so he may miss details in written text, but voiceover can read him email, and what he types into iMean can be copied into an answer. Performing all those steps independently is some way off, but the potential is life-changing.

Michael proposed the app he had in mind to 18-year-old programmer Richard Meade-Miller. "I didn't think it was going to be that hard because Apple has done most of it for you," says Michael, "but it turns out that to write an app you really need to be able to do programming in objective-C. For someone who learned Fortran 35 years ago, that's really difficult."

However, there were constraints. "We wanted the buttons to be as big as possible so Dan would have as little chance of error as possible." That forced some hard choices, such as limiting available punctuation marks to four, and shrinking the backspace button a little smaller than Michael had originally hoped in order to make room for Yes and No keys.

"When somebody like Dan sits down with this he may not be able to spell right away, but he needs to be able to say yes or no or say if something goes wrong on the screen. There should be a No button, bright red and very clear." Getting all that into the available screen space also meant creating a different view for numeric input, needed so Dan can do math problems and to speed entering large numbers.

The iPad's memory is also a constraint. "The program runs very quickly and smoothly, but anybody write an app for this platform has to be careful to release all the things that use memory on a regular basis." For the word prediction feature, iMean uses ZenTap, whose author supplied the code for Meade-Miller to integrate.

Word prediction – as Dan spells out words iMean offers him a changing display of three completed words to choose from – has speeded up the whole process for Dan. But it also, Michael says, has had a noticeable effect on his ability to read, "Because he's reading all day long." A final set of constraints are imposed by Dan's own abilities. Many autistic children do not point, an early developmental milestone. "Dan has started to point a little bit now as a result of tapping things on the letter board." Michael knew that, but he didn't realize how hard it would be for Dan, whose fingers sometimes shake and slip, to distinguish between tapping a key and swiping his fingers across a key – and a few keys are programmed to behave differently if they are swiped rather than tapped. "That may have been a mistake," he says. "It has forced Dan to really concentrate on tapping, so sometimes he double and triple taps.

Dan insisted on making a baseline video the first day so that later they can compare and see how much he's improved.

Their long-term goal is for Dan to be able to communicate with people independently. Whether they get all the way there or not, Michael says, "We know the app works the way we want. He can read a paragraph now instead of just a line – and it's only been three days."

Dan, by voice, is calling it his "stepping stone".