User Interface, Borgification and Compromising on a CrossoverPoint

Chris Johnson writes: "When you think about PalmPilots or the new crop of wearable computers or, perhaps, even computers themselves, you could sum it up with a single nonword - Borgification. By this, I refer to the way computers can extend a person's abilities, in the same way that wearing clothing can extend a person's ability to survive hostile climates, or standing on ladders can extend a person's ability to reach high-up things. This is a central theme of computer use, and the development of GUI speaks eloquently toward how far a computer's interactions can be bent to make them look like human interactions." Keep reading (below). Chris has lots more to say, and it's all good. He'll get you thinking, whether you agree with him or not.

There's a huge amount of effort being made to stretch computers in this manner. In the Linux field alone, there are lots of window managers and entire competing desktop environments. Looking at the commercial OSes, it turns out that the latest fad (starting with IBM's RealThings such as RealPhone, continued through Apple's Quicktime 4, and now taken up by Microsoft Media Player) is for simulated object interfaces- trading off access to consistent interface behavior for access to interfaces that are closely modelled on real physical objects. This includes the conscious decision to model a volume control (for instance) after a thumbwheel, despite the fact that the control is not used with your thumb and is suboptimal for GUI use compared to controls like sliders and scrollbars. The idea is that people are so accustomed to altering volume on the thumbwheel controls of radios and Walkmen that this overrides the need to optimise controls for the computer screen environment... and for better or worse, the commercial mainstream is diving in this direction.

My personal suspicion is that this may have been a 'Tar Baby'-like strategy on the part of Steve Jobs, who had to know that Microsoft Media Player would follow Quicktime down that road. That's as may be. The more interesting concern for Slashdot readers is this: should this industry trend prove a mistake, how can Linux and alternative operating systems seize an advantage?

It's interesting that the assumptions about user interface always seem to mean the computers will do all the 'stretching' toward the humans. For instance, on the one hand, J. Random Grandmother is the last person you'd expect to be wearing Borg implants, yet the direction of wearables seems always to be GUI displays so the person needn't hack with weird commands, voice recognition so the person needn't _type_, etc etc. and so on. This is an incongruous combination, designing these uber-geeky things for the NON geeks.

What would an alternative approach be like? Well, first of all, a very obvious move would be to keep the heads-up display but redefine it from a GUI screen to an Xterm. This could be cheaper to implement and would be better for the context (a tiny little screen). The OS could be Linux, certainly: but rather than asking whether it would be running KDE or Gnome, one would ask whether it's running bash or csh or tcsh or ksh. The device would store really large amounts of text information, for instance a Mac repair person such as myself might have in the gigs of disk space the entire Apple Tech Info Library and Archive, the body text of utilities such as Extensions Overload, and the total content of several mailing lists. Using such a device, I'd be working on something and would pause to type queries on my arm (presumably as grep searches or something). The HUD would return the entries I searched for, and might also occasionally produce notes I left for myself using cron to echo text to the screen or to kick off little scripts that would evaluate how I wanted things prioritized. Anyone who's dug into the classic Unix approach enough to become fascinated with the possibilities will appreciate what's being suggested here. Significantly, removing the GUI load from such a device and expecting entirely console-based interaction would simplify the design, allowing it to run very fast and efficiently on little embedded processors and not some desktop powermonger.

Naturally, when you talk about using stripped-down Linux implementations on wearables instead of using some sort of Windows, or a Linux desktop, the immediate assumption is that this is all about available computer resources, and that the future is about having enough resources that you can run KDE on a watch, or Windows on your garbage disposal. The only question seems to be if you have enough CPU and RAM. However, this distracts from the really interesting possibility- that the ideal geek wearable situation is not to have the computer stretch all the way towards the user, but to have each assimilate each other to some extent :) This is somewhat unobvious in the current marketing climate of the computer industry, but there's a lot of evidence to suggest that people can and do stretch the other way to 'jack in' to their computers more effectively.

A brief example to illustrate this: I was trying to make lists of syllables from different languages. This is maybe a weird thing to do, but it was what I was trying to do- ultimate goal was to make a naming system for my open source game project, but the important thing was this: I was presented with a problem for which there was no prebuilt solution. I found ways to translate writing of mine into other languages. I found ways (abstract) to break a text into 'trigrams'- So far so good. I ended up with some big text files in different languages, and a list of each one's 'trigrams', but I also needed/wanted to grab the most common first three letters and last three letters from the original words, as I meant to make additional lists of word starts and endings.

Enter grep. Interestingly, the grep I used was built into 'BBEdit Lite', not a Linux application- however, what I was using was not any specific application, but regexps, which I've not had lots of experience with. After giving each word its own line with a nongrep search and replace, and after some experimenting around with having the program just select the resulting patterns, I settled on ([a-z][a-z][a-z]).* replaced with \1, editing the pattern to be .*([a-z][a-z][a-z]) for the end-of-word cases. For the latter, I didn't need to experiment. Why? That gets back to the concept of rules.

The significant rules here are not strictly a list of the regexp commands- they are 'chunked' pieces of learning that can be extended and built upon. (parentheses) isolate bits of the matched pattern and save them for playback. This implies that if I had separate sets of parentheses, I could isolate separate bits of the pattern. [Brackets] match a range of characters, and the dash in the brackets means 'match from a to z'. This matches a character- so it's possible to place several of them in a row, matching a series of a-to-z characters. Lastly, I saw that .* matches everything else in the line, so if I put it outside the parentheses, it would select the whole line but would then replace it with only the parts I marked for 'yanking'. Putting this together, it becomes obvious that I could, for instance, do ([a-z])([a-z])([a-z]).* and replace each line with the argument \3\2\1 which would result in the first three letters of all lines being retained- only reversed!

Which, granted, is not the sort of thing consumers ache and pine for ;) however, discounting this too readily is a serious mistake, because what we're looking at is not a 'replace lines with the first three characters reversed' app- instead, it's a regular expression that is emergent from the rules of regexps. The only bits of regexps which entered into it were the ability to yank bits of text from a matched pattern, and to replace the yanked bits in whatever order I chose- yet, from these simple pieces, a more complicated result was reached. It required some looking up initially to be able to produce the regexp that I used for cropping all the words to three letters- but, interestingly, the idea of reversing the letters took no looking up at all. It was absolutely self-evident, and added itself to the possibilities of what could be done with such a grammar. The thought of 'And what else could you do?' was immediately answered- the underlying rules of how regexps work, including details such as 'the regexp is read from left to right', became instinctive and a novel result came out, a result that was structurally different from any of the experiments, but still obvious.

When you look at Linux as a whole, particularly if you have spent time doing little abbreviated installs on old 486es and seen the sort of programs you end up with, this one area (regexps) becomes just one of countless examples of this nature. The whole classic Unix approach of shell scripting and little programs that can be piped into each other, the sea of weird incantations, is deeply founded in this concept. There is no chance that such things can be made user friendly, because 'user' is not a concept that applies here. Approaching a computer this way is more like melding yourself with it, like the Borg metaphor we started with. Your brain becomes the amazingly powerful top level of the kernel- the whole Unix system becomes elaborate device drivers for your brain, completely dependent on your better human judgement for how to proceed. People sense this without being able to fully articulate it, and that's when you see them starting to claim that virtual teletype interfaces are far more powerful than OOP graphically-inherited widget collections.

Of course, the actual code of the virtual teletype interfaces is indeed more crude and primitive- but this misses the point entirely. A person who is willing to actually merge with the machine, go some way towards letting it focus on the things it's good at, is plainly going to gain an advantage over a person who refuses to meet the machine on its own territory. Imagine my example in the consumer sphere- how long would I have been waiting if I chose to simply bug commercial software developers for a program that read a file and put out only the first three letters of each line? Such a thing would never be done, so my project would have been stopped unless I was willing to do all that manually- but my willingness to deal with the computer on its level with regexps made it possible. Imagine similar situations around personal finances, or keeping lists of things to do, which are more typical consumer activities. The ability of a person to move and adapt and interact with life can be enhanced by not staying limited to predefined options, but at a cost: you never lose the requirement to put some effort into the process, to keep learning. There isn't a point at which you get to sit back with your elaborate information systems and passively consume, because the pressure of novelty doesn't stop.

When you add unrestricted access to programming tools such as C compilers, the possibilities for adding to your computer 'vocabulary' become just about limitless- still with that assumption that you're going to geek out, run your computer in a very active and involved manner. The benefit you get from this directly correlates to your ability to identify things for the computer to do. Some are incomprehensible to ordinary people, but in other cases you're evaluating information, making decisions, producing results with a speed and deftness that the commercial software users (by which I mean those who take a 'consumer' approach to computers) can't even begin to touch. It might be data analysis, or it might be something as simple as converting DEM height-elevation files to a normal picture format by opening them as raw data in Photoshop. It's about the approach to problems, and it tends to awe and disconcert people, who tend not to understand why they can't do likewise if only they had 'Product X', and don't see why you bother with the 'hard way' of doing things. It's like the difference between chopping down a tree with an axe, and chopping down a tree with an electric toothbrush :)

Of course, most people would find more use for an electric toothbrush. Indeed, an electric toothbrush is easier to use than an axe no matter who you are- it's less tiring, more convenient, and extremely approachable without training or lessons in how to use it properly- but its purpose is very tightly defined. In a way, that is the inevitable consequence of making it that approachable. By comparison, when you look at the classic Unix approach, you're basically looking at an axe: a super-sharp but basically crude tool for use on problems. Many uses of the tool take practice and effort to learn. You could cut down trees with a chainsaw, but an axe can do that and split wood and be used as a hammer or anvil or doorstop or bookmark or dirty-pan-scraper: many of the individual uses are awkward, but it's the ability to adapt the tool that deserves attention. Linux may end up repeatedly adapted to work in a purely passive, consumer mode, and it may be very effective in doing so, but this is likely to be a sideshow to the real potential of Linux, as a computer system more suitable to 'Borging' with than anything else out there.

The people who learn to solve problems in this way might end up the most powerful players in an information economy, the ones people turn to for advice and help and guidance. Power like that is never free- but the outsider rarely understands where it comes from. In the case of classic Unix approaches, it isn't the crude Teletype-like interfaces and programs- it's the state of mind, the need for new solutions and the willingness to do much of the work right alongside the computer that fosters those crude interfaces and odd little programs. The people who seek complete technical solutions will get them, but such a pursuit is a defining trait itself, slowing and limiting further growth and progress. New situations will arise, and the people who embrace the chaos and DIY-ness in computing will be quicker to identify the situations and adapt to them. In an era where the headlong rush of progress is thought to be almost intolerable anyhow, an advantage in this area could be vitally important.

There's one more use for an axe: you can carve furniture out of it. You have to have a good axe, and even then it's more difficult than using knives and planes and hammers, but it's still possible to do. And there's a name for doing this, of course. It's called 'hacking'.

- by Chris Johnson