Recording the inevitable of programming: The system crash
March 23rd, 2008Anyone who uses a computer knows what its like to have the system crash. Crashes are the digital worlds addition to that short list of inevitables, death and taxes. But what if you could record the crash and play it back, like a video recorder for software?
That idea inspired two software engineers, Jonathan Lindo and Jeffrey Daudel, to devise such a product. They have succeeded, and are now moving from the niche market where they proved the idea and onto a bigger stage.
System crashes and other software flaws are more than an annoyance. A 2002 study by the National Institute of Standards and Technology in the United States estimated that software flaws cost the economy there as much as $59.5 billion a year.
For software developers, the flaws that cause crashes rank among their biggest problems, especially the ones that cannot be reproduced, like the noise in the car engine that disappears when you visit the mechanic.
Lindo says he and Daudel found themselves overwhelmed by bugs they could not find while working together at an Internet start-up in 2002. “We were spending almost all of our time not fixing the issues, but trying to get to the point where we could just see the issue, and we said, Wouldnt it be great if we could just TiVo this and replay it? ” Lindo recalls.
Innovation by analogy is a powerful concept, says Giovanni Gavetti, an associate professor at the Harvard Business School who, with his colleague Jan Rivkin, has published research on how businesses can use analogic reasoning as a strategic tool. Human beings are analogy machines, he notes, dealing with new information by comparing it to things they already know something about.
It would take time for Lindo and Daudel to prove that their analogy worked. They were tackling a daunting problem - in fact, friends told them that they had a great goal, but one that was probably impossible to achieve. For one thing, they had to account for everything that can affect a program, from keystrokes, mouse movements and other software applications to network traffic and programming instructions that are designed to occur randomly (for instance, in a computer game, the villain should not always do the same thing).
Ideally, their tool would not slow down the system as it recorded what was happening. They also were developing it among the most complex of software environments: game platforms.
There were already programs on the market that could do things like log all the various inputs a program received. But none of them worked as the program was running, which is what developers really want, say analysts like Theresa Lanowitz of Voke, a technology research firm. In effect, that meant these products took snapshots, not the “video” that Lindo and Daudel thought was necessary.
Eventually, they got their technology working, and in December 2003 quit their jobs and started Replay Solutions - so named because it would replay software crashes.
Their product, ReplayDirector, works on the Xbox gaming platform and several versions of the Microsoft Windows operating system. One customer is Electronic Arts, which began using the product in the fall of 2006, according to Steven Giles, its director of online operations.
Giles says he was referred to Replay by a venture capitalist he knows. The venture capitalist, however, worried that the companys software was “smoke and mirrors,” and Giles initially felt the same way. But when he realized that it worked, he convinced more than 10 developer teams at Electronic Arts to license the tool (the venture capitalist ended up investing in Replay).
Giles said that he liked a number of things about the tool but that one stood out: its ability to capture bugs that cannot be reproduced.
“Thats something that nobody inside or outside our industry has really been able to solve,” he says. “We refer to it almost as magic.”
He declined to say how much Electronic Arts saves by using Replay, but says that “where its obvious that there are savings is when you see your most senior and expensive developers working on the game rather than chasing these ghost bugs.”
Replays gaming software sells for $50,000 a project, with negotiated pricing for multiple projects.
There are still pieces missing from the TiVo analogy. For instance, software developers cant yet fast-forward to see their crashes. Still, having tackled games, Replay is now expanding into new markets. Last week, it released a beta version of its next product, for developers who write in the Java programming language. Versions for other markets will also start appearing later this year, and the technology should transfer to almost any kind of software environment, says Vishwanath Venugopalan, an enterprise software analyst at the 451 Group, a research firm in New York.

Posted in 