Unix Programming - Problems in the Environment of Unix

Problems in the Environment of Unix

The old-time Unix culture has largely reinvented itself in the open-source movement. Doing so saved us from extinction, but it also means that the problems of open source are now ours as well.

One of these is how to make open-source development economically sustainable. We have reconnected with our roots in the collaborative, open process of Unix's early days. We have largely won the technical argument for abandoning secrecy and proprietary control. We have thought of ways to cooperate with markets and managers on more equal terms than we ever could in the 1970s and 1980s, and in many ways our experiments have succeeded. In 2003 the open-source Unixes, and their core development groups, have achieved a degree of mainstream respectability and authority that would have been unimaginable as recently as the mid-1990s.

We have come a long way. But we have a long way to go yet. We know what business models might work in theory, and now we can even point at a sporadic handful of successes that demonstrate that they work in practice; now we have to show that they can be made to work reliably over a longer term.

It's not necessarily going to be an easy transition. Open source turns software into a service industry. Service-provider firms (think of medical and legal practices) can't be scaled up by injecting more capital into them; those that try only scale up their fixed costs, overshoot their revenue base, and starve to death. The choices come down to singing for your supper (getting paid through tips and donations), running a corner shop (a small, low-overhead service business), or finding a wealthy patron (some large firm that needs to use and modify open-source software for its business purposes).

In total, the amount of money spent to hire software developers can be expected to rise, for the same reasons that mechanics' hourly wages go up as the price of automobiles drops.^[162] But it is going to become more difficult for any one individual or firm to capture a lot of that spending. There will be many programmers who are well off, but fewer millionaires. This is actually a sign of progress, of inefficiencies being competed out of the system. But it will represent a big change in climate, and probably means that investors will lose what little interest they have left in funding software startups.

One important subproblem related to the increasing difficulty of sustaining really large software businesses is how to organize end-user testing. Historically, the Unix culture's concentration on infrastructure has meant that we have not tended to build programs that depended for their value on providing a comfortable interface for end-users. Now, especially in the open-source Unixes that aim to compete directly with Microsoft and Apple, that is changing. But end-user interfaces need to be systematically tested with real end users — and therein lie some challenges.

Real end-user testing demands facilities, specialists, and a level of monitoring that are difficult for the distributed volunteer groups characteristic of open-source development to arrange. It may be, therefore, that open-source word processors, spreadsheets, and other ‘productivity’ applications have to be left in the hands of large corporate-sponsored efforts like OpenOffice.org that can afford the overhead. Open-source developers consider single corporations to be single points of failure and worry about such dependencies, but no better solution has yet evolved.

These are economic problems. We have other problems of a more political nature, because success makes enemies.

Some are familiar. Microsoft's ambition for an unchallengeable monopoly lock on computing made the defeat of Unix a strategic goal for the company in the mid-1980s, five years before we knew we were in a fight. In mid-2003, despite having had several growth markets it was counting on largely usurped by Linux , Microsoft is still the wealthiest and most powerful software company in the world. Microsoft knows very well that it must defeat the new-school Unixes of the open-source movement to survive. To defeat them, it must destroy or discredit the culture that produced them.

Unix's comeback in the hands of the open-source community, and its association with the freewheeling culture of the Internet, has made it newer enemies as well. Hollywood and Big Media feel deeply threatened by the Internet and have launched a multipronged attack on uncontrolled software development. Existing legislation like the Digital Millennium Copyright Act has already been used to prosecute software developers who were doing things the media moguls disliked (the most notorious cases, of course, involve the DeCSS software that enables playing of encrypted DVDs). Contemplated schemes like the so-called Trusted Computing Platform Alliance and Palladium threaten^[163] to make open-source development effectively illegal — and if open source goes down, Unix is very likely to go down with it.

Unix and the hackers and the Internet against Microsoft and Hollywood and Big Media. It's a struggle we need to win for all our traditional reasons of professionalism, allegiance to our craft, and mutual tribal loyalty. But there are larger reasons this struggle is important. The possibilities of politics are increasingly shaped by communication technology — who can use it, who can censor it, who can control it. Government and corporate control of the content of the nets, and of what people can do with their computers, is a severe long-term threat to political freedom. The nightmare scenario is one in which corporate monopolism and statist power-seeking, always natural allies, feed back into each other and create rationales for increasing regulation, repression, and criminalization of digital speech. In opposing this, we are the warriors of liberty — not merely our own liberty, but everyone else's as well.

^[162] For a more complete discussion of this effect, see The Magic Cauldron in [Raymond01].

^[163]See the TCPA FAQ for a rather hair-raising summary of the possibilities by a noted security specialist.