Am I asking for too much?

I am currently participating in a interesting thread on Matt Berther’s bog on my podcast. I commented on his post Blasting open source because I felt my podcast was not meant to pick on up source projects although part of it could be interpreted that way. So since then what I have been explaining is why I don’t see any innovation happening in software today.

Although I like .NET and Java I think there has been no real innovation in software in the last 30 years. The primary things I see as the backbones of what we use computers for are:

1. Databases - CODASYL defined the standard for network databases in 1969 (RDMBS were first described by Codd in the 1960’s and 70’s at IBM)
2. Word Processing - The Unix Concepts of text formatting for publishing came around in the 1970’s (as a matter of fact UNIX was first designed as a text editing and text formatting system). Although UNIX text processing is much more akin to HTML then WYSIWYG editors like MS Word
3. Spreadsheets - First patented in 1971, VisiCalc was actually being distributed for the Apple II in 1979
4. General Programming via high level Languages -
   a) Interpreted Languages (If you want the full deal Smalltalk was around in the 1970’s. Object oriented, dynamically typed and reflective)
   b) Compiled Languages (Cobol was created in 1959 and C in 1972)
5. WIMP Interfaces - (Pioneered by the Xerox Alto in 1973)
6. The Internet which started in 1969.

So if you take as a baseline of 1980 we have had these systems around for at least 28 years. Most have been around longer. Sadly I have been programming for 30 years so I have played with some of these at one time or another in my career.

Do I expect too much? I can’t help but see the feature set of these products though and say that on the whole the pace of innovation in computers disappoints me. Those advances we do see to have picked up come about more because of hardware advances then software advances. As Brooks says in No Silver Bullet

…the anomaly is not that software progress is so slow, but that computer hardware progress is so fast.

As I see it we continue to attack the accidents of software engineering. I quote from Brooks

If we examine the three steps in software technology development that have been most fruitful in the past, we discover that each attacked a different major difficulty in building software, but that those difficulties have been accidental, not essential, difficulties.

The three that he describes include:

• High Level Languages. Now this would include java and C#.
• Time Sharing.
Now this would be the entire concept or personal computers, laptops, PDA’s etc…
essentially ways to get the answer immediately.
• Unified frameworks. Here
he talks about UNIX, but you can also include in this the .NET framework and the
Java class libraries.

So my contention is that one of the reasons software innovation is so slow is we spend too much time reinventing what we have had since the 1970’s and not focusing on advancing the art of computer science. Am I wrong? So many of the things we see in the Open Source and Commercial World are clones of each other. Are we as developers taking the simple way out? Sure we can design a new enhanced WIMP interface, but that is because we already know most of the requirements except for that little tweak we are going to make? It is fun to do these kinds of projects because we don’t have to think about the hard stuff like requirements analysis, test cases and design. We just set down and code another build system, or IOC Container or OS.

We continue to attack the accidents because the essentials are so difficult. I was doing some reading on the System.Speech namespace in .NET and Natural Language Processing in general. One of the big things required to get these to work is to put together grammars for what we want the computer to do once it has recognized the words. The problem is almost no one is building the grammars. Why is that? Because it is hard. It would require us to step outside of the fun and simple things we do and take on some difficult tasks. The same thing could be said for coming up with systems that use common component architecture. How much simple would things be if we started building components and plugging them into applications. As David said in my Podcast, I want to drag and drop a CRM solution.

What does it take to get to these? We have to give up on some of the fun things we do and start attacking the essential problems in software development. Am I expecting too much? Are we as Software Engineers unable to tackle our discipline as a engineering science and start innovating again?

** Portions of this entry were posted a a comment on Matt’s blog ***