Am I asking for too much?

I am currently participating in a interesting thread on Matt Berther’s bog on my podcast. I commented on his post Blasting open source because I felt my podcast was not meant to pick on up source projects although part of it could be interpreted that way. So since then what I have been explaining is why I don’t see any innovation happening in software today.

Although I like .NET and Java I think there has been no real innovation in software in the last 30 years. The primary things I see as the backbones of what we use computers for are:

1. Databases - CODASYL defined the standard for network databases in 1969 (RDMBS were first described by Codd in the 1960’s and 70’s at IBM)
2. Word Processing - The Unix Concepts of text formatting for publishing came around in the 1970’s (as a matter of fact UNIX was first designed as a text editing and text formatting system). Although UNIX text processing is much more akin to HTML then WYSIWYG editors like MS Word
3. Spreadsheets - First patented in 1971, VisiCalc was actually being distributed for the Apple II in 1979
4. General Programming via high level Languages -
   a) Interpreted Languages (If you want the full deal Smalltalk was around in the 1970’s. Object oriented, dynamically typed and reflective)
   b) Compiled Languages (Cobol was created in 1959 and C in 1972)
5. WIMP Interfaces - (Pioneered by the Xerox Alto in 1973)
6. The Internet which started in 1969.

So if you take as a baseline of 1980 we have had these systems around for at least 28 years. Most have been around longer. Sadly I have been programming for 30 years so I have played with some of these at one time or another in my career.

Do I expect too much? I can’t help but see the feature set of these products though and say that on the whole the pace of innovation in computers disappoints me. Those advances we do see to have picked up come about more because of hardware advances then software advances. As Brooks says in No Silver Bullet

…the anomaly is not that software progress is so slow, but that computer hardware progress is so fast.

As I see it we continue to attack the accidents of software engineering. I quote from Brooks

If we examine the three steps in software technology development that have been most fruitful in the past, we discover that each attacked a different major difficulty in building software, but that those difficulties have been accidental, not essential, difficulties.

The three that he describes include:

• High Level Languages. Now this would include java and C#.
• Time Sharing.
Now this would be the entire concept or personal computers, laptops, PDA’s etc…
essentially ways to get the answer immediately.
• Unified frameworks. Here
he talks about UNIX, but you can also include in this the .NET framework and the
Java class libraries.

So my contention is that one of the reasons software innovation is so slow is we spend too much time reinventing what we have had since the 1970’s and not focusing on advancing the art of computer science. Am I wrong? So many of the things we see in the Open Source and Commercial World are clones of each other. Are we as developers taking the simple way out? Sure we can design a new enhanced WIMP interface, but that is because we already know most of the requirements except for that little tweak we are going to make? It is fun to do these kinds of projects because we don’t have to think about the hard stuff like requirements analysis, test cases and design. We just set down and code another build system, or IOC Container or OS.

We continue to attack the accidents because the essentials are so difficult. I was doing some reading on the System.Speech namespace in .NET and Natural Language Processing in general. One of the big things required to get these to work is to put together grammars for what we want the computer to do once it has recognized the words. The problem is almost no one is building the grammars. Why is that? Because it is hard. It would require us to step outside of the fun and simple things we do and take on some difficult tasks. The same thing could be said for coming up with systems that use common component architecture. How much simple would things be if we started building components and plugging them into applications. As David said in my Podcast, I want to drag and drop a CRM solution.

What does it take to get to these? We have to give up on some of the fun things we do and start attacking the essential problems in software development. Am I expecting too much? Are we as Software Engineers unable to tackle our discipline as a engineering science and start innovating again?

** Portions of this entry were posted a a comment on Matt’s blog ***

Founding Brothers

A friend of mine, Chuck Steffens, has convinced me to start reading Founding Brothers: The Revolutionary Generation by Joseph J. Ellis.  I must say this is a fascinating look at American history.  Specifically the time of the American Revolution and the founding of the republic.  I will do a book review in the next few days, but so far (up to chapter 3) it is a great read.

I downloaded it to my Kindle last night at midnight and started reading it.  The only thing I can say about the kindle is I am glad they don't have a ton of Computer Books available on it yet.  I would go broke every time I went to amazons web site. :-).  The science fiction and historical stuff us costing me a fortune.  On the plus side I have finally started to read Winston Churchill's series on World War II after having owned the books for years.

Google App Engine

I just ran into a reference for this.  It looks pretty cool.  Develop your application offline and then deploy it to the Google servers.  The man page is at http://code.google.com/appengine/. 

I have been meaning to play with Python and Iron Python.  I have just never had a reason beyond curiosity.  Now it looks like it might be worth learning.

Mono 1.9 is Released

Mono 1.9 is released.  This is the last version before 2.0 which supports the entire 2.0 version of .NET on Linux.  This version also supports some C# 3.0 features including Linq to XML and Linq to Objects.  Very Cool
http://www.go-mono.com/mono-downloads/download.html

They have also released the Mono Develop IDE.  Full support in an IDE very similar to Visual Studio. 
http://www.monodevelop.com/Main_Page

Checkout the screen capture with their integration with NUnit

http://www.monodevelop.com/Image:Nunitmain.png

It is all about Portability...

As cell phones get more and more powerful, you wonder how soon they replace laptops.  Primary thing missing are good keyboards, mice, decent size monitors, better performance and memory.  The Windows CE development environment is there already.  So let's take a look at these one at a time...

• Now I can get a 8 gig memory card from SanDisk for about $50.  This kind of makes me wonder.  How much document memory etc do you need?
• Now for keyboards I can get any number of virtual keyboards although the support looks a bit shaky.
• For Mice can I use any standard Bluetooth mouse?
• Now here comes a projection system.  Is this a possible replacement for the monitor?

Mind you,  none of these are that great now, but with cell phone providers adding more and more content to the phone and revenue from voice communications stabilizing you will probably see cell phone providers going after any source of revenue they can find.  Of course I don't think I will see Visual Studio there anytime soon, but who knows?

Accidental Properties of Software: Programmer Productivity

While researching for these articles I ran across this blog entry describing something the author refers to as Yannis's Law: Programmer Productivity Doubles Every 6 Years.  The whole premise of the article is to dispute Brooks statement that there will be no order of magnitude improvement in the essential properties of software development.  The author goes on to prove that programmers are seeing a 11x improvement in software development since 1986.  There are two places this assessment is wrong.  First Brooks was looking for an order of Magnitude improvement in 10 years (by 1996) not 21 years (2007).  Second this is an attack on the accidental property of software (it's construction), not the essence of software, the mental crafting of the conceptual construct.

I do not dispute there are a great many cool tools and languages out there to increase programmer productivity, but we should all have seen the breakdown of the Software Development Life Cycle by now.  How much of the SDLC is related to software construction?  If we say it is 30% (which is probably a bit high) we are still looking at over 70% of the project time that is not being addressed by this improvement.  As we reduce the construction time to zero (which would be magic :-)), we are still only attacking the accidental properties of software.  What are we going to do about the other 70%?  We have still not addressed the essential property.

Attacks on the Accidental and Essential Properties of Software

Over the next several months I will be posting articles that discuss various aspects of software and the state of the industry. These will be titled as either attacks on the essential properties of software or the accidental properties of software as defined by Brooks in my post I first learned about Agile Development….

Brooks went on to provide a clearer definition of Accidental and Essential Properties in No Silver Bullet Refired (Chapter 17 of the Mythical Man Month).  In particular Brooks uses Dorothy Sayers definition of creativity:

instead I follow the English  dramatist, detective story writer, and theologian  Dorothy Sayers in seeing all creative activity to  consist of (1) the formulation of the conceptual  constructs, (2) implementation in real media, and  (3) interactivity with users in real uses.[6] The part  of software building I called essence is the mental  crafting of the conceptual construct; the part I  called accident is its implementation process.

The goal is to try to discuss the progress towards a silver bullet for software development. These are my thoughts on progress in the methodologies and tools to make developing software easier. These are things we are doing as an industry to attack the Essential properties of Software. There are also that are just white noise interesting, fun and useful, but in the end just attacks on the accidental properties of software and have no real impact on our ability to quickly create quality software that attacks the essence of building software, or as Brooks says:

I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation. We still make syntax errors, to be sure; but they are fuzz compared with the conceptual errors in most systems.

If this is true, building software will always be hard. There is inherently no silver bullet.

So lets look around for what we have done in the last twenty years to attack the essential properties of software and progress towards the silver bullet of software (from the essay No Silver Bullet written in 1986).