Why You Will Need Fewer But Smarter Developers

An integrated development environment (IDE) may require fewer programmers, but they must be even smarter and more experienced than ever before.  Here’s why:

Development has always involved tool selection:  Modeling and requirements tools for the analysts; an integrated design & development environment for the programmers; test automation tools for QA; project management tools for the project managers; etc.    The classic analogy was with building contractors, where developers, like carpenters and metalworkers, only required a good foreman and some practice using their equivalent of joiners, lathes or band-saws. 

Unfortunately, unlike real joiners or lathes, the software equivalents have become amazingly complex.  Why is this complexity necessary?  IDE’s are an attempt to solve a long-standing software and system development management problem for corporate applications:  If you allow analysts, programmers and testers to use separate tools, how do you ensure they will work together productively using a consistent process to achieve high quality results, especially in corporate IT development shops, which must constantly integrate new & old data, systems and networks? 

I don’t doubt that the product managers of these integrated development environments (IDEs) added every feature for very good reasons, with the goal of making development more productive.  Some of the products I’ve recently reviewed (such as IBM’s Rational tools) are truly a tour de force achievement, encapsulating a huge percentage of the “software body of knowledge” in their conception, design, and functionality.  Just reading the help screens can give you a capsule PHD in the history of software engineering and management.  Furthermore, today’s IDE’s even attempt to automate the business process of software development itself, requirements elicitation, a good deal of coding, almost every good software engineering idea we’ve had over the last forty years.

But IDE complexity distracts developers from understanding the business problem underlying any project.  Every added feature drains attention from achieving a useful business goal.  IDE’s require many weeks of training, many months of practice, and understanding why they work as they do is difficult for anyone with less than a decade’s work experience or a higher degree.  This involves complexity well above and beyond learning any programming language.

Part of the reason for IDE complexity is an attempt to create more specialization of development roles.  For large projects, this allows for more divisions of labor to allocate work most cost effectively.  For example, developers are only required to understand a requirements formalization delivered by selected analysts.  Of course, this means that the developers cannot be expected to apply much judgment to shaping their work products, and bad requirements will tend to propagate downstream.  Probably even more severe is the impact on QA, who will now just test formal requirements, again without applying any good business judgment, and many defects deemed “obvious” by your customers will be discovered.

The most expensive defects are requirements defects, because their impact is felt throughout the development lifecycle.  Pending some new methods for validating requirements, IDE’s will introduce new risks, with no guarantee that lifecycle cost reductions will be achieved.

If you believe, as I do, that an understanding of the business problem is essential to high quality results, corporate IT managers, once they decide to take the IDE-plunge, are left with two options:  Use the IDE for the largest projects, and for smaller projects, only use IDEs when a sufficient staff of no-shit IDE experts is available. 

The largest projects have greater ability to spread capital & labor & training costs around, especially offshore.  Such projects must have a larger staff devoted to requirements analysis and dedicated to understanding the business problem.  The project’s size helps amortize IDE training costs, via both formal training on the IDE tool and by providing an opportunity for employees to use the IDE to solve a real-world problems.   However, the risk of requirements defects needs to be rigorously managed, via prototypes, constant customer contact, formal methods – whatever tools are at hand.

For projects with less than about 20 full time equivalents, I believe IDE’s should be used only when you have a staff that can operate in both your business and technology spaces, and who have mastered the IDEs already.   Understanding a business problem and the use of technology to solve it requires developers to understand two different linguistic and social spaces, one business and the other technology, and such talent is no doubt hard to find or develop.  But if you don’t use such a staff, there is too much risk that your business problem won’t be solved, through misunderstanding of requirements, or because the solution won’t be cost-effective, due to tool training costs and low productivity.

Parting thought:  It may be that the rapidity of Internet application development, relative to corporate applications, is due to its simpler programming model and its “let a thousand tools bloom” attitude.  Internet applications, of course, do tend to be much less integrated than corporate applications, and many of them are green-field.  But it may also be true that the newest market for software development benefits from its more old-school, non-integrated development environment approach to software development.

 

 

Comments

[bob]

Ah yes. A common understanding of 'how the business works' shared amongst the buyers, designers, coders, testers. Oh those cross cultural misunderstandings. The very impossibility of writing downs requirements for every screen widget, meaning of every label, expectations behind every button...without a common understanding based on common experience you're writing thousands of pages.

[David]

Note to Bob: Thousands of pages of requirements without context, each easy to implement by someone who has no idea what the solution is for. Kafka was a prophet -- see my coming post, "everything I know about software I learned via literary criticism".