This blog has moved. Go to SoftwareDevelopmentToday.com for the latest posts.

Saturday, March 07, 2009

Perfection in software is cheaper! not more expensive...

The fallacy of perfection = expensive has many forms. Two of which I just run into in the blogosphere. What is this fallacy? Simple, you say to yourself: "perfection in software development is way to expensive, therefore I should not even try to achieve it!", but that's a false problem, because perfection is only expensive if you continue to work in the way you do today. In order to take the next step to perfection you can change the way you work (process, guidelines, tools, etc.) and in the process you may end up with a cheaper way to do what you do today and be closer to perfection!

In fact there's is no single piece of evidence that would corroborate the idea that being better (i.e. trying to achieve perfection) is more expensive than, not trying to achieve it.

But let's get back to the examples that I just ran into on the net:

The first fallacy: the cost of Zero-Defects



In
a post by psabilla in shmula we are faced with the idea that Zero Defects would be too expensive to achieve therefore, the author suggests, it should not even be tried.

In here the author even presents a graph of the theoretical cost of achieving Zero Defects. It worthwhile to notice that this graph is a completely hypothetical graph, no connection with actual real data collected from a real project trying to really achieve Zero Defects.

Further along the author unmasks the prejudice that does not allow a more clear view into why Zero Defects are not only possible, but if achieved will yield a much faster _AND_ cheaper process for software development.

The author writes:
As defects are identified and eliminated, there will be theoretically few defects. But, this means that identifying defects will require more effort and will become more and more difficult, thus increasing the costs of this activity, along with the subsequent costs to fix the defects identified: The costs to inspect and test increases as there are fewer and fewer defects.


In the paragraph above take note of the causality that unmasks the prejudice. The author starts by using a seemingly obvious phrase (removing defects leads to less defects being present) and then exposes the prejudice behind the article: less defects means that it will be more expensive to identify other defects.

Think about that phrase for a minute: "if we have less defects it will be more expensive to find other defects".

Here are the problems with that phrase

  • It assumes that the role of testing is to find+remove defects: This is wrong, very wrong, because if a defect is added to the code base and only found much later that is indeed a much more expensive process than if we remove the defect closer to the source. Ideally immediately after it has been added to the source. I've written about this here. In other words: the role of testing is to prevent defects from being added to the source in the first place!
  • The second (wrong) assumption is that if we have less (presumably a lot less) defects we are then going to spend more money on finding the few that are left. Well, I don't know what projects the author has worked on, but as of today I don't know of any software project that would have achieved zero defects upon release. This actually means that at some point in the software development+release process the project team will say "alright, we have spent enough time looking for defects and not finding them, it is time to release". This suggests that project teams are smart enough to know when the cost of searching for a defect is higher than the benefit it brings.


Assuming the arguments above are sound we have to acknowledge that the basis for the argument by the author is false and indeed Zero Defects is not only possible, but also (given people's pragmatism) does not represent any additional cost to projects.

The second fallacy: You can't get rid of defects without Inspections



Jurgen Appelo, uses some of the data/arguments from the previously cited post to go on to say that Zero Inspections is an impossible and too costly goal. Note that Jurgen's post touches on other issues, I will not comment on those and restrict myself to the issue surrounding the impossibility or prohibitve cost of Zero Inspections.

The author builds on the (already demonstrated wrong) argument that Zero Defects is too expensive to also say that we cannot live without Inspections.

The author confuses Inspections with normal ways to prevent errors from going to far. I would agree that we need to have some inspections, but not in the way that Jurgen, or indeed Tom Gilb have suggested.

There's some data already available (albeit disputed) that Inspections provide little or no value and mostly focus on inspecting the format of the artifact (templates, coding conventions, grammatical defects, wrong color, wrong use of terms, etc.) and technicalities (curly braces where none are needed, too short method name, etc.) instead of the fundamental Defect magnets that logical flaws of thread-safety represent. This is also my experience.

Taking the example of code reviews, these are often more useful for knowledge sharing than for defect identification (not to mention that it is quite expensive to review code -- although you should do it in many situations :).

My point is this: Inspections have been around for 10's of years, why is the software industry consistently ignoring them: because they don't deliver enough value! At least Inspections in the sense of "let's get together in an additional meeting and review this artifact to make sure it has good quality, and then follow that with a formal process to change/update the artifact and get it approved" (code can be the artifact).

Ad-hoc inspections are often much more convenient, cheaper and value-adding but this is not what Jurgen or Tom Gilb suggest should be done! Hence my problem with Jurgen's article.

Zero Inspections is not only possible (been there done that, even with large - big L - projects) but it is also a catalyst for the teams to prepare not to need those Inspections. Imagine this: if you have an Inspection coming up for that last module you changed how motivated are you going to be do your job in the best way possible and aim for Zero Defects? The answer is: not motivated! In fact, my experience is that people will just aim to "avoid being embarrassed/humiliated during the Inspection meeting".

If you have the practices that support it and aim for Zero Defects (automated test, early Acceptance Test definition, etc.) you are going to be much more motivated (and supported) to achieve a very low (approaching Zero) defect count!

Oh, and by the way it follows that if you can achieve (proven) Zero Defects why do you need the Inspections?

Conclusion



There's a hidden (ok, not too much) argument here. If you aim for (and approximate) Zero Defects then your process will be faster (less rework due to defects, less inspections) and cheaper (less technical support needed, less reimbursements to annoyed customers).

But there's a catch! The catch is: you cannot achieve Zero Defects or Zero Inspections unless you invest in and change the way you work! It takes effort, dedication and constant experimenting and learning. However, if you don't do it your competitors will. Sooner or later. And when they do you will find yourself with a more expensive product that annoys your customers. What do you do then?

Labels: , , , , , ,

at 20:33
RSS link

Bookmark and Share

11 Comments:

  • Sorry, but your arguments are flawed.

    One example: "blind spots" or "editing blindness" where people are unable to see the errors in their own work because they've been working on it for so long.

    The only possible solution is to have someone else inspect (or review) the work. This person will find errors 10x faster than the author himself. Which evidently makes the inspection cheaper than trying to deliver with zero defects.

    Sure, TDD can alleviate some of that problem. But I don't see how to unit test my documents, or my book, or my blog posts.

    By Blogger Jurgen Appelo, at March 08, 2009 12:24 PM  

  • @Jurgen - Vasco seems to say High Ceremony inspections are not cost-effective, but some kinds of informal review are effective, and then seems to go the other way in the conclusion in saying that with automated tests, why do you need reviews?

    I was inclined to read the last one as a rethorical question. Let's suppose it is not.

    High Ceremony Inspections - I don't know, I've never attended one. Have you, Vasco or Jurgen? I'd be interested to hear some experiences. I do find low-ceremony forms of review like pair programming and having someone else review my writings very valuable.

    Why do I need inspections:
    Pair programming and collective code ownership free me from the illusion that I wrote enough tests on my own. In order to get the automated tests to a sufficient level to aid in our quest to zero defects we need continous review!

    With text I definitely have 'editing blindness'.

    Last year I had an essay torn apart by a professional editor. That was something else :) (very effective).

    I guess what makes a review or inspection effective is doing it one-on-one or two-on-one at the most. I can imagine why a High Ceremony Inspection would only find superficial 'defects'. In a group setting it is difficult to have dissent on fundamentals - before you know it groupthink takes over.

    By Anonymous Anonymous, at March 08, 2009 1:21 PM  

  • that last Anonymous comment was by me (because of NoScript I did not get the name/url textboxes...)

    By Anonymous Anonymous, at March 08, 2009 1:34 PM  

  • I think you're talking about different kinds of 'inspections' Jurgen and Willem seem to be talking about code reviews or similar inspections while Vasco seems to be talking about inspections in the lean sense like testing, code reviews etc.

    My reply to this post got a bit long so I replied on my own blog. Trackbacks still don't seem to work so here's the link:

    By Anonymous Anonymous, at March 08, 2009 4:20 PM  

  • @Jurgen inspection that is done as a matter of course (ie every day as part of getting something to DONE) is not only needed, it is a key part of what I advocate in my article: aim for Zero Defects. Inspection in the way that Tom and Kai Gilb advocate is a waste of time. I know the jury is still out on the actual proof for this statement, but that is clearly my experience

    TDD is in itself a form of design more than it is a form of inspection. ATDD on the other hand is a form of inspection that should be looked into as part of the testing/inspection/quality strategy for a project. Pair programming would be another form of inspection. If you aim for Zero Defects you will have many inspections (much more than otherwise). But those will be light weight with no ceremony. In fact that teams that do this type of inspections don't even call it that. They just call it work :)

    By Blogger Unknown, at March 08, 2009 9:56 PM  

  • @Willem I have worked many years in an "Inspection-heavy" company. There was nothing wrong with that per se. That was a company that was applying waterfall, and Inspection can be useful in Waterfall because of the "big-batch" approach (large documents that are "ready" before being handed over to designers, etc.). In a waterfall setting High Ceremony Inspections can be a way to reduce defects by detecting them earlier than what you would otherwise. However it is a very expensive and hugely inneficient way to do it.

    In an iterative and incremental process High Ceremony inspections are not needed because of the nature of the process. You build what would be inspections into your every day work, no overhead and much more efficient because you find defects (almost) immediately.

    By Blogger Unknown, at March 08, 2009 10:00 PM  

  • @Mendelt I'll comment on your post in your blog: http://bit.ly/tJPd

    By Blogger Unknown, at March 08, 2009 10:02 PM  

  • "There's some data already available (albeit disputed) that Inspections provide little or no value..."

    Where is the data?

    By Anonymous Anonymous, at April 22, 2009 7:24 AM  

  • @Anonymous I searched for that type of data from public sources but do not have that. I'll ask permission to share highlights of that data from the author of the paper.

    By Blogger Unknown, at April 22, 2009 10:09 AM  

  • To argue for zero defects as a viable goal seems like a questionable intellectual exercise. The perfection of nature is in the built in "error" of DNA mutation that facilitates survival and adaptation of species to ever-changing environments. Human errors (and unexpected outcomes) seem like a law of the universe. How else would we have created post-it notes :)

    Interesting to note that this article on zero defects has two spelling erorrs. Thanks for the mental stimulation.

    By Anonymous Dano, at February 19, 2013 7:19 PM  

  • @Dano Zero defects does not mean avoiding unexpected events. In fact it means making sure your process acknowledges and handles those unexpected events :)
    You are welcome! ;)

    By Blogger Unknown, at February 19, 2013 10:26 PM  

Post a Comment

<< Home


 
(c) All rights reserved