Job Opening: Principal Software Engineer - Mob (US, CAN, MX)
Hunter Industries Careers

The Hidden Costs of Estimates

Cost of estimate = CO8020R * COD + DT + ECT + PL

I get a lot of great questions about #NoEstimates, and do Q&A from time to time but this question in particular I thought deserved its own blog post. I wanted to name this post "You wouldn't believe what these organizations paid for their estimates!" but I thought that would be too sensationalist, unlike the #NoEstimates movement. So here we go:

What are the “costs” of software estimates?

There is no tangible "first order" cost to estimates directly except the time it takes to make them. The problem is they encourage all sorts of bad behaviors. #NoEstimates for me is more about being lean that not estimating. Lets try to develop software without them and get good at practices like continuous delivery and zero bugs. Once we are good at those things, we can create a lean value stream map and see if estimation fits anywhere where it will have value.

First, A quick rough calculation would be as follows:

The key concepts are explained below. As you can see however there is a lot of second order effects that are identified in organizations that still use estimation with software projects.

Cost of estimate = CO8020R * COD + DT + ECT + PL + COPS + COPDP


So what does that notation actually mean?

The following are the terms in the above equation that discuss the concepts we are trying to asses when talking about the cost of estimates.

(CO8020R): "Cost of the 80/20 Rule" - 80% of features developed for a project is in response to requests from 20% of the users. One thing I have noticed over the years is that estimation enables people to ignore the side effects of the 80/20 rule. Practicing pure Kanban for development prioritized on highest expected value to the customer creates an environment where the features that we should not develop naturally get de-prioritized. The idea with the rule is that 80% of the features are not useful to the core audience. Lumping features of this type in a big bang release or even a sprint will be more palatable if estimates accompany them. The alternative here is to use a pull system like Kanban to prioritize and deploy work.

(COD): Cost of Delay is the value not realized for the duration that a feature does not exist. This value can only be calculated after the feature is delivered. Since the scope of the project is likely to be artificially bloated by the 80/20 rule we have to multiply the time we spend on low priority work times the cost of delay of the high priority work. This is especially significant when doing big bang releases or non Kanban releases. The time it takes for the feature to be in production will amplify this cost.

(DT): Design Trade-offs are typically sacrifices made to the quality of the product in favor of speed to market. Long term cost of Design Trade-offs can come in many forms. The most commonly discussed in software development is choosing to not refactor, implement a design pattern, or write unit tests for code before it goes out. The true cost of this type of decision is made evident when you consider Martin Fowler's Design Stamina Hypothesis. Other trade-offs could be readability of code, usability of the application, and operating costs of the functioning system. Estimates have a tendency to be converted to artificial deadlines even by the developers doing the work. This prompts developers to rush and even subconsciously  make design trade-offs that will cost orders of magnitude more for all aspects of future development. Creating an environment where developers want to rush is a sure fire way to dramatically increase costs over time and make everyone feel comfortable with it.

(ECT): Estimation Creation Time is the most straight forward item on this list. This cost is the time spent by the people making the estimates in the first place which can be significant. Based on the estimation technique used, there may be developers and executives involved in the estimation for weeks to months of the project. The subtle aspect of this cost is the fact that this duration is also multiplied by the cost of delay. This measure alone has been greater than 10% of the cost of projects I have developed for.

(COPL): The Cost of Parkinsons Law can be visualized as the amount of bureaucracy created in the software development life cycle. Parkinson's Law is the adage that "work expands so as to fill the time available for its completion." In a pure Kanban and continuous delivery system there is no set time for a release to go out. The idea is to release when done and pull the next work item. In a system containing estimates and batched releases like waterfall or sprints under estimation has a subtle side effect of creating a habit around using all available "extra" time to refactor. This would be a good thing however that means that all slack time is allocated for tasks that may be already completed. This then has the side effect of reducing slack time for items that unexpectedly take longer. The final result is a system that incentivizes rushing, and creating expensive to maintain code.

(COPS): Cost of  poor security is typically introduced when tasks chosen by the team are easier than security tasks ahead of security implementations. When time runs out on the perceived deadline the incentive to release an unsecure product is high. The application will appear to work but will have a delayed cost, especially if the security holes lead to a large scape breach. This cost is difficult to quantify but could easily lead to class action lawsuits and other equally expensive problems.

(COPDP): Cost of poor data protection. Teams also may be in a situation where the pressure leads them to release a product without proper or adequate data protection. Similar to COPS, this cost has a delayed but significant effect. Ignoring GDPR carries a 4% gross revenue penalty with it regarding negligence. 

What do we do with this information?

So given we have an idea of how much estimation is actually costing us. Now we should ask ourselves what value are we really getting out of them? I don't think it can be calculated until we have a system that employ's lean principals and has optimized its value stream map. Rather than working on estimation, work toward continuous delivery with zero bugs. When you can release to production daily and you have no known bugs in your system experiment with whether you need them or not. Unless you already know intuitively :)








2 comments:

  1. Nice. Thanks for blowing that up, Chris. All so true.

    ReplyDelete
    Replies
    1. I appreciate to Kudos! I have spent way too much time thinking about this :)

      Delete

 

Follow