Software Cost Estimation – Why not?

Sometimes I feel I am not living in the same world everybody else does. I was (and am), involved in professional software estimation, benchmarking, and contracting processes for almost half of my life, and still most people I talk with don’t have a clue what this is about. Especially the ‘groovy, cool, new, young, super chill youngster generation’, has no idea what I am talking about. ‘Software estimation is hard, and always goes wrong, so why even try it?’ ‘#Noestimates’, check it out on twitter, it’s a real thing. And these people are serious! They believe in the power of the team. This is easier said than done when you have a business to run. Like a close friend of mine, who is working for a large international system integrator, always tells me: ’No client ever gave us a million dollars with the request to make whatever software solution, as long as it brings some business value’. I’m just so much annoyed whenever people say that software estimation is not necessary any more in the agile world of delivery. It is, and even more than before!

Agile teams often use story points, an arbitrary effort unit of measurement that is relative to a certain amount of work. Story points are not repeatable, not verifiable, not objective and not defensible. But the agile community seems to manage to convince IT management on this planet that metrics based on story points are useful to support management decisions.

It’s really like someone convinced the painters in the world to use ‘wall points’ instead of a standardized metric like square meter to measure the size of the walls and to estimate their wall painting jobs. I don’t think many people would accept a quote like ‘I’ll charge you $ 100 per wall point’, right? Simply does not make sense, as the number of wall points can never be measured objectively, as it is not standardized. To give a quote like ‘$ 100 per square meter’ does make a lot of sense, because it is standardized, and therefore transparent, predictable and defensible.

 

Only just today, I spoke to 2 different customer organizations, both complaining that their supplier, working agile, completely underestimated the effort necessary to implement a ‘must have’ piece of software, resulting in severe cost and schedule overruns. How is this even possible in agile, you ask? Very simple! Although scope is supposed to be variable, organizations still need a certain minimal set of functional (and non-functional) user requirements to be ready and implemented at some moment in time and they also allocate some budget to that, because they must comply to financial and accounting laws. If you underestimate the effort (and you will if you use only expert estimates!), the team will too small and the software won’t be ready when it should be.

Software cost estimation is still very relevant, also in agile times. At the annual conference of the International Cost Estimation and Analysis Association (ICEAA), last June in Phoenix (AZ, USA), ICEAA and Nesma introduced the Software Cost Estimation Body of Knowledge (sCEBoK). In this conference, over 450 cost estimators of organizations like Boeing, NASA, Lockheed Martin, US Navy, etc. convened to discuss professional cost estimation practices, also regarding software. For them it’s not an option to hide behind arbitrary story points, they need comply to international standards and best practices to make sure the estimate is as accurate, but also as defensible as possible.

As a social experiment, I decided to post a message to inform the industry about the Nesma/ICEAA initiative in several LinkedIn groups, and I was especially interested to see the reactions from the group ‘Agile and Lean Software Development’, which has over 140.000 members. The text:

‘Estimation maturity processes on average are still quite low. Large sums of money continue to be wasted because projects were not estimated professionally, resulting in cost and schedule overruns. One of the issues across all of the industry is the fact that the software cost estimator is often not a recognized profession. In many organizations, estimates are based on experience and opinions of biased human experts, instead of being based upon relevant historical data and parametric models. Nesma and ICEAA (International Cost Estimation and Analysis Association) are working together to create a training curriculum and certification for ‘Software Cost Estimator’, a professional role for estimating software related activities and products.

 In cooperation with a number of supporting organizations and professionals in the industry, Nesma and ICEAA have been working together to develop a Software Cost Estimation Body of Knowledge (sCEBoK). In line with the training and certification programs ICEAA already has for cost estimation, the goal is to develop a training and certification program specifically for the software cost estimator. The plan is to have the first professional software cost estimators certified next year at the ICEAA conference in May 2019 in Tampa, FL and at the IWSM in November 2019 in Haarlem, the Netherlands’

Please check it out if you have the chance: https://www.linkedin.com/groups/37631/37631-6422686037318860800

 

It’s really incredible what this community of practitioners think of software cost estimation. Just a few of the reactions:

‘Why should we want to grow mature in wasting our time?’

‘Never impressed by accurate estimates on new development.’

‘Sorry if this comes across as sarcasm, but you have invented a more fancy crystal ball to produce more precise random numbers.’

‘I think your better off taking this to waterfall and project manager communities as agile communities are just going to laugh at this.’

‘ I’ve seen some steaming horse crap posted in this group, but this tops the pile’

‘Budget what you can afford. Have teams deliver continuously so you can see what you are getting for your money. Continuously improve. If you are not getting enough value, root cause why. ‘

‘Yet another Certification for hopefuls to buy… Just what this industry needs.’

 

And many more harsh and unfair comments, from the community that does not seem to care about the financial implications of software development at all (although there were also some supporting comments, I must admit).

Executives on this planet, please take note of these reactions, as they embody the true feelings and beliefs of many practitioners in the industry. They really feel they don’t need estimation (as it is not their money anyway) and they don’t wish to be accountable for anything, expressing velocity in ‘random, non-repeatable, non-standardized arbitrary units of measure (e.g., story points)’ and thus never accountable for anything. Throwing money at agile teams and hoping for the best might work in some cases but will disappoint you in most cases! ‘Software is an R&D process and results are unpredictable!’ This is not true! Software estimation is not easy, but it can be done in an accurate way, preventing a lot of schedule and budget overruns as we witness today!

Actually, this is the basic problem of the whole Software industry. By denying that it is possible to accurately measure the size of software requirements with the use of international standards, estimation maturity remains low. If you study the work of professor Abran, for instance his study on the prediction power of story points versus functional size measurement (https://tinyurl.com/y9jf98tq), you see that even the arbitrary effort unit of measurement invented for teams to remain incomparable and unaccountable (story points), does not nearly has as much prediction power when it comes to software estimation as functional size measurement does.

The fundamental question remains… do the executives in this world dealing with agile teams really keep on buying this? At some point you’d think they understand they are scammed into throwing more money at the teams with questionable results to follow: less functionality than expected/required, lower quality, higher cost, especially in maintenance.

ICEAA and Nesma are confident that the certified Software Cost Estimator will become a real profession in the industry, which is going to help organizations to improve their process maturity regarding estimation, control, performance measurement, benchmarking and sourcing. The Software Cost Estimator will help IT executives keeping grip on their agile delivery teams without bothering the teams with overhead and waste, while bringing enough transparency to make informed decisions.