18/10/2014 at 17:07 #2002
The software productivity community has come a long way in measuring the productivity of IT projects. In the beginning of the 1980’s measure productivity consisted of dividing program length by effort spent. In the 1990’s it became clear in industry that measuring productivity by counting program length had serious issues (c.f. Edsger Dijkstra’s note on counting lines of code). As a result alternative measures of output became popular, of which function points became the dominant standard.
Since the dominance of function point counting many incremental improvements have been introduced: the stricter codification of counting rules (leading to the Couting Practices Manuals of IFPUG and Nesma), the improvement of the measurement theoretical properties of function points and improvement of the measurement scale (leading to Mk II standard of the UKSMA), application of function point counting to earlier project phases (e.g. the quick counting rules of the NESMA counting rules), adoption of the counting rules for embedded systems (the COSMIC function point standard by COSMIC) and the partial automation of function point counting (the automated function point standard by OMG).
However these improvements have not addressed the fundamental issue that we are measuring IT productivity and not business productivity of IT projects.
Business productivity of IT projects can be defined as the degree to which IT projects contribute to the goals of the organization. IT projects should either decrease the costs of doing business (i.e. increase efficiency) or increase the value of the outputs of the organization (e.g. a better product of better service). In the past, when trying to analyze the outputs of IT projects, too much emphasis has been placed on the efficiency part (lower costs) and too little emphasis has been placed on the effect of IT projects on the value of the organization’s services or products.
I believe that this lack of emphasis on the added value of IT projects to the outputs of an organization might have to do with the fact that the value of improved products or services are only realized when the organization and its processes also change. This makes the attribution of business value to an IT project always slightly elusive.
Still when we do not measure the true value of an IT project to a business we will never be able to ensure that our IT projects are in effect adding value to the business. And without more focus on added value to the business IT will not matter to the business and the business will see it as a nuisance rather than an asset.
How can we measure the business productivity of IT?18/10/2014 at 17:35 #2005
Many thanks to Joost Schalken and Joost Koen for raising the subject in their workshop Business Productivity at the 2014 IWSM Mensura conference in Rotterdam.19/11/2014 at 10:00 #2215
This question is quite topical right now, particularly as more and more organizations are modernizing their development methods, and particularly the communication between business and IT. Agile and Lean development have in common to primarily improve effectiveness, i.e. developing (only) what’s needed now. But with, say, function points and person hours, we are primarily measuring efficiency. To illustrate: One project develops 100 FP of which only 60% are useful as-is when delivered. Another project develops 100 FP of which 90% are effective, but needs 20% more hours to do so. Which is “better”? And this question doesn’t even go as far as dealing with effectiveness on the business level. At the end of the day, you need to evaluate your business case.
I am also looking for a way to measure effectiveness.
Frank, could you share some results of the workshop? I’d be very interested.22/12/2014 at 12:12 #2583
Good point Andreas and I totally agree the different focus areas (efficiency/effectiveness) of these modern approaches . But I do think that there is always a tipping point for the cost being ‘instantly effective’ on the long run that is not sustainable to be viable.
I agree that the biggest changes of the mindset in measurement programs or processes should be in the area of business effectiveness and -goals and not only in the IT part, but I do think that it not a matter of “OR”.. but its effectiveness “AND” efficiency. IT is often a big part of Business, and with an inefficient process the business values and gains are lowered! An inefficient process can lead to longer delivery times, lower quality etc.. lowering also the time to market, customer satisfaction etc.
Also, which agile team on its own decides that the organization could better start a separate team to develop or implement another application as replacement for the one they are working on themselves? Or which agile team objectively can say that they cost 10x more than another team (no matter which valid arguments they have in their current situation!) while the business value or customer satisfaction (or other business measures the organization values most) for other teams are maybe only 1.5x lower?
My point here is that teams can be ‘learned’ a “business and end-to-end/client-to-client mindset” over multiple IT applications and business processes, but there will always be more abstract ‘levels’ or ‘views’ needed where other type of decision making is needed. And in these cases I honestly think that objective efficiency measures are as important as the effectiveness measures.
That said, I think the measures for business productivity are more dynamic as they can change depending on the goals at the point in time. And there is maybe also a difference between the business values/goals/productivity of IT, and the real business goals/values of the organization. The “business productivity of IT” may be very good at this point in time because the application is very flexible and supports the business processes very well with a very high customer satisfaction, but if the software platform needs to be decommissioned because the IT landscape needs to be more standardized because of a organization buyout, than what is the point of business productivity of IT without taking into account the real business goals at that time?
Another example, what is a good customer satisfaction for a limited customer base the organization currently has, when the strategic business goal is to gain a bigger customer database (grow market share, introduction in new markets, product diversification) and then in 2 years maybe improve quality again.
In the end I think there is not a very standardized way to measure business like productivity/goals of IT. And maybe there should not be? Because to be effective the measures need to be tailored to the current business strategy and goals, taking into account the complex and changing environment.16/05/2015 at 00:10 #3531
You’re probably right that standardizing the measurement of business productivity is very difficult. But it could be very useful if people with a business perspective use software metrics to compare their perception of business value to the value of the software. I just posted a separate topic on why people on the business side don’t “buy” Function Points, called “Developers don’t need Function Points“.17/05/2015 at 18:30 #3549
Harold van HeeringenParticipant
This topic is related to the question I have asked myself a few times recently: How to measure business value. Modern software development methodologies, like DevOps, focus on delivering business value. But what is that… business value? I used to think it was functionality… if a software team delivers more functionality, this should result in more business value, right? But maybe it’s more complicated, as some functionality may really make a difference (e.g. being able to actually buy something in your app) while other functionality is nice to have, but does not necessarily result in a sale (e.g. you can view the products in the app, but still need to call in your order through the phone). Also, would repairing a defect count as creating business value? Maybe a cosmetic one that no-one ever encounters not, but what about a blocking defect? Or should we regard solving defects not as creating business value, but rather as preventing business loss. I think this is a very interesting discussion and I hope to see some more thoughts in this topic!18/05/2015 at 23:27 #3563
A couple of months ago I was at a session of “Tomorrows IT leaders” in Amsterdam where the CIO of booking.com explained how they measure business value. They release a new feature (which they do up to sixty times a day) and roll it out to half of their customers and they measure the effect in booked properties. The cahnge that generated the most value was changing the text “Book now, pay when you stay” to “Book NOW, pay LATER”. This has nothing to do with functionality, but is a key element in the way booking.com is earning their business.
You must be logged in to reply to this topic.