How to bring objective metrics into application development team performance (and why you’d want to)

Many senior managers who have gone through a transformation from a traditional software development organization to an organization with self-organizing Agile teams, understand that this was a necessary exercise to ensure that value is delivered faster to their everchanging business environment. Where the survival of companies is often a direct result of delivering new user functionality faster than the competition, these managers are usually happy they went through this, but they also recognize that adopting an Agile way of working is not a silver bullet or a guarantee for success.

”Although the concept of self-organizing teams sounds great, they still need to be managed in the context of the entire organization, its mission, vision and strategy.”

Senior management needs to make decisions regarding budgets, investments, staffing, what to do, and what not to do. They need to decide on the strategic directions, the targets and goals, the operating model but also monitor progress and make corrections when necessary.

What we see in the market is that organizations struggle with the latter, and this is largely due to a lack of objective management information. Many Agile teams use progress metrics that make sense in their team, but which can’t be aggregated into useful management information. However, it’s important to understand the performance of the Agile teams, as they are providing the value to the business and its customers. We see that companies operate on different levels of maturity when it comes to the control they have over their application development teams. The question is: on which maturity level is your company?

A well-known model to look at maturity of processes is the Capability Maturity Model Integration (CMMi) model.  This model assesses the development processes on 5 different levels (see figure).

CMMi maturity levels

This is a good principle and can be used to assess control over Agile development teams as well. The Agile Control Maturity Model (ACMM), based on the CMMi levels, allows senior management to understand the control level regarding their agile teams and the value they deliver. The following table shows the way IDC Metri looks at the different levels of Agile control maturity.

Low maturity levels (0, 1 and 2)

Most organizations currently are on level 0 or 1. They went through the transformation, and CIO’s and other senior management got advice from Agile coaches and other consultants on how they should manage the new organization where self-organizing teams have a lot of power to decide, and the business is very closely working with the teams to create value. However, on level 0 or 1, you have no idea how much value is created for the budget spent, and how this relates to the competition. Furthermore, predictability is low and it’s hard to manage the interdependencies between teams and to forecast when certain pieces of software will be ready. External teams are contracted based on time and materials (hourly rates) and not output-based (maximum value for money).

High maturity levels (3, 4 and 5)

As an organization gets to a higher maturity level, senior management gets more and more in control. Objective metrics are displayed in up-to-date dashboards, enabling them to make decisions based on facts. This brings along many benefits, like for instance:

  • The ability to demonstrate to stakeholders that you (the CIO and Senior Manager) is in control of the performance and value creation of the Agile teams.
  • Focus on actual value creation by improving productivity and delivery speed, while improving the quality of the product.
  • Objective measurements are used to understand which are the high performing teams and which are the low performing teams. Then the reasons this can be investigated and improvement actions can be carried out.
  • Software Cost Estimation accuracy improves enormously, as data from the  organization and teams can be used to calibrate the parametric models.
  • Leading indicators result in increased predictability of the teams, eliminating schedule or cost slippages.
  • Based on objective measurements and targeted removal of critical violations, the risk in the application portfolio reduces significantly while quality, maintainability and cost levels improve.
  • Output-based metrics can be used in contracting Agile teams, enabling organizations to select the team that provides the most value for money (productivity times average rate) instead of the cheapest one that may not be productive at all.

And there are many more benefits.

In the next figure, typical productivity improvements we observed are shown, where 0% is the market average for comparable teams/applications.

Being an expensive function, Agile development teams should be managed to ensure value is optimized for the investment. But, how do you measure the value? When IT leaders are struggling with this common challenge, IDC Metri has prepared a checklist to help you manage and measure your teams’ performance. Click here to download the checklist.


Harold van Heeringen – Principal ConsultantHarold van Heeringen

Harold van Heeringen graduated from the university of Groningen (the Netherlands) in business economics in 1997 after which he worked for IT service provider Sogeti for 17 years as a senior consultant software metrics/software cost estimation. In 2015, he moved to sourcing advice and benchmarking services provider Metri, where he works as a principal consultant and practice lead for IT Intelligence services, including Estimation & Performance Measurement and Agile Value Management services. Next to working in IDC Metri, he is active in several not-for-profit organizations that are active in this space. Harold has been president of the not-for-profit International Software Benchmarking Standards Group (ISBSG) from 2011 until 2019 and is still an active director in the board. Harold is also board member of Nesma, the international organization focusing on functional size measurement and cost estimation. In Nesma, he is responsible for international partnerships and cooperation. Harold is one of the Nesma delegates in the team that has created the Software Cost Estimation Body of Knowledge (SCEBoK). Harold regularly publishes white papers, blogs and articles on software metrics, cost estimation, agile value management, performance measurement, benchmarking and related topics.