In today’s software industry, where dynamism, adaptability to change and competitiveness are the key words, measurements have become a necessary and important tool for the organizations. Using only the gut feeling or intuition when making decisions regarding company’s future development is no longer sufficient to support its growth objectives. We need to know how good we are in delivery quality to our customers, in managing deadlines, workload and resources and based on these to decide what to do, how to do it and when to act, without walking in the darkness of uncertainties and assumptions.
There is a tendency to avoid using measurements in the organization as it is assumed they are not 100% accurate. Measurements don’t need to be precise, therefore we refer to confidence intervals, numbers in ranges, thresholds, trends, and not necessarily to exact numbers. In order to reduce uncertainty sufficiently, most of the time we do not need that much amount of data. We’ll be surprised how much we can learn from a small amount of data that is understood and correctly used.
Before deciding to implement a Measurements process in an organization, it is important to have a clear picture of the company’s capabilities, its business objectives and strategy on medium and long term. If the organization does not plan ahead for what it wants to achieve in the future and what it needs to know, the collected data and the metrics are not likely to meet its needs. A Measurements process is a long term investment that will prove its real value after collecting a ﬁrst set of data, which, when successfully collected, will help to review objectives, to develop improvement plans and to determine the next set of metrics to be defined and implemented. The measurement process should be understood as a continuous cycle: measure, review, set goals, reﬁne, learn.
From my experience an effective solution for defining and implementing a measurement process in the organization involves two approaches:
1. Top-down approach, when the measurements needs are defined by the top management based on the company’s business objectives, high level targets and performance indicators;
2. Bottom-up approach, when the measurement needs are defined by the teams, at project level, based on project specific goals and objectives.
A measurement process is implemented successfully when the measurements collected from these two approaches meet and complement each other and provide valuable information for all levels in the organization: executives, managers and teams.
TOP DOWN APPROACH
First aspects to be considered when we implement a Measurement process using the top-down approach are the business objectives of the company. These need to be expressed in terms and indicators that are easily understood and can be further used as reference at projects level (e.g. profit margin, business growth rate, customer satisfaction, personnel turnover, quality indicators and so on).
It is important to reach a certain level of quality awareness and maturity mindset in the organization before implementing a Measurements process. Without these, the metrics will not be seen at their real value and will not be recognized as a tool for learning and improving. In addition, a set of work processes, tooling and standards need also to be in place. Roles and responsibilities need to be clearly defined and understood, the people involved in the measurements process to be properly trained and qualified for this area. Measurements Specialists should be able to change a complex perspective into a simple one and to help in translating a non-specific goal into a set of measurable dimensions. By exploring the facts in detail, they should be able to present alternative approaches that make quantitative evaluation possible, clear and highly meaningful. They should also be aware of the considerations according to which certain dimensions should not be measured from cost effectiveness reasons (e.g. the effort to measure is higher than the benefit of the measurement) or from ethical reasons.
After the Measurements process is defined and agreed with the top management, Measurement Specialists are responsible to:
- implement it in a planned and organized manner,
- ensure that metrics are collected uniformly and consistently,
- involve management and operational stuff at the appropriate times for analysis,
- ensure that metrics are meaningful and useful to different levels of the organization
BOTTOM UP APPROACH
In the start-up phase of a project, the Measurements specialist together with the Project Manager should assess the benefits of using the Measurements process and tailor it to projects’ needs. Next step is to guide the Project Manager and the team in setting the project’s objectives, identifying appropriate metrics to be collected and the way they will be analyzed to monitor the progress on the objectives.
An efficient way to guide and help the team identify the project’s objectives is by addressing questions like:
- Which are the key aspects that best characterize your project’s success?
- Which are the quality indicators for your project?
- What factors might impact the quality of the project in a significant way?
- How will you know that the customer will be happy with the product you deliver?
- At the end of the project how you will assess that it was a success or not?
- If the company would have a historical metrics repository which information would you like to know?
Sometimes it is useful to provide examples of common metrics to help the team formulate objectives and performance indicators.
Once we had identified projects’ measurable objectives and agree them with the team, next step is to define the data to be collected, collection frequency, tools and standards to be used, the ones responsible for collection and analysis and how the results are going to be shared with the team.
My recommendation is that Measurements specialist should regularly support the Project Manager and Team Leader in the analysis of the data collected and make sure they are accurate and consistent. Measurements specialist can also participate from time to time in teams’ retrospectives to see how measurements and trends are being used to improve and how (s)he can support the process further.
The effort involved in the collection of metrics at project level (time spent, investment in training and tooling) should be evaluated not only from the project’s cost perspective but also from the perspective of the additional value they bring to the organization. Some metrics might be used exclusively for the project while others can be also used at company level. It is the Measurements specialist’s job to ensure that project metrics that bring added value at company level are selected and stored in the organization’s metrics repository.
THE ADDED VALUE OF MEASUREMENTS PROCESS
When a Measurements process is in place for a longer period of time (more than a year), consistent historical data are made available in the company’s metrics repository. From this point further the Measurements process can be used at its real value:
- recommended thresholds for different indicators can be identified based on trends
- charts, dashboards can be used and made available to interested stakeholders (even to the client)
- one metric value can be analyzed in correlation with other metric(s) values and specific improvement actions can be defined,
- decision factors can use some of the metrics in Pre-sales to increase competitiveness
- project teams are challenged to set more ambitious objectives, learning from the past and continuously improving
For an agile company with an improvement mindset, Measurements process will not stop here. Current metrics and tools requires continuous refinement. While new metrics are made available to the company repository, they need to be analyzed and weighed against the business objectives and market positioning. While measurements’ capability and expertise of the company develops and improves, more efficient and intelligent tools can be implemented and used as support in metrics collection and analysis.
A company that continuously monitors and effectively manages its metrics system will find and use it as a unique source of competitive advantage.
This post was originally posted on LinkedIn on October 23, 2016
About the author
Ramona Muntean is responsible for performance measurement and the implementation of best practices to support projects in processes improvement at ISDC (a member of the Endava group), a member of the Endava Group in Romania. In her day-to-day activities she identifies best practices, extract lessons learnt and support the improvement process in the teams and further on at corporate level.