Many organizations outsource (parts of) their IT services. To select a supplier these organizations use as decision criteria: Price and Quality (measured as timely delivery, scope and number of defects found during acceptance testing). They trust that they can trust the quality / price ratio as the best decision factor. Unfortunately in practice this means that they often do NOT select the best supplier. The chosen suppliers deliver often failing projects. They exceed the agreed timeframe, cost much more than the original price offer and deliver less products, functionality or quality as expected. The Dutch government has conducted a survey (see “Commission Elias”) on the topic why ICT projects fail and why these ICT project has such large budget overruns.
In our opinion the Dutch government will only succeed with a professional transparent cost calculation. Otherwise the list of failed projects will continue to grow. This article describes how to solve this.
The client organization
Your organization is considering outsourcing an ICT project to a reliable IT supplier, but which IT supplier should be selected? You decide to use quality and price as the selection criteria. Quality is the most important factor and therefore, you decide 70% of the score is determined by quality. Of course, it should also be done at a reasonable price, therefore, 30% of the score is determined by price. To score quality you generate smart questions to verify the quality and you send the request for proposal to potential ICT suppliers. After some time you receive their proposals and your team starts evaluating these offers.
After many considerations your team has found the following results:
|Quality score||Price||Quality points*||Price points**||Total points|
|Supplier A||10||€ 200,000||7||1.5||8.5|
|Supplier B||9||€ 150,000||6.3||2||8.3|
|Supplier C||8||€ 100,000||5.6||3||8.6|
|Supplier D||10||€ 500,000||7||0.6||7.6|
* 70% of the Quality score
** 30% of the (Lowest Price / Price)
Supplier C has the most points (8.6) and wins the bid. You are confident that you have taken the right decision and together you uncork the champagne bottles and start with the project.
During the project you notice that the costs slowly start to rise above the expected € 100.000. The interface with your back-office system is not included and this necessary change request will raise the project price with € 50,000. In addition, the supplier has also not included the database conversion in its winning price. Yes, the supplier had not explicitly described in his proposal that the database conversion was excluded, but you originally thought that this was included in the proposal. But, this is necessary and the project price is raised again with € 100 000. The project is slowly getting very expensive.
A month later it turns out that the chosen (cheap) server is no longer on the market and a much more expensive one has to be bought. Furthermore some licenses are not included in the winning bid price and due to a problem with the test environment, the project will be delayed with some months. The project costs are now € 600,000, – an overrun of more than € 500,000, – and it is not clear what the total overrun cost and delay will be.
The project is clearly out of control and you ask an external specialist (Frank Vogelezang) for advice. Frank the external consultant looks at the proposal process. He noticed that all suppliers scored on quality close to each other and that the lowest price is apparently the main reason for the choice of the supplier. The difference in quality is only 1.4 points while the difference in price is 2.4 points.
Both supplier A and D have the highest scores for quality (10). But these suppliers are not chosen because of their price. Frank reminds you subtly that the original price of supplier D (€ 500,000) is now much lower than the current and rising project price. You think by yourself that supplier D would also have a similar cost overrun as your current supplier C.
But is this true?
Comparing suppliers C and D
Both supplier C and supplier D are very well aware of the project conditions and they both know the goals you want to achieve, the technology environment and the risks.
Supplier C has as sales strategy to offer the lowest bid price. Supplier D is still new and naive and thinks that a sales strategy focusing on quality will win the bid. Supplier D has not much experience with this kind of procurement processes and has not yet learned that most of the (government) contracts are won by the lowest price.
To reach the winning price supplier C uses several options within the contract. These are:
- Do not include what not explicitly is requested in the offer
- Eliminate supplier risks and give them to the contractor
Supplier D knows very well which requirements are lacking in the request for proposal, but unlike supplier C, supplier D will take the additional necessary cost components in the total price.
Supplier D does a complete risk assessment and takes measurements for some of the major risks although this will raise his project price. These measures ensure that the impact of these risks disappears or at least are decreased. For some risks he makes a financial reservation (contingency). The contingency budget for the risks should be sufficient in case one or two of those risks actually occurs.
Supplier C has used a different approach to the risks. He eliminates all risks in his offer. Not very explicit, but using conditions and assumptions he can manage that these risks will become your (contractor) risks so that he does not run into financial risks. Rather he had not done it this way but the competition is fierce. Supplier A and supplier B have won previous tenders and supplier C knows now that he had to offer a sharper price to win this deal.
To win the deal, the vendors have used different winning strategies. The result for the contractor is that they have to choose for a supplier that will be much more expensive, will not deliver in the agreed timeframe and probably will deliver less in the end than suggested by the original offered proposal.
Moral of the story
Above situation is fictional but unfortunately this is a real practice. Supplier D has to change his way of offering or find another way to deliver a competing price so he can survive. A high quality strategy will not result in winning bids, only with a low price strategy this will be achieved. Although the client has the intention to select suppliers on quality attributes, unfortunately by this kind of request for proposals the selection of suppliers will be decided only on price and not on quality. Supplier D will ultimately face bankruptcy unless he is going to act like its competitors.
And what does this way of outsourcing do to the client? The client continues to face many additional cost overruns which may be substantial. According to the commission Elias (of the Dutch government) the government alone spills every year between 1 and 5 billion Euros because of overruns of their ICT projects. We can therefore assume that the total amount of additional cost runs into the billions.
Price as single criteria alone is not good enough. To assess price as a selection criteria it is required to include all relevant elements which determine the price. If our client had known what products and services where included in the price he/she could have made a better decision. Supplier D would then be the best choice. To be able to compare different suppliers on price and all relevant elements there must be a standard guideline. This guidelines is the “Basis of Estimate” or AACE®, the international association of cost engineers. The standard known as AACE® International Recommended Practice No. 74R-13.
AACE is best known for large engineering projects of many millions and billions of euros. The “Basis of Estimate” as applied for the software services industries is relatively new and unknown in the world of ICT. Benefits of the “Basis of Estimates” (BoE) for ICT proposals are:
- BoE helps to make complete and reliable supplier calculations (professional calculations) – this is an advantage for the supplier and the contractor;
- BoE gives transparent and comparable costs (through a generic reply template with the BoE elements) – an advantage for the contractor;
- BoE has no influence on acceptable commercial concessions by the supplier;
- BoE limits the opportunities to use cost as the main winning strategy for projects because it makes clear what is included and what is not included in the price.
Contractors just need to demand for specified open cost calculations in tenders that are setup according to the guideline of the BoE. For a level playing field this is also better for suppliers. At this moment suppliers needs to masks the risks for the contractor. More transparency means that they will include these (commercial) risks into the offer. More transparency will be welcomed by all parties if everyone are obliged to do this. This only requires that contractors imposed this on suppliers.
The Basis of Estimate material is free and available on this website or at the AACE foundation.
The Basis of Estimate by the NESMA® (Netherlands Software Metrics Association) developed in collaboration with MAIN® (Measurement Association International Network) and AACE®, the International Association of cost engineers.
About the author
Jelle de Vries is a group member of the BoE NESMA and Principal Consultant at Ordina. This article was developed in collaboration with Marten Eisma (CGI) and Ray Sadal (Capgemini) and was originally published in Dutch in Outsource Magazine.
A blog post represents the personal opinion of the authors
and may not necessarily coincide with official Nesma policies.