Performance Assurance Framework
Author: Richard Wood BSc MRICS
Publication Date: 26/02/2016
Crossrail’s approach to performance measurement was based on 6 core areas aligned to corporate objectives, using traditional quantitative based KPIs but supplemented with qualitative based assessments of the supply-chain’s maturity to provide a much richer picture of supply-chain performance. This enabled the targeted sharing of knowledge and innovation to support supply-chain capability improvement and drove a 54% increase in the supply-chain’s performance levels over a 3 year period, and resulted in 14 out of 17 assessed contracts operating over-and-above the requirements of the contract. 8 critical success factors were identified which were key to this success. This document is relevant to any project developing performance measurement, innovation or knowledge sharing initiatives.
Read the full document
The Crossrail senior management team and their stakeholders required a mechanism for assuring that the 1st tier supply chain were performing to a level that would enable strategic objectives to be met – safely delivering a world-class railway on time and on budget, setting new standard for major project delivery, and leaving a legacy of improved supply-chain performance within the UK Infrastructure sector. There was a desire to operate a lean client team but also a recognition that Crossrail had to enable the supply chain to perform if they were to be successful in achieving the programmes desired outcomes.
Crossrail Performance Assurance Framework
In response to this challenge, the Crossrail Performance Assurance model was designed in the early part of 2012 following consultation with key members of the programme and supply chain, with the aim of:
- Aligning the model to Crossrail’s corporate objectives
- Integrating the model with existing audit regimes to improve efficiency of audit as a whole
- Providing an assurance mechanism for Crossrail’s Directors and Stakeholders in relation to the supply chain’s performance across the programme
- Providing a mechanism for objectively measuring performance, facilitating project and programme management action
- Enabling wider programme stakeholders, including Government, to benefit from robust and objective performance data
The resulting performance assurance model spanned 6 core areas, aligned to the corporate objectives as shown.
Under each of the six segments, a detailed suite of objective performance measures were defined covering two aspects:
Figure 1 – Performance Assurance Framework Model
- Inputs (Qualitative, or ‘lead’ indicators): Approach to the delivery, quality of systems and processes, competency, policies, procedures, management arrangements etc.
- Outputs (Quantitative, or ‘lag’ indicators): Traditional, quantitative KPI’s, measuring how the contract has performed.
Across these two aspects, performance is measured on a scale of 0 to 3 as shown in Figure 2:
Figure 2 – Performance Measurement Scale
Input and output data was gathered during a four period ‘assessment window’ as shown in Figure 3:
Figure 3 – Performance Assurance Periodic Timeframe
Output data was based upon information already required to be reported by contractors within the Works Information. To calculate the output performance level, data was averaged over the four period assessment window to ensure a representative sample. Data was then compared to pre-agreed performance levels to ascertain the performance assurance score. An example output measure for each of the six performance assurance functions and the corresponding performance levels are shown in the table below:
The Input maturity score was calculated based upon a collaborative review of processes, procedures and management arrangements with each of the 1st tier contractors during an on-site meeting, conducted by each of the specialist functions. A catalogue of objective performance measures was used to calculate performance levels. An example input measure for each of the six performance assurance functions and the corresponding performance levels are shown in the table below:
Measuring across these two scales provided more informative management data. Traditionally within the industry, performance has predominantly been measured using output measures – KPI’s, which are generally retrospective by nature. Measuring input maturity alongside the outputs helped to identify future risk by testing the maturity of the approach; the lower the input maturity, the more likely that the various functions would struggle to deal with future challenge and complexity, and outcomes could suffer. Conversely, outputs would improve should the functions have implemented robust input procedures. This is shown in Figure 1. This richer, forward-looking picture wouldn’t be identified just through recording traditional retrospective KPI’s, and presented the opportunity to affect future performance.
Figure 4 – Performance Assurance Scoring Framework
The two different types of factors were equally important in driving enhanced performance, as the correlation between the maturity of the processes and procedures and the measurable results of those processes facilitated the identification of best practice and opportunities for efficiency, and allowed targeted improvement actions to be allocated intelligently.
The model aggregated the qualitative and quantitative data into bi-annual performance reports including tailored improvement reports and supplier performance benchmarking. The performance assurance process is detailed within the Performance Assurance Procedure.
A separate micro-report has been developed to articulate how performance results were presented and published across the programme.
Figure 5 – Performance Assurance Scoring Example
The Performance Assurance process:
- Provided a platform which underpinned a 54% improvement in the average performance level between May 2012 and August 2015 from contractual compliance to ‘added-value’ levels. In August 2015, 14 of the total 17 contracts assessed were shown to be operating above the base requirements of the contract – achieved through the targeted sharing of best practice, promoting innovative approaches and providing detailed suggested performance improvement actions.
- Allowed the intelligent management of the programme through rich, objective and comparable performance data
- Stimulated innovation
- Improved collaboration and knowledge sharing across the programme
- Provided a platform for differentiation
- Generated a legacy of improved performance and capability
- Satisfied the assurance requirements of external stakeholder, sponsors and regulators
Critical Success Factors
The following critical success factors were identified in development and implementation of the Performance Assurance process:
The performance assurance model was optimised over time since inception with these critical success factors in mind, which resulted in a culture of performance improvement across both the Crossrail project management teams and main contractor teams, resulting in collaborative performance improvement planning. A key enabler to achieving this culture was to incentivise performance improvement through positive PR, both internally across the programme, and externally utilising external press agencies. This approach was successful, given the scale and industry profile of the Crossrail programme.
Recommendations For Future Projects and Programmes
Consideration should be given to the method of incentivising performance improvement and should incorporate the cultural aspects described above, rather than just utilising contractual bonuses.
Although the Crossrail performance assurance process focused on the performance of the supply chain, there was recognition that the Crossrail project management teams had a direct impact on the contractor’s ability to perform. Future assurance arrangements should consider the applicability of collective assurance covering both the client-side project management team, as well as the main contractor. Assurance activity could also extend to key members of the second tier supply chain.
Associate Director – Turner & Townsend
Head of Cost and Control, Crossrail
Richard has over 10 working on the UK’s major infrastructure projects and programmes, including leading an integrated cost and controls managed service on the £2.5bn T2 redevelopment scheme at Heathrow, and played a key role in establishing cost and controls processes on their £3bn Q6 programme. Richard has worked on Crossrail since 2014, and has held several positions over this time, including Head of Commercial Assurance and Head of Performance Assurance. Richard took on the role of Head of Cost and Controls in March 2017.