Performance of the Crossrail team and the supply chain in getting things right first time (RFT) is considered vital to the success of Crossrail.  Failing to perform affects cost adversely, causes delays to the programme and impairs the quality of the delivered railway.

Prior to commencement of construction it was realised that a robust ‘quality’-related reporting process with a meaningful set of metrics which were common to all construction contracts was required.

This resulted in the development of the quality dashboard which has been issued on a periodic (4 weekly) basis since construction started.  It offers a clear and concise set of performance data, for the Crossrail team and Tier 1 Contractors, which is utilised to identify areas of weakness, key issues and also high performance.  This enables management to identify trends and make informed decisions to rectify problems, prevent their recurrence and provide a platform to help drive continual improvement.

The content of the quality dashboard with explanation of the reporting areas and key metrics and targets utilised are described below.  There are two Corporate Quality Key Performance Indicators and four Right First Time Metrics.

Right First Time Metric 1 (RFT1) – Inspection and Test Plan Performance (ITP)

Inspection and Test Plans are the cornerstone of the assurance regime on Crossrail. They are a key planning tool considered equally as important as a Method Statement and are essential in helping ensure that the programme is built ‘Right First Time’ and are able to demonstrate by certifying the Works that they have been built correctly.

Metric Definition:

This metric measures the number of incidents where there was no accepted Inspection and Test Plan (ITP) in place for the actual or planned start of work. An incident where the planned start of work has been delayed due to an ITP not being in place will also be counted within this metric.

An accepted ITP is defined as having an ITP in place before the commencement of a new activity that has been coded either Code 1 or Code 2.

Target:

The target for this metric is zero incidents.

Right First Time Metric 2 (RFT2) – Partial Rework Rate

Recent construction industry estimates of the cost of rework put the direct cost in the region of 3% of total contract value for building projects and 12% for civil projects [1][2][3].  This would be an estimated amount of £400M for Crossrail which equates to the cost of a new station. In an attempt to focus minds on the cost of not getting it ’Right First Time’ a system for capturing costs of rectifying each non-conformance, the cost of rework was developed.

Metric Definition:

This metric measures the partial cost of rework in a period divided by the Actual Cost of Works Performed (ACWP) in the period, expressed as a percentage.

The partial cost of rework is defined as the cost of extra work performed to rectify a non-conformance. It is termed ‘partial’ because the true cost of the extra work can sometimes be impossible to quantify and/or it excludes certain factors.

Target:

The target for this metric is below 5% for Systemwide and Stations and Other Contracts and below 0.50% for Bored Tunnels.

Metric Calculation:

The Rework Rate measures the partial cost of rework divided by the Actual Cost of Works Performed (ACWP), expressed as a percentage. Using the data for Rework Severity Categories 1, 2, 3 and 4 (as shown on the table below) the Supervisor’s Representative (PFE) calculates the Rework Rate as:

% Rework Rate =                 Total of the OMEs for NCRs in the period   X 100%

(for the period)                  Total of Actual Cost of Works Performed in the Period

Category 1 NCRs are to be individually reported for review.

Order Of Magnitude (OME)

Right First Time Metric 3 (RFT3) – Satisfactory Observations

Observations are defined as site monitoring / surveillance of the work in progress, as witnessed by members of the Crossrail Field Engineering Team, in order to assure that the works are being completed as specified.  The number of Observation activities completed depends on the scale and complexity of current activities.  An average of 500 No. observations have been completed by a team of 120 Field Engineers on a periodic (4 weekly) basis.

Metric Definition:

This metric measures the number of satisfactory observations of the work in progress, as witnessed by members of the Delivery Team.

Observations are defined as a site observation of the work in progress in order to assure oneself that the works are progressing in a satisfactory manner such that the product requirements will be achieved at inspection. In other words, observations are a means to assess the capability of the Contractor(s) to achieve the required quality right the first time.

Target:

The target for satisfactory observations is 90%.

Metric Calculation:

Each observation will be scored as follows:

The overall percentage satisfaction score for each Contractor will then be calculated according to the following formula:

((Total Score) / (Total Number of Observations)) x 100%

The total score is the sum of scores achieved for the observations in a period, and the total number of observations that were performed in that same period.

For example, here is a scenario where there were a total of 20 observations carried out, of which 12 were satisfactory, 3 were unsatisfactory but did not need an NCR, and 5 were unsatisfactory and required an NCR:

((12+1.5+0)/20) x 100% = 67.5%

Right First Time Metric 4 (RFT4) – Contractor Non Conformance Reports

It is very important that errors in workmanship are reported, allowing them to be dealt with in a controlled, transparent and efficient manner, rather than being swept under the carpet. Contractors who do raise them ahead of  the Crossrail team are applauded. This metric was established to help drive the correct behaviours on the project by encouraging Contractors to raise NCRs when a mistake/issue happens.

Metric Definition:

This metric measures the percentage of Non Conformance Reports (NCRs) that were first raised by the Contractor.

NCRs first raised by the Contractor are defined as those NCRs that were initiated by a member of the Contractor’s team for a defect found, before that defect was observed by a member of the Crossrail team.

Target:

The target for Non Conformance Reports is 95%.

Corporate Quality Key Performance Indicator (WC1) – Certification completed on time

Crossrail is building two railways, the physical and the digital railway. The Programme Delivery Agreement (PDA) requires that Crossrail is progressively assured as it is built. Certification documents (of which there are over 3000) are the key deliverables that demonstrate assurance of the railway. Therefore monitoring production and acceptance of  certification packages make this KPI extremely important.

Metric Definition:

This metric measures the certification packages to be submitted and accepted within 60 days from the forecast submission date. Target is 90% of packages to be accepted on time.

Each package has a forecast submission date and an automatic acceptance date of +60days. Any package accepted on or up to the target date is ‘On time’ and any package which is not accepted by the target date (>61 days) is ‘not on time’.

Target:

The target for the Corporate Quality Key Performance Indicator is 90% of packages to be accepted on time.

Example:

33 packages are due for acceptance within Period 9. 32 packages were accepted on or prior to their target date. 1 package was accepted five days late. Therefore the KPI result for Period 9 is 32/33 (97%), which is above the target. The cumulative results are 404 packages due from P1-P9, 380 accepted on time. The cumulative total to date would be 380/404 is 94%.

Corporate Quality Key Performance Indicator (WC2) – CARs closed within due date

Corrective Action Requests (CAR’s) are raised during internal (Crossrail ) Audits and Supply Chain – Contractor audits.  They are raised as a result of a deficiency in contractual or procedural requirements.  Each CAR requires action to be taken within agreed timescales to rectify the deficiency and to prevent its recurrence.

Metric Definition:

This metric measures the number of Crossrail raised audit Corrective Actions Requests (CARs) closed within the due date.

The Corrective Actions Requests KPI is a ratio of the number of CARs that were closed within the due date compared to the total number of CARs that were due. If a CAR has gone beyond its closure date it is classified as overdue. The CAR metrics is calculated as follows:

((Total number of CARs closed within the due date)/(Total number of CARs due)) x 100%

Target:

The target for the Corporate Quality Key Performance Indicator is 90%.

Example:

196 CARs due for closure between P1-P9, 172 CARs were closed within their due date and 24 CARs were overdue. The cumulative metric would be 172/196 (88%), which does not meet the target of 90%.

What is the Benefit to Future Projects?

Future projects will be able to leverage the existing best practice metrics and targets to optimise delivery and tracking. The data collected on performance will be comparable against the metrics collected for Crossrail to identify opportunities for improvement.

References

[1] A Guide to Construction Rework Reduction, Construction Industry
Institute, July 2011
[2] A Rework Reduction Model for Construction Projects, P Love et al,
IEEE Transactions in Engineering Management, Nov 2004
[3] Measuring the Impact of Rework on Construction Cost Performance, B
Hwang et al, Journal of Construction Engineering& Management , ASCE March 2009