Introduction

Traditionally construction industry contractors have been reticent to identify defects and report them either within their own organisation or to their client, in fear that they would be hit with large rework costs, penalties and loss of reputation. This has resulted in a negative culture within construction industry contractor teams who would prefer to bury bad news where possible by “sweeping issues under the carpet”, only for them to arise as much more serious problems years later when the completed asset is in operation, bringing with it potentially serious safety issues to users and the public.

It was realised during early days on the Crossrail Programme, and from Lessons Learned gained from the previous HS1 (High Speed 1) Channel Tunnel Rail Link Project that a step change improvement in the process of notifying and dealing with Defects in an open, honest, and transparent way is what the industry needed.

Crossrail introduced a new approach that incentivised Contractors to highlight and notify ALL defects promptly, which, as they were then known, enabled them to be dealt with in a transparent and efficient way.  This meant that, most importantly, the end-product was built technically correct, and also that by analysis of ongoing defect data action could be taken to prevent issues recurring. This helped to dramatically reduce the industry typical rework cost of 6-10% of project cost down to below 1%.

Through a commercial mechanism within an amended version of the NEC 3 (New Engineering Contract) Contract, Contractors were incentivised to ‘race’ to notify defects to the client as soon as they occurred. If the Contractor notified a Defect, they were in effect paid to rectify it. However, if the Client notified a Defect in advance of the Contractor the costs to rectify would be disallowed, meaning the Contractor would be liable to pay for the rework in full. 89% of over 25,000 Defects raised during 12 years of construction were notified by the Contractor.

Below are some key statistics taken from the Crossrail programme:

• At the peak of the programme, over 100 NCRs were raised each week.
• There have been over 25,000 NCRs raised across the Crossrail programme.
• 89% of the NCRs raised were by the Tier 1 Contractor teams.
• The partial cost of rework was 0.83% (Total Partial Rework Cost / Total Actual Cost of Work Performed x 100).

The Crossrail NCR system allows each NCR to be triaged and categorised via the “NCR Classification” field. The top three categories of NCR from the Crossrail NCR system were:

1. Workmanship
2. Incorrect Installation
3. Design

Once the NCR has been processed and the investigation has concluded, the root cause is assigned via the “Root Cause Category” field. The top three root causes from the Crossrail NCR system were:

1. Discipline/Workmanship – Defective construction
2. Discipline/Workmanship – Other
3. Design Problem – Other

During the programme lifecycle, data from the Crossrail NCR system was exported and reviewed to formulate a weekly/periodic quality report. During this exercise the data was reviewed, and new fault categories (see Appendix A – Fault Categories) were assigned, and emerging trends were shared across the programme. The top trends from the weekly/periodic quality report data are shown below within Figure 1 – Fault Categories Pareto along with Figure 2 – Analysis into top trend:

Figure 1 – Fault Categories Pareto

Figure 2 – Analysis into top trend

Since the Elizabeth Line opened in May 2022, a very low level of defects has emerged. This is testament to the world leading Defect reporting culture and processes implemented, which ensured that the completed railway is fully technically compliant and safe to operate. This paper sets out in more detail how this achievement was realised focussing on the following areas:

• Works Information (WI) requirements placed upon the supply chain
• Crossrail NCR system hosted within eB (Enterprise Bridge – Document Control and Management Solution) supplied by Bentley Systems
• Crossrail and Contractor roles and responsibilities
• Programme reporting
• Lessons Learned for Future Projects

Works Information (WI) Requirements

Crossrail developed its Works Information (WI) requirements using the Channel Tunnel Rail Link Works Information as a baseline. The requirements relating to NCR’s are located within the following books and were cascaded throughout Crossrail’s supply chain:

• Works Information: Volume 1
• Works Information: Volume 2B

A selection of the key clauses relating to the culture for identification, raising and closing NCR’s has been listed within this section.

WI: Volume 1

Volume 1 includes the NEC3 Contract and sets out from the very beginning that the Employer, the Contractor, the Project Manager, and the Supervisor shall act in the spirit of mutual trust and co-operation. This statement underpinned the open culture necessary to support the identification, raising and resolution of NCRs during the Crossrail programme.

Volume 1 Clause 11.2 (19) provides a clear definition of a “Defect” to avoid ambiguity:

11.2 (19)

“A Defect is

• a part of the works which is not in accordance with the Works Information or
• a part of the works designed by the Contractor which is not in accordance with this contract or the Applicable Law or the Contractor’s design which the Project Manager has accepted.”

In order to encourage the Contractor to proactively raise NCRs, Volume 1 Clause 11.2 (24) shown below provides the contractual incentive. Otherwise, there would be risk of disallowed cost.

11.2 (24)

“Disallowed Cost is cost which the Project Manager decides… the cost of:

• correcting Defects notified to the Contractor by the Supervisor which the Contractor failed previously to notify to the Supervisor having had the opportunity to notify such Defect,”

To reinforce the openness for raising NCRs, clause 42.2 states:

“Until the defects date*, the Supervisor notifies the Contractor of each Defect as soon as he finds it and the Contractor notifies the Supervisor of each Defect as soon as he finds it.”

* The defects date is typically 52 weeks after the completion of the works; however, the number of weeks varies on each contract.

Given the complexity of the programme, Tier 1s would need to subcontract elements of the works out to subcontractors. Volume 1 Clause 26.1 provides clear direction on ensuring the core requirements are also cascaded through the Tier 1s supply chain to ensure the requirements are not weakened:

“If the Contractor subcontracts work he is responsible for Providing the Works as if he had not subcontracted. This contract applies as if the employees and equipment of a subcontractor or supplier of any tier were the Contractor’s.”

WI: Volume 2B

Volume 2B, Part 20, clause 20.8 focuses on Inspection and Testing. Within this section of the requirements, it touches upon the need to capture the close out of NCRs during the completion of Inspection and Test Plans (ITPs). Furthermore, the section provides guidelines around exactly when an NCR should be raised:

“Where a Defect in a works item is noted that cannot be put back in compliance within the same shift the Contractor shall raise a NCR. Where a specified work activity has not been carried out in accordance with agreed procedural requirements, a NCR is raised.”

To ensure full transparency across the programme, Crossrail mandated that each NCR be entered within the Crossrail NCR system.

“Unless otherwise accepted by the Supervisor, the Contractor shall enter each NCR, including sub-contractor NCRs, in the PTR* system”

* Project Technical Requests (PTRs): An automation tool used to initiate, route, track, and approve Requests for Information (RFI), Field Change Document (FCDs), and Non-Conformance Reports (NCRs).

Within Appendix 20A of the Works Information an application matrix is provided detailing when an NCR is used, when it is applicable, who it is raised by, the closure and some examples.

Figure 3 – Extract of WI Appendix 20A

In the event of concessions or design changes, the section also mandates that these shall be referred to the Project Manager by the Contractor for appropriate resolution:

“Each NCR requiring a concession or design change shall be referred to the Project Manager by the Contractor for appropriate resolution. Any agreed remedial action shall be completed prior to the commencement of any further activities that may render the non-conforming item inaccessible, difficult to repair or increase the cost of the repair.”

To reinforce the culture of raising and closing NCRs in a timely manner, Volume 2B, Part 20, clause 20.9 focuses on Contractor self-certification and the potential for the Project Manager to issue a notification of a defective quality management system. Upon receipt of this notification, the Contractor would be required to formulate a SMART action plan to address the points raised within the notification and this would be tracked until all actions are closed:

“If the Contractor fails to demonstrate that specified requirements are being met, the Project Manager may notify the Contractor that its Quality Management System is defective. A defective Quality Management System is demonstrated by any of the following:

• Defects and/or Non-conformities not being identified by the Contractor in a timely manner;
• Defects and/or Non-conformities not being resolved in a timely manner, including failure to meet criteria for clearance identified in the Works Information;
• failure to prevent recurring Non-conformities;”

Crossrail NCR System

If a nonconformity is identified outside an audit, it is recorded and transmitted to the relevant process owner using the Project Technical Request module within the Bentley Systems eB (Enterprise Bridge – Document Control and Management Solution).

The general workflow for the NCR module within eB is shown within Figure 4. Specific Roles and Responsibilities are defined further on within this case study.

Figure 4 – NCR workflow

There are four paths the NCR could be directed:

• Repair: The process of restoring a nonconforming characteristic to function in a safe and reliable condition, even though that item still does not conform to the original requirement.
• Rework: The process by which an item is made to conform to original requirements by completion or correction.
• Reject: The process of eliminating a nonconforming item from its intended use. Rejected items may be discarded, destroyed or downgraded when the downgraded item can be clearly identified and controlled.
• Use-As-Is: A disposition permitted for a nonconforming item when it has been established that the item is satisfactory for its intended use.

A Defect can be identified at three stages in the NCR process:

1. Initiate: The Crossrail or Contractor Initiator initiates an NCR and decides that the non-conformance constitutes a Defect.
2. Propose Action: The Contractor proposing the action decides that the non-conformance constitutes a Defect.
3. Review: The Supervisor’s Delegated Representative reviews the NCR and identifies that the non-conformance constitutes a Defect.

As shown in Figure 1, input is required for NCRs selected as Repair and Use-As-Is from the Engineering team to endorse and record the decision. Whereas the Rework and Reject options are routed straight to the implementation stage to resolve, after which the NCR will be inspected and closed out if sufficient evidence is provided. All evidence is maintained within the NCR system as documented information.

All NCRs are allocated a unique identifier where “CXXX” denotes the contract reference and “000xyz” is a sequential number allocated automatically once the NCR has been initiated:

CXXX-NCR-000xyz

Once the NCR has been initiated, the user would complete numerous fields (some being mandatory) which would capture details such as contract reference, a headline summary of the issue and details of the nonconformity. Appendix B- Sample NCR form.

During the initiation of the NCR, the user categorises the NCR based on the information available to them at that moment in time within the “NCR classification” field. These are shown below within Figure 5. Only once the investigation into the root cause has been concluded, the Supervisor’s Representative from the Crossrail Delivery Team can assign the root cause within the “Root Cause Category” field.

Figure 5 – NCR classification

Each NCR is categorised based on a cost estimation by the Supervisor’s Representative from the Crossrail Delivery Team. The NCR cost categories are shown within Figure 6 – NCR Severity Categories. The NCR severity category is assigned at the Review stage along with an estimated cost.

Figure 6 – NCR Severity Categories

While it is Crossrail’s belief that all non-conformances (NCRs) are preventable, it was decided to focus particularly on  high profile, or recurring NCRs.A Quality Incident Event Procedure (QIEP) was developed to ensure high cost, safety related or recurring NCRs were corrected and the root cause was established in an agreed timescale to avoid recurrence. In addition to this, where necessary, a quality alert was distributed out to the programme; see Appendix C – Sample Quality Alert.

A QIEP NCR is defined as:

• CAT 1 NCRs
• Potential/suspected CAT 1 NCRs
• NCRs with a major safety risk, or
• A number of recurring CAT 2 NCRs that are causing concern due to their frequency.

The QIEP is supported by senior management to drive the issue to conclusion and share knowledge across the programme where applicable. As the process evolved, the QIEP ensured an executive team member was selected and acted as a sponsor. This significantly helped drive the successful conclusion of the QIEP and helped removed any blockers along the way.

The general process and timescales are shown in Figure 7 – QIEP overview.

Figure 7 – QIEP overview

The Root Cause Analysis (RCA) is undertaken via the Crossrail SPOT-On (SPOT-On: Specific Problem Obstacle Tool – On Non-conformances) process, where Crossrail delivered an extensive training programme to the project. In total468 Crossrail and Contractor staff were trained, 36 of whom chose to obtain certification as Root Cause Analysis Practitioners via independent assessment of their root cause analysis report by the British Quality Foundation.

The SPOT-On process was derived from the 8D (Eight Disciplines of problem solving) and lean six sigma DMAIC (Define, Measure, Analyse, Improve and Control) principles.

To support the learning legacy, the following documents are included within the appendices:

• Appendix D – Quality Incident Event Process (QIEP) Procedure
• Appendix E – (QIEP) CAT 1 NCR – Incident Investigation Report
• Appendix F – Specific Problem Obstacle Tool (SPOT-On) Template

Crossrail and Contractor roles and responsibilities

There were a number of roles which had a part to play in the initiation, processing and closure of NCRs within the Crossrail NCR system. The high-level organisation corresponding with the New Engineering Contract (NEC) contractual roles is shown in Figure 8.

Figure 8 – Contractual roles of individuals.

See below for more detailed descriptions of the key roles involved within the NCR system:

Contractor

• The organisation and team responsible for the delivery of the Works Information. Crossrail’s NCR system was typically used by various Contractor personnel.

Employer

• The Employer provides information which the contract requires him to provide in accordance with the Accepted Programme.

Project Manager (PM)

• It is the project manager’s duty to manage the contract on behalf of the employer, achieving their objectives on budget, programme, to specification and quality.

Supervisor

• The prime purpose of the Supervisor is to ensure on behalf of the Employer, that the works are provided to the standard and performance required in the Works Information.
• Witness and/or carry out tests and inspections.
• Notification of Defects.

Supervisor’s Representative (SR) (previously known as Project Field Engineer (PFE))

• The Supervisor delegates day to day responsibility to the Supervisor’s Representative (the Project Field Engineer) in accordance with the letter of delegation.
• To act on behalf of the supervisor for the assigned NEC3 contract(s), responsible for ensuring Crossrail’s compliance with sponsors’ requirements and the Crossrail assurance process. overseeing the technical aspects of construction, testing and commissioning activities within area of responsibility, supported by Field Engineers for each contract, geographic location and/or for system.

Lead Field Assurance Engineer (LFAE)

• Directly reporting to the Supervisor’s Representative on Site, for the assigned NEC3 contract(s) and to act as the Technical and Engineering Lead on behalf of the Chief Engineer in order to ensure Crossrail’s compliance of the Works with Sponsors Requirements and the Crossrail Assurance Process.

Engineering Manager (EM)/Framework Design Consultants (FDC)/ Chief Engineer’s Group (CEG)

• Providing engineering leadership through the Chief Engineer’s Group to the Framework Design Consultants to check quality and delivery to schedule of contractor clarifications/proposed changes of Employer’s design.
• The Engineering Manager in conjunction with the FDCs (where applicable) reviews and determines whether the repair or use-as-is solution would require a concession to the infrastructure manager or judge whether the chosen solution sufficiently meets the specification to allow the NCR to proceed.

Programme Reporting

During the project lifecycle several reports were produced in order to improve the performance for the raising, processing and closure of NCRs.

Quality Performance Index (QPI)

This report was generated by the central quality team and produced periodically on a 4-weekly cycle. The QPI was published in a league table format and issued out to all Contractors to action as they did not want to be seen at the bottom of the table. See Appendix G – Sample QPI.

With specific focus on NCRs the QPI reported the following:

• Contractor nonconformance reports raised: This metric measures the percentage of NCRs that were first raised by the Contractor. The target was 95% raised by the Contractor delivery team.
• NCR Closure Rate: This metric identifies the percentage of NCRs closed, out of the total number of overdue NCRs at the start of the period. The target was a rate greater than 40%.
• Percentage of Overdue/Open NCRs: This metric measured the open and overdue NCRs within each contract. The target was less than or equal to 5%.

Quality Dashboard

The quality dashboard was generated by the central quality team and produced periodically on a 4-weekly cycle. See Appendix H – Sample Quality Dashboard

With specific focus on NCRs the Quality Dashboard reported the following:

• The Quality Performance Index: The dashboard provided a 6-week rolling trend for the QPI scores for each contract.
• Contractor nonconformance reports raised by the contractor/open/overdue/closed: The dashboard provided a 6-week rolling trend for the percentage of NCRs raised by the contractor as well as open, overdue, and closed at a programme level.
• Partial Rework Rate: This metric measures the partial cost of rework in a period divided by the Actual Cost of Works Performed (ACWP) in the period, expressed as a percentage. The target is 5% for Stations and 2% for Systemwide. The partial cost of rework is defined as the cost of extra work performed to rectify a non-conformance. It is termed ‘partial’ because the true cost of the extra work can sometimes be impossible to quantify and/or it excludes certain factors. The dashboard provided a 6-week rolling trend for the partial rework rate as well as a table detailing specific contract information. See Figures 9 and 10 below.

Figure 9 – Partial cost of rework calculation

Figure 10 – Rework cost for programme

• Top 10 NCR Categories: The top ten NCR categories were presented in a pareto for the given period to provide a snapshot of the current trends. The categories were based on the exported data from the Crossrail NCR system and re-categorisation, if necessary, by the central quality team.
• Key issues in the Period: This would provide a narrative on items such as QIEP NCRs, positive and negative trends relating to NCRs.

Weekly/Periodic Quality Report

During the peak of the programme, a weekly quality report was produced by the central quality team and disseminated throughout the Crossrail and Contractor organisations. The principal purpose of the report was to ensure lessons were learned and shared with the programme so that appropriate mitigations could be put in place to avoid similar occurrences. As the programme began to taper, the report was published on a 4-weekly periodic basis. See Appendix J – Periodic Quality Report

With specific focus on NCRs the Weekly/Periodic Quality Report reported the following:

• NCRs by rework category: A summary of the cost banding category for NCR’s raised within the week/period.
• NCRs at a glance: A snapshot of the following:
  • Total number of NCRs raised:
  • % of NCRs Raised by Contractor:
  • Total estimated Rework Cost Last Week (All Contracts):
  • Estimated rework cost
  • Actual cost of work
  • % Rework cost
  • Most Recurring NCR Category:
  • Top NCR Classification:
  • Top NCR Discipline:
• NCR alert: The Weekly/Periodic Quality Report selected an NCR from the week or period and highlighted it to the programme with the intention of sharing the issue and any lessons learned.
• NCR Summary: A summary table of NCRs raised/closed in the week and the percentage raised by the Contractor. The table also provided a summary by each contract.
• 4 week rolling NCR trend: This chart provided the top trends within the week/period by categories to allow the recipient to understand the current trends via pareto analysis.
• NCR “pick and mix”: This section provided a selection of three NCRs from the week/period and provided a short summary of the NCR, the contract and the cost category.

Lessons Learned for Future Projects

With reflection on the Crossrail NCR system and various reports generated, the following lessons learned should be considered:

1. In terms of the NCR categories and root cause categories, these should be developed and agreed with all key stakeholders during the formation of the NCR system. Crossrail did have these defined within the NCR database as drop down menus, however the decision was made to review and refine the categories to form the weekly/periodic report. The exporting and recategorization of information is time consuming and unnecessary if work has been undertaken from the outset to ensure the categories are correct.
2. It is important to develop a system to monitor the workflow to ensure that NCRs do not remain too long at any stage e.g., Initiate, Engineering Review etc.
3. It is important to provide guidance with completing the NCR form so that accurate and detailed information is supplied e.g., comprehensive NCR details, location, classification, drawings, clear picture etc.
4. Where possible, progress closure of NCRs at the earliest opportunity to reduce the chance of any issues resolving the NCR due to access constraints, or where the initiator or other persons involved may leave the project.
5. Development of flexible and transparent reporting to ensure visibility throughout the organisation on site and at the highest level.
6. Buy-in and support for the NCR process at the highest level to allow the profile of NCRs and the cost of quality be raised.
7. Consideration of competence for persons involved in the review process to ensure an effective and consistent review process within the workflow.
8. Review or audit of NCR closeout to reduce any risks of insufficient evidence and assurance documentation on closeout.
9. Where an electronic system is used, develop effective and ‘user friendly’ interface. System to include clearly defined reporting and data export functionality.
10. Regardless of the size and complexity of the project, it is vital that there is active monitoring and reporting of the ‘quality’ progression of NCRs ensuring visibility of this throughout all levels of the organisation.
11. NCRs should be proactively reported by client and contractor organisations as soon as they are identified to avoid the risk of latent defects, or the discovery of NCRs at late stages in the project.