Introduction

Validation is an assessment of the operational system that exposes and quantifies the systems’ limitations. The intent of validation is to determine if the user’s needs are satisfied for different uses (often referred as scenarios). When the functions are provided, the physical entities are adequate and the user’s behaviours are as needed, the service is deemed fit for the use intended by the set of requirements. The concept of validation suggests that requirements can be mapped into physical, functional, and behavioural needs of key stakeholders. This mapping is indeed essential to confirm that the service satisfies the key stakeholder’s requirements and is found to be acceptable. From the perspective of integration, validation is the confirmation that integration had satisfactory results. Should there be problems discovered during validation tests, they are more often of two cases: either functionality is performed inadequately for the expected use, or there are inordinate losses to achieve the desired level of performance. In both cases, the functionality is present and the issue is one of performance ^[1].

In Crossrail, verification and validation tests are carried out at the testing and commissioning stage of the project to verify that components, sub-systems and ultimately systems function according to the relevant standards and specifications. Testing is conducted in phases to build up systems in a controlled and traceable manner. The build-up of these systems is demonstrated below in Figure 1.

Figure 1: System Build Up

The integrated tests shall be undertaken when systems are connected to other completed systems. In order to undertake integrated testing, systems must be fully functional and meet their respective performance requirements. As stated, a system is defined as an assembly of sub-systems working together in order to satisfy specific operational or functional requirements of the system. Sub-systems are parts of an elementary system that can be tested together without interfering with other parts of the system. Prior to integrated testing, interfaces between the systems involved are tested by simulation only.

The integrated testing proves software, (e.g. signals between control systems), physical, (e.g. connection of the systems), and indirect interfaces, (e.g. pressure differentials on room doors), between systems to ensure they work in unison to meet the operational requirements.

Scope of Integration Tests

Integration tests in Crossrail can broadly be classified into the following two categories:

1. Performance Tests required by Crossrail as part of the evidence and safety justifications to prove that an integrated railway has been delivered. Against each test, this paper will also make a link to the Acceptance Criteria as defined by Crossrail.
2. Operational Readiness Tests required by Crossrail and the Infrastructure Managers in order to prove the railway is compliant with Crossrail’s Operational Concepts. This category also includes tests required by Crossrail, London Underground and any other significant party in order to prove the railway integrates with systems which are already in operation (such as integration between Crossrail and London Underground Fire Alarms at Interchange Locations).

Performance tests to prove that railway is fully integrated

The tests listed herein are focused on the integration between systems (from various parties). The performance tests cover such elements as Interface tests as defined in ICDs, EMC tests, cold/ heated smoke tests, tests to demonstrate compliance with statutory requirements and supply rules for public utilities connection, acoustic tests, tests to determine fault recovery time for critical plant, validation tests for RAM parameters, integrated performance tests, and stress tests, (drain down tests for UPS, over-temperature tests for example). These tests should only be conducted when systems are fully connected and in a final state. Their purpose is to confirm:

• System Interactions
• Software Interactions (in the case of software interface),
• Response Times (for end to end tests, response time is the total length of time to execute a control/monitoring command, and Isolation and possession requirements).

Figure 2 below presents a list of tests covered in the integration testing regime.

Figure 2: Station Services Integration- Performance Matrix

Operation Readiness Tests

The tests listed here in Figures 3-6 highlight some of the operational readiness tests for stations, shafts and portals. These tests focus on the operation of the station and the attempt to prove readiness of the systems and progression to trial operations. Tests in this section have been broken down into four sub-sections to ensure that the station operates in different situations and no operational requirements have been missed. These sub-sections are ^[2]:

• Normal Operation. This refers to the conditions which a part of the railway is designed to accommodate. This would include the peaks, e.g. rush hours, and troughs in demand experience during the day.
• Degraded Operation. This refers to the state of the part of the railway system when it continues to operate in a restricted manner due to the failure of one or more components.
• Emergency Operation. This refers to a current unforeseen or unplanned event which has life threatening or extreme loss implications and requires immediate attention, e.g. a fire.
• Platform Train Interface. This refers to the interfacing elements between platform and train e.g. platform screen doors, train doors, illumination etc.

Figure 3: Operation Readiness Tests- Normal Operations

Figure 4: Operation Readiness- Degraded Operations

Figure 5: Operation Readiness- Emergency Operations

Figure 6: Operation Readiness- Platform Train interface

Technical Details of the Integration Tests

In accordance with the terms and conditions of contract, the Contractors will develop the integration test plan, co-ordinate with the respective interfacing parties on site, and plan tests in accordance with technical/ statutory/ utility supply requirements including constraints, test method, instrumentation, acceptance criteria, programme and required attendance. In order to deal with the complexities of Crossrail it was decided to generate a common approach across the whole of the railway. The common approach grouped the tests listed in Figures 2-6 into a number of operational style test scenarios. By following this approach Crossrail gained the following benefits:

• Commonality of approach to assist with the review, witness and overall acceptance of systems
• Consistent set of test which the Infrastructure Maintainers could help shape. This allowed them to ensure all their requirements were met and thus gain confidence in the systems being delivered
• De-risk the handover process as Crossrail had gained the Infrastructure Maintainers buy-in early in the process.
• Allowed Crossrail to develop provide greater clarity to the senior management as every Station was being measured against the same approach.

The output records arising from the performance of this schema will be used as validation evidence, before the Acceptance Certificate is signed off, to prove that all the relevant systems are fully integrated and ready to proceed with dynamic and operational testing of the railway.

Details of the approach are as follow:

Test Scenarios

Grouping integration tests into a small number of test scenarios for ease of management and co-ordination among different parties e.g. Station, TVS and Communications Contractors. The test scenarios were designed to group activities involving similar systems even if there were multiple tests to be conducted. For example, testing of the LV switchboards functionality was grouped alongside the noise testing. This was done because for the LV switchboard all systems are required to run at full load thus it is perfect opportunity to test the noise levels in key areas. Against each scenario Crossrail defined the generic test script to be followed which included:

• Test Items
• Test Sequence
• Test Methods
• Acceptance Criteria
• Roles and responsibilities

The scenarios are shown below in Table 1.

Table 1 List of Test Scenarios

Each scenario was assigned a co-ordinator, whose presence and function was to:

• Develop a coordinated test script that all interfaces bought into
• Coordinate the events to ensure they were all completed
• Develop a test report which highlighted were all the test records pertaining to the scenario were captured. This was required because each Contractor was self- assuring their own records.

Furthermore the scenarios were required to be built into the Contractors programmes so that the planning team were able to develop an integrated programme across the whole project.

For this schema to be successful it had to be endorsed by the Technical Authority, Infrastructure Maintainers and Operators. Consequently a number of works were conducted with the relevant bodies to ensure each script met their requirements and the results documented. A sample test script is shown in Appendix 1.

Challenges on implementing Integration Tests

It is not without challenge to implement a line wide and consistent set of integration tests for station services. Some of the difficulties encountered were:

1. The integration test scope, (and certainly not detail test scripts), is not clearly defined in the Contract Works Information. As a result, clarification work on the test requirements has to be communicated and agreed with the Contractor before implementation.
2. Integration test programme in a worksite has to be co-ordinated with the line wide commissioning programme as many of the integration tests will require the service of line wide systems, (TVS, SCADA), and the train service. The integration test window can be very restrictive.
3. Inadequate information in interface control documents and system architecture to complete the integration test plan.
4. Alignment with operation requirements, some of these requirements were not agreed until a late stage of project.
5. Lack of interest by Contractor to develop a cross boundary integration test plan as they are ‘outside’ their work scope.

It is imperative that Crossrail has to take lead to formulate a consistent line wide integration test plan and see to it that they are consolidated into a integrated T&C for the whole line.

Conclusion and Lesson Learned

Integration tests were successfully implemented for Crossrail station services to validate that the engineering systems are fully integrated and suitable for use. The test plan is in complete alignment with the stakeholder’s requirements and developed in collaboration with the future operators and maintainers.

The following lessons can be drawn from the experience from implementing integration tests for Crossrail station services:

1. The construction work packages, which are normally area based, have to include a clear and defined scope on integration testing.
2. The construction master plan has to make due allowance in terms of time and resource in carrying out integration tests.
3. Early engagement with future maintainers and operators to gain their endorsement on the scope and acceptance criteria of integration tests.

References

[1] Gary O. Langford, Engineering Systems Integration: Theory, Metrics and Methods, CRC Press, 2012.

[2] Railway Safety Principles and Guidance Part 2 Section B Guidance on stations, HSE 2005.

Appendix A- Sample printout of a scenario test script