Appendix F - Appendix F to Part 236—Minimum Requirements of FRA Directed Independent Third-Party Assessment of PTC System Safety Verification and Validation
(a) This appendix provides minimum requirements for mandatory independent third-party assessment of PTC system safety verification and validation pursuant to subpart H or I of this part. The goal of this assessment is to provide an independent evaluation of the PTC system manufacturer's utilization of safety design practices during the PTC system's development and testing phases, as required by the applicable PSP, PTCDP, and PTCSP, the applicable requirements of subpart H or I of this part, and any other previously agreed-upon controlling documents or standards.
(b) The supplier may request advice and assistance of the independent third-party reviewer concerning the actions identified in paragraphs (c) through (g) of this appendix. However, the reviewer should not engage in design efforts in order to preserve the reviewer's independence and maintain the supplier's proprietary right to the PTC system.
(c) The supplier shall provide the reviewer access to any and all documentation that the reviewer requests and attendance at any design review or walkthrough that the reviewer determines as necessary to complete and accomplish the third party assessment. The reviewer may be accompanied by representatives of FRA as necessary, in FRA's judgment, for FRA to monitor the assessment.
(d) The reviewer shall evaluate with respect to safety and comment on the adequacy of the processes which the supplier applies to the design and development of the PTC system. At a minimum, the reviewer shall evaluate the supplier design and development process regarding the use of an appropriate design methodology. The reviewer may use the comparison processes and test procedures that have been previously agreed to with FRA. Based on these analyses, the reviewer shall identify and document any significant safety vulnerabilities which are not adequately mitigated by the supplier's (or user's) processes. Finally, the reviewer shall evaluate the adequacy of the railroad's applicable PSP or PTCSP, and any other documents pertinent to the PTC system being assessed.
(e) The reviewer shall analyze the Hazard Log and/or any other hazard analysis documents for comprehensiveness and compliance with railroad, vendor, supplier, industry, national, or international standards.
(f) The reviewer shall analyze all Fault Tree Analyses (FTA), Failure Mode and Effects Criticality Analysis (FMECA), and other hazard analyses for completeness, correctness, and compliance with railroad, vendor, supplier, industry, national, or international standards.
(g) The reviewer shall randomly select various safety-critical software modules, as well as safety-critical hardware components if required by FRA for audit to verify whether the railroad, vendor, supplier, industry, national, or international standards were followed. The number of modules audited must be determined as a representative number sufficient to provide confidence that all unaudited modules were developed in compliance with railroad, vendor, supplier, industry, national, or international standards
(h) The reviewer shall evaluate and comment on the plan for installation and test procedures of the PTC system for revenue service.
(i) The reviewer shall prepare a final report of the assessment. The report shall be submitted to the railroad prior to the commencement of installation testing and contain at least the following information:
(1) Reviewer's evaluation of the adequacy of the PSP or PTCSP including the supplier's MTTHE and risk estimates for the PTC system, and the supplier's confidence interval in these estimates;
(2) PTC system vulnerabilities, potentially hazardous failure modes, or potentially hazardous operating circumstances which the reviewer felt were not adequately identified, tracked or mitigated;
(3) A clear statement of position for all parties involved for each PTC system vulnerability cited by the reviewer;
(4) Identification of any documentation or information sought by the reviewer that was denied, incomplete, or inadequate;
(5) A listing of each applicable vendor, supplier, industry, national or international standard, process, or procedure which was not properly followed;
(6) Identification of the hardware and software verification and validation procedures for the PTC system's safety-critical applications, and the reviewer's evaluation of the adequacy of these procedures;
(7) Methods employed by PTC system manufacturer to develop safety-critical software; and
(8) If directed by FRA, methods employed by PTC system manufacturer to develop safety-critical hardware.