Applying AI and Test Automation in Safety-Critical Rail Systems Without Compromising Safety
KJR Queensland General Manager Graham Cummins and KJR Consultant, Rail expert, Craig Brennan explore where automation and AI genuinely add value in rail environments, where caution is required, and why governance, traceability, and deep domain expertise remain non-negotiable in safety-critical assurance.
With nearly 30 years of experience in software testing, software quality assurance, and test automation, KJR has supported major rail operators, infrastructure managers, and rolling stock manufacturers across Australia. This includes complex environments where software directly underpins passenger safety, operational readiness, workforce protection, and public confidence.
In these contexts, testing is not a supporting activity, it is part of the safety system itself.
Safety-critical rail testing: why the rules change everything?
“In safety-critical environments, testing is part of the safety ecosystem. Rail software isn’t just about business processes, it underpins passenger safety, operational readiness, reliability, and public confidence in the rail network.” – Graham Cummins (KJR QLD General Manager)
In conventional software environments, testing strategies are often shaped by speed, cost, and coverage optimisation. In rail, the priorities fundamentally shift.
As Craig Brennan explains, safety-critical rail environments require a markedly different approach to software testing and quality assurance:
- Testing coverage is effectively expected to be complete for defined safety-critical behaviours
- Testing approaches are typically functional and expert-driven, often using grey-box testing methods
- There is strict separation between design and testing responsibilities to preserve independence and assurance integrity
This separation is not procedural overhead, it is a core component of safety assurance discipline, ensuring that no single team controls both the creation and validation of safety-critical behaviour.
In contrast, non-safety-critical environments may rely more heavily on partial coverage, risk-based sampling, and overlapping development/testing responsibilities. In rail, that margin does not exist.
Where test automation genuinely adds value in rail systems
Test automation is often discussed in terms of efficiency gains. In safety-critical rail environments, however, its real value lies in consistency, repeatability, and assurance confidence. As Craig Brennan explains:
“Automation adds the most value in train control systems, timetabling, monitoring systems, and maintenance platforms, particularly where behaviour is repeatable, deterministic, and auditable.”
These conditions allow automation to strengthen confidence that critical system behaviour has not changed unintentionally.
Typical high-value application areas include:
- Train control systems
- Timetabling and operational scheduling systems
- SCADA and infrastructure monitoring systems
- Maintenance and diagnostics platforms
- Selected signalling support components (with appropriate governance controls)
Organisations operating across complex rail and logistics ecosystems can explore how these principles apply more broadly in KJR’s transport and freight sector work, where safety, reliability, and operational performance intersect.
Automation testing has been successfully applied in these areas across multiple rail programs, particularly when embedded within strong engineering and compliance frameworks.
However, even where automation is technically feasible, its application in safety-critical signalling systems must be approached with caution, ensuring alignment with regulatory expectations and assurance practices.
The risks of applying automation without governance
While automation can increase efficiency, it also introduces risk if applied without sufficient structure. The most significant risks in rail environments include:
1. Safety risk through missed defects
Without correct procedures and skills, automation may fail to detect critical faults, leading to safety incidents.
2. Operational disruption risk
Incomplete or misaligned requirements coverage can result in downstream impacts such as:
- Train delays
- Scheduling failures
- Reduced operational reliability
3. Loss of assurance traceability
Without robust governance, automation can weaken:
- Requirement traceability
- Auditability
- Compliance confidence
This is where software quality assurance frameworks become essential. Automation must strengthen, not replace, the assurance model that regulators and operators rely on.
Addressing these risks requires structured quality engineering and assurance capabilities, supported by proven testing frameworks, automation strategies, and governance models.
AI in rail: insight, not decision-making
AI is increasingly being explored across the rail sector, particularly in maintenance optimisation, anomaly detection, and design validation support. However, its role must be clearly defined.
AI has strong potential in:
Maintenance intelligence
AI can analyse maintenance logs to:
- Identify recurring failure patterns
- Detect anomalies across systems
- Support predictive maintenance strategies
Design and testing support
AI can assist engineers by:
- Reviewing design artefacts for inconsistencies
- Identifying potential edge cases
- Supporting test analysis and coverage expansion
Test execution augmentation
In testing environments, AI may help:
- Detect patterns in test results
- Identify anomalies in execution logs
- Support testers in expanding scenario coverage
This represents one of the most practical near-term applications of AI in rail operations.
However, the boundary is clear: AI supports insight. It does not make safety decisions. It does not replace:
- Acceptance criteria
- Human engineering judgment
- Regulatory compliance obligations
- Formal assurance processes
This distinction is central to AI governance in safety-critical systems.
Governance, traceability, and regulatory confidence remain unchanged
A key misconception in the adoption of AI and automation is that new technologies require new regulatory frameworks; in rail, this is not the case.
Regulatory expectations remain consistent:
- Strong engineering processes
- Full traceability to requirements and standards
- Documented procedures
- Independent review and verification
Whether systems use traditional engineering methods, automation, or AI, the assurance expectations remain the same.
This aligns closely with international rail safety standards such as EN 50128, which define strict requirements for software used in railway control and protection systems.
The implication is clear: technology evolves, but assurance discipline does not relax.
These challenges are not unique to rail. Across safety-critical industries, consistent governance, traceability, and assurance discipline remain essential to maintaining system integrity and regulatory confidence.
Why domain knowledge is still the most critical control
Despite advances in automation and AI, one principle remains unchanged in safety-critical rail testing: domain expertise is non-negotiable.
Testers must maintain deep knowledge of:
- Rail operational standards
- Safety requirements
- System behaviours
- Regulatory expectations
Without this expertise, neither automation nor AI can be effectively governed or validated. This reinforces a key principle of modern software quality assurance strategy: automation scales execution, but humans retain accountability.
Case study: Rail sensing systems and traceability-driven assurance
KJR’s long-standing work with a global transport and freight technology provider demonstrates how structured testing and automation improve safety and reliability in complex rail environments.
In this program, a digital and analogue rail track sensing system was developed to detect anomalies across rail infrastructure using:
- Laser frequency variation analysis
- Brake pressure monitoring
- GPS-based tracking systems
The system was designed to operate on moving trains, transmitting data to a central tracking platform for analysis and decision-making.
The challenge
Only a subset of system use cases had been formally documented into test cases, creating gaps in assurance coverage.
KJR’s approach
KJR was engaged to:
- Develop formal test process documentation aligned to system use cases
- Execute structured test cases with full evidentiary traceability
- Ensure alignment with EN 50128 railway safety standards
- Expand simulation capabilities to replicate real-world train movement scenarios
The outcome
The engagement delivered:
- Full test execution across multiple system releases
- Improved system validation coverage across all testable use cases
- Enhanced simulation tools for repeatable testing scenarios
- Stronger traceability between requirements and system behaviour
This project demonstrates how test automation and structured assurance frameworks directly contribute to risk reduction, defect detection, and operational confidence in rail systems.
Case study: Scalable automation in rail vehicle information systems
In another engagement with a multinational rail systems provider, KJR delivered an end-to-end automation capability for monitoring systems responsible for continuous collection of train and infrastructure data.
The challenge
The client’s existing environment included:
- Manual testing processes
- Complex and partially undocumented frameworks
- Legacy configurations and inconsistent test execution methods
The solution
KJR:
- Established a replicable automation framework
- Deployed virtual environments for controlled test execution
- Developed and adapted automated test scripts
- Enabled automated retrieval and reporting of system outputs
The outcome
The program delivered:
- Conversion of manual testing into scalable automated processes
- Reduced operational overhead in test execution
- Improved reusability of test assets across systems
- Enhanced organisational capability in test automation and software quality assurance
This engagement highlights how automation, when correctly implemented, can modernise testing capability without compromising assurance integrity.
- Case Studies
Multi-national firm specialising in rail vehicle information systems
KJR delivered test automation capability for a multinational corporation specialising in rail vehicle, track and infrastructure monitoring, and passenger information systems. The team required automation assistance on their monitoring system responsible for reporting on the continuous state of trains through continuous collection of sensory and transport data.
Major transport agency servicing QLD roads and public transport
KJR was engaged by the organisation to deliver manual testing and regression testing to assure the functionality of their customer facing transport app and website. Over the last five years this has expanded to include automation test framework implementation and test strategy and project advisory.
KJR’s longest-standing customer
KJR’s longest-standing customer, a global technology company specialising in the Transport & Freight industry was implementing a sensing system to track the state of rail tracks that distribute large
loads in Western Australia’s mining areas. KJR was trusted to deliver formal test documentation and executing formal testing of the system.
Austroads
KJR’s automated assurance framework enabled Austroads to validate complex, high-volume data pipelines with speed and accuracy. This empowered better decision-making, regulatory readiness, and scalable insights for future national road charging initiatives.
Bringing AI and test automation together, responsibly
The future of rail assurance is not about choosing between traditional testing, automation, or AI. It is about integration, under strict governance. When applied correctly:
- Automation increases consistency and coverage
- AI enhances analysis and insight
- Human expertise ensures safety and accountability
But without governance, traceability, and domain knowledge, these same tools can introduce systemic risk.
Final thoughts
In safety-critical rail environments, success is not defined by how quickly technology is adopted, but by how safely it is integrated.
Confidence in rail systems does not come from speed or tooling, it comes from assurance:
- Systems must be tested, validated, and traceable
- Processes must withstand regulatory scrutiny
- And ultimately, public trust must be maintained
At KJR, this principle underpins every engagement across rail, transport and freight, and mining, construction, and maritime ports industries, ensuring that modern engineering practices strengthen, rather than compromise, safety outcomes.
Partner with experts who understand that in rail, safety is not optional; it’s engineered!
Frequently Asked Questions (FAQs)
Testing in safety-critical rail environments requires a much higher level of rigor than standard software testing. It typically involves near-complete coverage of safety-critical behaviours, strict independence between development and testing teams, and strong emphasis on traceability, compliance, and auditability. Unlike general software systems, rail testing is directly tied to passenger safety, operational reliability, and regulatory compliance.
Test automation delivers the most value in rail systems where behaviour is repeatable, deterministic, and auditable. This includes train control systems, timetabling, SCADA systems, maintenance platforms, and selected monitoring and signalling support systems. In these areas, automation improves consistency, increases coverage, and strengthens confidence that critical system behaviour has not changed unintentionally.
The key risks include missed defects due to poorly designed test coverage, operational disruptions caused by incomplete requirements validation, and reduced assurance if traceability is not properly maintained. Without strong governance and domain expertise, automation can create a false sense of confidence rather than genuine safety assurance.
AI is increasingly being used in maintenance analysis, defect pattern detection, and test support activities. It can help identify anomalies in large datasets, analyse maintenance logs, and assist testers by highlighting edge cases or inconsistencies. However, its role is supportive rather than authoritative.
No. AI does not make safety decisions in rail environments. It can support analysis and provide insights, but all safety decisions must remain governed by human engineers, formal acceptance criteria, and regulatory assurance frameworks. AI is a decision-support tool, not a decision-maker.
Regulators generally do not distinguish between traditional engineering methods, automation, or AI in terms of assurance expectations. The focus remains on whether systems are developed and tested using robust engineering processes, full traceability to requirements, and compliance with relevant safety standards such as EN 50128.
Governance is essential. It ensures that automation and AI are applied consistently, transparently, and in alignment with safety and compliance requirements. Without governance frameworks, organisations risk introducing inconsistencies, gaps in traceability, and reduced confidence in system reliability.
Domain knowledge ensures testers understand rail-specific operational behaviour, safety requirements, and regulatory obligations. Even with automation and AI support, human expertise is critical for interpreting results, defining meaningful test coverage, and ensuring systems meet real-world operational needs.
When applied correctly, test automation increases confidence by providing repeatable, consistent, and auditable validation of system behaviour. It ensures that critical functions continue to operate as expected across updates, releases, and system changes.
KJR helps organisations apply AI and test automation in a controlled, safety-aligned way. This includes strengthening test processes, improving traceability, developing scalable automation frameworks, and ensuring compliance with safety-critical standards while enabling modern engineering practices.
