AI Governance in Practice: Trusted AI in Age Verification Systems
Across Australia, organisations are embedding AI into production systems that directly influence customer experiences, compliance obligations, and operational decision-making. As this shift accelerates, a more pressing question is emerging for technology and quality leaders alike: are we governing AI well enough to trust it at scale?
This question is explored in Episode 25 of the Trusted AI podcast, where Andrew Hammond, General Manager of ACT & NSW at KJR, is joined by Iain Corby, Executive Director of the Age Verification Providers Association. With extensive experience advising on global age assurance frameworks and policy, Iain Corby brings a regulatory and industry-wide perspective, while Andrew Hammond contributes a delivery and quality engineering lens grounded in real-world implementation.
“We’re trying to help organisations cut through the hype and understand what AI is actually being used for in practice.” – Andrew Hammond ACT GM KJR
Their discussion focuses on AI-powered age verification and online safety, but the implications extend far beyond that domain. It offers a practical lens into how AI governance must evolve to support safe, responsible, and scalable adoption across modern digital systems.
This conversation is particularly relevant in the context of Australia’s Age Assurance Technology Trial (AATT). KJR was part of the consortium delivering the trial, serving as the project’s Test & Evaluation Partner. With decades of expertise in technology risk management, trusted AI adoption, and software quality assurance, we led the technology evaluation component of the trial, helping ensure that age verification systems were robust, scalable, and aligned with industry standards and user expectations.
From AI Adoption to Accountability
The past year marked a significant turning point in how organisations interact with AI. Tools that were once considered experimental have now become embedded in everyday workflows, from summarising legal documents to supporting customer interactions. However, with increased usage has come a more nuanced understanding of risk, particularly around reliability and trust.
Iain Corby highlights this shift through firsthand experience:
“When working with AI, you have to read every item, every line of output. You cannot trust it at all.”
This observation reflects a broader reality that many organisations are beginning to confront. AI systems can deliver substantial efficiency gains, but without appropriate oversight, they can also introduce new forms of risk. In the Australian context, regulators such as the eSafety Commissioner are already signalling higher expectations around accountability, particularly in areas affecting vulnerable users.
For organisations operating in regulated or customer-facing environments, AI governance is therefore no longer optional. It has become a foundational requirement for maintaining trust, compliance, and long-term sustainability.
Rethinking Accuracy in AI Systems
One of the most important insights from the age verification use case is the need to rethink how accuracy is defined and measured. Unlike traditional software systems, where deterministic outputs are expected, AI models operate on probabilities and confidence levels.
According to Iain, global benchmarks for age‑assurance systems show a consistent pattern:
- In the UK, approximately 95% accuracy is considered a strong standard
- In the United States, regulators are experimenting with tiered thresholds based on age groups
- In Australia, the approach remains phased and continually evolving as policy and capability mature
Iain contextualises this effectively:
“The baseline here is 0% accuracy… even if you get to 70 or 80%, that’s a big step forward.”
For QA managers, test leads, and senior practitioners in quality engineering, this represents a fundamental shift. Testing AI systems requires moving beyond binary pass/fail criteria and adopting more sophisticated approaches, including statistical validation, risk-based thresholds, and continuous performance monitoring.
In this sense, AI governance and software quality assurance are becoming increasingly interconnected, with testing functions playing a critical role in defining what constitutes “acceptable risk” in real-world environments.
Why Not All AI Should Be Governed the Same Way
Age estimation technologies, for example, rely on machine learning models trained on large datasets of known attributes, whereas large language models generate outputs based on probabilistic pattern recognition.
This distinction is not merely technical; it has direct implications for governance. Different AI systems present different risk profiles, including:
- Bias and fairness concerns in machine learning models
- Hallucination risks in language models
- Behavioural and psychological risks in conversational AI systems
Treating all AI under a single governance framework can lead to ineffective controls and overlooked risks. Instead, organisations need to adopt a more nuanced approach that includes:
- Clear classification of AI systems
- Tailored testing and validation strategies
- Explicit documentation of limitations and assumptions
For testing professionals, this creates an opportunity to lead in defining structured approaches to testing AI systems within an AI governance framework.
The Persistent Challenge of Hallucinations
One of the most widely discussed risks associated with AI, particularly large language models, is the phenomenon of hallucinations. These occur when systems generate outputs that are plausible but factually incorrect.
Corby shares a compelling example from his own experience, where an AI-generated summary referenced a non-existent legal footnote. While the overall summary appeared credible, the fabricated detail highlighted a critical limitation.
This type of issue underscores why governance cannot rely solely on technological capability. Andrew Hammond reinforces the importance of human oversight in maintaining trust:
“If you have the trust of people you’re communicating with, you can’t afford to just rely completely on the machines.”
From a testing and QA perspective, this translates into the need for robust validation mechanisms, including multi-model verification, human-in-the-loop processes, and structured review workflows. These practices are essential for ensuring that AI outputs meet the standards expected in professional and regulated environments.
Standards, Certification, and Scalable Governance
As AI adoption grows, standards and certification frameworks are emerging as critical enablers of scalable governance. In the age verification space, international standards and certification schemes are helping organisations navigate complex technical and regulatory requirements without needing deep in-house expertise.
These frameworks provide several benefits:
- They simplify procurement decisions by establishing clear benchmarks
- They offer assurance to regulators and stakeholders
- They support consistency across implementations
Importantly, many of these standards are designed to evolve alongside technological advancements. Rather than prescribing fixed thresholds, they often reference “state-of-the-art” capabilities, ensuring that governance frameworks remain relevant over time.
For Australian organisations, this creates an ongoing obligation to continuously assess and improve AI systems. Testing teams play a crucial role in this process by validating performance, identifying drift, and ensuring alignment with current best practices.
Data Quality and the Risk of AI Feedback Loops
Another emerging concern is the growing prevalence of AI-generated content and its impact on data quality. As more models are trained on synthetic or AI-produced data, there is a risk of creating feedback loops that reinforce inaccuracies and biases.
This issue extends beyond model performance and into the broader domain of data governance. For organisations relying on AI, maintaining high-quality data inputs is essential for ensuring reliable outputs.
Testing and QA teams can contribute significantly in this area by:
- Validating training datasets
- Monitoring output consistency over time
- Identifying anomalies and degradation patterns
Incorporating data quality checks into AI governance frameworks is therefore critical for sustaining long-term system integrity.
Implications for QA and Testing Leaders in Australia
For senior practitioners in software testing, quality assurance, and digital delivery, the implications of these developments are substantial. AI governance is not a separate discipline; it is becoming an integral part of modern QA practice.
This shift requires:
- Expanding testing strategies to include probabilistic and scenario-based validation
- Embedding governance controls into development and deployment pipelines
- Collaborating closely with data, risk, and compliance teams
At the same time, there is a clear need for upskilling. As Corby notes, even experienced users are only leveraging a small fraction of AI capabilities. Bridging this gap will be essential for organisations looking to maximise value while managing risk effectively.
Moving Towards Responsible AI Implementation
Australia is entering a phase where AI adoption must be matched by equally mature governance practices. Regulatory expectations are increasing, and organisations are under greater pressure to demonstrate that their systems are safe, reliable, and accountable.
The age verification use case illustrates that achieving this balance is possible, but it requires deliberate effort. Strong AI governance, supported by robust testing and quality engineering practices, is key to moving from experimentation to responsible implementation.
For organisations navigating this transition, integrating governance into existing QA and testing frameworks offers a practical and scalable path forward. It enables teams to manage risk proactively while continuing to innovate.
Final Perspective
As AI continues to reshape digital systems, the role of testing and quality assurance is expanding in both scope and importance.
By embedding AI governance into testing practices, organisations can improve reliability, support compliance, and build confidence in AI-driven systems. In a rapidly evolving landscape, this capability will be a defining factor in long-term success.
For Australian organisations looking to strengthen their approach, aligning AI initiatives with established quality engineering practices and governance frameworks is essential.
Contact KJR to strengthen the reliability and trustworthiness of your AI and digital systems.
Frequently Asked Questions (FAQs)
AI governance refers to the frameworks, processes and controls used to ensure AI systems are safe, reliable, compliant and trustworthy. As AI becomes embedded in critical systems, governance is essential to manage risk, maintain accountability and support responsible adoption at scale.
AI-powered age verification is a real-world use case that highlights governance challenges such as accuracy, bias, privacy and compliance. It demonstrates how AI systems must be tested, validated and monitored to ensure they perform reliably in regulated environments.
Unlike traditional software, AI systems produce probabilistic outputs rather than fixed results. This means testing must go beyond pass/fail criteria and include statistical validation, risk-based thresholds and continuous performance monitoring.
AI hallucinations occur when systems generate outputs that appear credible but are factually incorrect. These errors can undermine trust and create significant risks, especially in regulated or high-stakes environments where accuracy is critical.
Trust is built through strong governance, robust testing, human oversight and transparent processes. This includes validating outputs, monitoring performance, and clearly understanding the limitations of AI systems in real-world conditions.
Different AI systems, such as machine learning models and large language models, have different risk profiles. Effective governance requires tailored approaches, including customised testing strategies, risk assessments and validation methods.
QA and testing teams are critical in defining acceptable risk, validating system performance and ensuring reliability. They help embed governance into development processes through structured testing, data validation and continuous monitoring.
- Case Studies
Age Assurance Technology Trial
Age Assurance Technology Trial OFFICIAL TEST & EVALUATION PARTNER Age Assurance Technology Trial Test & Evaluation Partner KJR is part of the consortium to deliver the Australian Government’s Age Assurance Technology Trial. The consortium is led by the Age Check Certification Scheme (ACCS) and will report to the Australian Federal Government
