Test and Evaluation of Large Language Models
This practical half-day workshop is designed to equip participants with a hands-on understanding of how to test and evaluate large language models and related generative AI applications for use in real business environments. The session focuses on ensuring these technologies are not only technically sound but also aligned with organisational goals—enabling informed adoption that drives measurable business value, impact, and strategic growth.
Courses coming in 2025 to Canberra, Brisbane, Sydney and Melbourne. Register your interest today!
or contact your local KJR General Manager:
Queensland
Graham Cummins
graham.cummins@kjr.com.au
0416 551 811
Victoria
Anil Kumar
anil.kumar@kjr.com.au
0404 441 695
ACT & New South Wales
Andrew Hammond
andrew.hammond@kjr.com.au
0416 007 055
Key information
- Face-to-face Workshop
- Half-Day
- Venues to be confirmed
- Price: $500 ex-GST, plus booking fee
What You'll Learn
Key Risks and Quality Characteristics of LLMs
Evaluation Metrics and LLM Testing Approaches
How to create practical automated tests to detect:
Hallucination • Bias and safety risks • Privacy risks
Monitoring LLMs in production
Reporting and collecting test evidence as part of an AI governance process
Workshop topics:
Session 1:
Evaluating Large Language Models
⚡ Introduction to Large Language Models
Overview of LLMs, Key Characteristics of LLMs
⚡ Testing Large Language Models
Testing in LLM Development, Types of Tests, Test Case Design
⚡ Evaluation Metrics and Techniques
Common Metrics, Qualitative Evaluations, Challenges
⚡ Practical Exercise
Hands-on Exercise, Group Discussion
Session 2:
Practical Techniques & Best Practices in LLM Testing
⚡ Advanced Testing Techniques
Uncovering Biases & Ensuring Fairness, Testing for Robustness, Scenario-based Testing
⚡ Optimizing LLMs Through Iterative Testing
Iterative Development, Feedback Loops, Case Studies
⚡ Best Practices and Future Directions
Best Practises, Emerging Trends, Ethical Considerations & Responsible AI
⚡ Interactive Session
Applying Advanced Testing Techniques
⚡ Wrap up
Key Takeaways, Resources, Conclusion
What's included?
Expert Workshops
Learn from leading industry experts in tech and business.
Interactive Sessions
Engage in interactive sessions that facilitate collaborative learning and discussion among participants.
Networking Opportunities
Connect with peers and industry who share a common interest in AI deployment and governance. Lunch will be provided at the beginning of the workshop.
Who is it for?
This workshop is intended for technical practitioners and any professionals looking to understand the process of testing and evaluating LLMs and related generative AI applications for use in a business context.
Where is it delivered?
Participants have to join in-person to attend the program.
How to register?
📭 There will be only 20 seats available for this cohort.
More Questions?
Reach out to KJR at info@kjr.com.au if you have any questions.
This program is delivered by KJR. KJR is a founding member of the Queensland AI Hub
KJR is an Australian Software Quality Engineering Consultancy and a leading practitioner in Trusted AI Adoption. Founded on over 27 years of experience in quality assurance and verification, we help organisations unlock real business value from AI, ensuring deployments are not only compliant and ethical but also strategically aligned to drive innovation, efficiency, and growth.
Your Trainers
KJR CTO
ACS AI Ethics Committee Vice-Chair
Mark is an IT professional with a passion for digital culture. During the week, he leads his team at KJR through straight-talking solutions to software problems. In his spare time, he can be found crafting soundscapes and audio/visual installations that embrace technology’s propensity for playfulness and expression. As CTO, Mark’s technical knowledge and lateral thinking abilities are counted on to lead critical software projects safely out of the red and into the light. And, as a world-class software risk analyst and advisor, he thrives on the satisfaction of bettering lives with technology that actually works.
KJR Founder
Dr Kelvin Ross is an entrepreneur, technologist and researcher. He is the founder and Chairman of KJR, CTO to Datarwe, Director of the Queensland AI Hub, and IntelliHQ, developing the ecosystem for AI innovation in healthcare. He is an Adjunct Associate Professor at the Institute for Intelligent and Integrated Systems, Griffith University. He has over 30 years of experience in software engineering and enterprise IT applications. Kelvin holds a PhD in safety critical systems engineering he is a firm believer of AI’s added value to the Community.