Over 10 years we help companies reach their financial and branding goals. Engitech is a values-driven technology agency dedicated.

Gallery

Contacts

411 University St, Seattle, USA

engitech@oceanthemes.net

+1 -800-456-478-23

Practical AI Assurance Workshop

Test and Evaluation of Large Language Models

This practical half-day workshop is designed to provide participants with a practical understanding of the process of testing and evaluating large language models and related generative AI applications for use in a business context.

Courses coming in 2025 to Canberra, Brisbane, Sydney and Melbourne. Register your interest today!

 

 

or contact your local KJR General Manager:

Queensland

Graham Cummins
graham.cummins@kjr.com.au
0416 551 811

Victoria

Anil Kumar
anil.kumar@kjr.com.au
0404 441 695

ACT & New South Wales

Andrew Hammond
andrew.hammond@kjr.com.au
0416 007 055

Key information

What You'll Learn

✅ Key Risks and Quality Characteristics of LLMs

✅ Evaluation Metrics and LLM Testing Approaches

✅ How to create practical automated tests to detect:

Hallucination • Bias and safety risks • Privacy risks

 ✅ Monitoring LLMs in production

✅ Reporting and collecting test evidence as part of an AI governance process

Workshop topics:

Session 1:

Evaluating Large Language Models

Introduction to Large Language Models 

Overview of LLMs, Key Characteristics of LLMs 

Testing Large Language Models  

Testing in LLM Development, Types of Tests, Test Case Design 

Evaluation Metrics and Techniques

Common Metrics, Qualitative Evaluations, Challenges 

Practical Exercise

Hands-on Exercise, Group Discussion

Session 2:

Practical Techniques & Best Practices in LLM Testing

Advanced Testing Techniques 

Uncovering Biases & Ensuring Fairness, Testing for Robustness, Scenario-based Testing 

Optimizing LLMs Through Iterative Testing  

Iterative Development, Feedback Loops, Case Studies

Best Practices and Future Directions 

Best Practises, Emerging Trends, Ethical Considerations & Responsible AI 

Interactive Session

Applying Advanced Testing Techniques 

Wrap up

Key Takeaways, Resources, Conclusion 

 

What's included?

  • Expert Workshops

    Learn from leading industry experts in tech and business.

  • Interactive Sessions

    Engage in interactive sessions that facilitate collaborative learning and discussion among participants.

  • Networking Opportunities

    Connect with peers and industry who share a common interest in AI deployment and governance. Lunch will be provided at the beginning of the workshop.

Who is it for?

This workshop is intended for technical practitioners and any professionals looking to understand the process of testing and evaluating LLMs and related generative AI applications for use in a business context.

Where is it delivered?

Participants have to join in-person to attend the program.

How to register?

📭 There will be only 20 seats available for this cohort.

More Questions?

Reach out to KJR at info@kjr.com.au if you have any questions. 

This program is delivered by KJR. KJR is a founding member of the Queensland AI Hub

KJR is an Australian Software Quality Engineering Consultancy and a leading practitioner in Responsible AI implementation. Founded on over 27 years of experience in quality assurance and verification, KJR has always understood the importance of managing IT risk and especially does so now with the rapid maturation of various emerging technologies. We have harnessed our expertise to develop practical approaches to assuring compliant and successful deployments of AI solutions. 

Your Trainers

Dr Mark Pedersen
KJR CTO

Mark is an IT professional with a passion for digital culture. During the week, he leads his team at KJR through straight-talking solutions to software problems. In his spare time, he can be found crafting soundscapes and audio/visual installations that embrace technology’s propensity for playfulness and expression. As CTO, Mark’s technical knowledge and lateral thinking abilities are counted on to lead critical software projects safely out of the red and into the light. And, as a world-class software risk analyst and advisor, he thrives on the satisfaction of bettering lives with technology that actually works.

Edit Template
Dr Kelvin Ross
KJR Founder

Dr Kelvin Ross is an entrepreneur, technologist and researcher. He is the founder and Chairman of KJR, CTO to Datarwe, Director of the Queensland AI Hub, and IntelliHQ, developing the ecosystem for AI innovation in healthcare. He is an Adjunct Associate Professor at the Institute for Intelligent and Integrated Systems, Griffith University. He has over 30 years of experience in software engineering and enterprise IT applications. Kelvin holds a PhD in safety critical systems engineering he is a firm believer of AI’s added value to the Community. 

Edit Template

    Practical AI Assurance Workshop Inquiry

    Your email address will not be published. Required fields are marked *

    Other Workshops/Courses

    More training options

    Courses