Background
This long-term client wanted to implement a Common Extract Database (CED) to unify all
its disparate data sources, and integrate with cloud data sources. This provided an
opportunity to then deliver some AI-enabled services such as Leak Detection and customer
billing prediction. This required complex data ingestion and transformation using databricks
on the Azure platform. KJR was required to assist with an AI risk analysis and to conduct
thorough testing of data and transformations. This was a 14 month project.
Challenge
- Data validation of nearly 200 objects containing data on millions of water meters and
customers. - Risk assessment of customer-facing AI application.
Solution
- AI implementation Risk Analysis – KJR held several workshops and 1-1s which revealed
to the client they had to do a lot more thinking about the ethics of data quality/selection and
AI governance, and we uncovered a lot of risks for them, eg monitoring model drift, data
transformation errors, data ingestion error handling etc. - Data verification testing: checking that the right fields and tables were ingested and used
at each stage by the SI. This required very complex SQL scripting to compare the correct
fields across multiple tables.
Deliverables
- Test Cases and Test Reports on Validation Testing of Tb of data ingested into the Data
Platform and made available for AI. - AI implementation Risk Analysis Recommendations
Key Outcomes
- Identification of key AI implementation risks, for example monitoring model drift, data
transformation errors, data ingestion error handling - Assurance for the customer that data had been ingested and aggregated and
transformed correctly. - Assurance for the customer that PII data was secure.
- KJR also improved the customer’s awareness of AI risks and of possible unintended
errors in data transformation.
Value to Client
Incorrect data and insecure PII data can be very costly to clients, especially in customer-facing
aspects of a utility.
KJR testing gave the client assurance their implementation is correct and secure.
Tools & Technologies
- KJR AI Risk Assessment Process
- SQL for data testing
- Azure platform
- databricks
