HomeShowcasesDigital Assets & AIMetadata Enrichment – with Gen AI
Metadata Enrichment – with Gen AI

Metadata Enrichment – with Gen AI

Generative AI automates and standardizes business metadata, enhancing data documentation and operational efficiency. It’s a game-changer for data discovery and governance.

Introduction

 

In today’s digital world, data is more important than ever. At UOB, millions of data streams converge every day to fuel our operations and analytics. Managing this data effectively isn’t just critical for regulatory compliance, it’s the key to unlocking valuable insights about our business. To make sense of this wave of data, we need metadata, or data about data.

 

Metadata gives meaning to the data, enabling analysts to parse through the noise and find the treasure within, while ensuring that our Artificial Intelligence (AI) models are built on solid foundations.

 

However, producing metadata is time-consuming and labour-intensive, especially at the scale of millions of data points, meaning there is often missing or insufficient metadata for analysis and reporting.

 

As a result, UOB turned to the language capabilities of Generative AI to systematically populate metadata at an enterprise scale. These advanced technologies, such as Microsoft 365 Copilot, allow us to create metadata like data definitions up to 11 times faster, significantly improving data discoverability and governance.

 

When our staff focuses more on reviewing than producing, it gives us more time to analyse the data, uncover insights, and make better decisions. Allowing us to provide even greater value to our customers and stakeholders.

Metadata Challenges

Laptop

Inadequate Metadata

Metadata has known gaps in terms of quality and completeness which requires a unique combination of domain knowledge and technical expertise to fulfill.

thought-leadership

Reduced Data Discoverability

Data discoverability and data usability decreases as power users do not have enough description and business context about technical data.

password-login-credentials

Regulatory Non-Compliance

Bank is unable to meet regulatory compliance if business metadata is missing or incomplete.

uob-career

Low Data-Reliability

Lack of metadata also decreases data reliability and data quality across the bank and in turn decreases operational efficiency.

Solution Methodology

 

Gen AI is known to have the ability to analyse data, recognise patterns and provide recommendation and insights.

 

UOB leverages this Gen AI capability to interpret technical metadata and comprehend business context to furnish relevant business definitions such as Business Terms, Business Descriptions.

 

We take the data dictionary which has only technical information like table names, column names, source system names and feed it to Microsoft Copilot. Along with it we also share some reference information about our bank’s systems with it.

 

And Copilot is capable enough to analyse this technical information from the data dictionary, decipher it, understand its business context with the help of the reference documents, analyse then generate relevant business metadata.

 

How is Business Metadata Generated – using Gen AI

 

Here's our comprehensive approach to generating business metadata.

 

Mitigating Gen-AI Risks

 

Generative AI comes with its own share of risks and as part of a comprehensive AI Governance strategy, tailored guardrails are implemented to ensure robust risk management. Effective prompt engineering, and reference data grounding has been adopted to promote output accuracy and soundness.

 

Human in the loop moderation and user feedback is also used to mitigate the infamous “AI hallucination" risk.

 

Projected Benefits

binoculars

Enhance Data Discoverability

It enables data discoverability, sharing and reuse on data dictionaries, by allowing users to quickly search for, find and use relevant datasets with.

extra-mile

Effective Data Governance

Metadata is central to agreed data governance standards, delivering compliance with corporate policies.

megatrends

Improves Data Quality

as metadata provides information on the quality and reliability of the dataset.

12-hour-cooling-period

Delivers Time and Efficiency Savings

as users can find and use relevant information more quickly themselves, without requiring support from data teams.

colleagues

Increases Collaboration Internally

and externally by enabling people to work together with shared, mutually understood data.

recycle

Ensures Compliance

Metadata enables data stored in different systems and databases to be interoperable, providing an up-to-date record of information and any changes made to it.