As financial firms race to integrate GenAI into model documentation, research and analysis highlights the complex, evolving art of validating AI with AI.
display-middle-ads-here
Despite being the number one topic occupying the
headspace of risk managers all over the world, the thorny issue of generative
artificial intelligence (GenAI) is one that – for now at least – seems still to
elude consensus.
The practical realities and emerging strategies for
implementing and validating GenAI were certainly front of mind at CeFPro’s USA
conference in the context of model risk
management (MRM).
The session focused on a use case from ValidMind that
shed light on the challenges and innovations involved in integrating GenAI
features into tools used for automating model documentation and validation.
The core of the presentation centered around a
GenAI-driven feature designed to automate the interpretation of test results, a
function critical to both developers and validators operating within tightly
regulated industries.
Though specific in its nature, the use case also
served as a microcosm for broader questions confronting the financial sector.
How should organizations validate AI-generated
content? What constitutes a ‘well-written’ interpretation of technical model
results? And how can subjective judgments be systematized to ensure consistency
and regulatory compliance?
In this case, the GenAI feature in question was used
to interpret outputs from various tests – from correlation matrices to scatter
plots – and generate narrative explanations that assessed test methodologies
and gave context for specific results.
The speaker – a senior figure within the risk
management community – emphasized the dual structure of such interpretations as
both a general explanation of the test and a specific analysis of the results.
The challenge, it emerged, lies in the subjective
nature of such interpretations, where domain experts may disagree on what
aspects are most relevant and how much detail is appropriate.
The evaluation of these interpretations presents its
own array of difficulties.
While expert-labeled benchmarks represent the gold
standard, they are inherently limited by human subjectivity and resource
intensity.
As an alternative, metric-based evaluations using
open-source tools like Ragas provide scalable ways to assess qualities such as
faithfulness, relevance, and logical consistency.
