Naresh Raheja explores the operational, governance, and compliance risks hidden beneath the promise of artificial intelligence. Drawing on failures across multiple industries, he highlights risks including hallucinations, algorithmic bias, automation bias
Artificial intelligence systems may appear highly
capable, articulate and sophisticated, but many still fail in ways that create
profound operational, compliance and governance risks for financial institutions,
according to Naresh Raheja.
Raheja, formerly a Senior Risk Specialist and Examiner at the OCC, argues that banks are now entering a critical phase where AI risk shifts from experimental technology concerns into core operating-model risk.
And as generative AI tools become embedded within
customer service, documentation, risk analysis and decision-making workflows,
this transition is set to accelerate exponentially.
“AI capability does not guarantee AI reliability,”
Raheja says.
Drawing on lessons from failures across industries
including airlines, healthcare, legal services, retail and media, Raheja
outlines ten major dimensions of AI risk that banks must now confront. Those
include hallucinations, algorithmic bias, instability, misinformation, data leakage,
automation bias and failures involving human oversight.
He warns that AI systems frequently perform
impressively in controlled demonstrations but become unreliable once exposed to
complex real-world environments, edge cases and changing operating conditions.
“Modern AI systems can appear very capable but may
suffer from reliability issues,” he writes in a self-published paper on the subject.
“They may be articulate, knowledgeable, and persuasive, yet they can sometimes
fail in ways that matter when deployed in real-world environments.”
Among the examples highlighted by Raheja is the Air
Canada chatbot dispute in which a customer receives incorrect bereavement fare information
from an AI assistant and the airline is later held responsible for the
chatbot’s inaccurate guidance.
Another case involves a UK parcel delivery chatbot
manipulated into insulting its own employer publicly. For banks, Raheja
believes such failures carry significant implications because customer-facing
AI systems increasingly influence financial decisions involving payments, lending,
hardship support, collections and product eligibility.
“This creates potential risks for consumer
protection, UDAAP, complaint management, mis-selling, and legal liability,” he
says.
Raheja also expresses concern about “hallucinations”
where AI systems generate convincing but false information.
He points to legal cases involving fabricated court
citations and AI-generated finance articles containing inaccurate financial
explanations as examples of how persuasive but unreliable outputs evade human scrutiny.
“The banking analog is not only a hallucinated
customer answer, but also a hallucinated control narrative, policy
interpretation, credit memo, model validation summary, board report, or
regulatory response that appears polished enough to pass review,” he warns.
