In today’s rush to adopt the latest machine learning technologies, it’s easy to forget that innovation must be matched with practicality.
At CeFPro’s 2025 AMR USA conference, I had the opportunity to present a perspective on refining financial crime models to adapt to fraud patterns and regulatory changes, emphasizing how we can bring cutting-edge ideas like transformers into our world - carefully and thoughtfully.
I started the session by stripping away the intimidating jargon around AI. I asked the audience to imagine a simple classroom scene: a teacher posing a question to four students, each an expert in a different subject.
Their answers varied in relevance, and the teacher combined the most useful parts to create a final response. This, I explained, is the essence of the self-attention mechanism, the foundational idea behind transformers.
Just like the teacher in the classroom, self-attention evaluates and weighs different pieces of information based on relevance before building an output.
This intuitive explanation served as a stepping stone to a deeper exploration of how transformers could be used in financial crime detection. We transitioned from the classroom to the complex and messy world of financial risk management.
Fraud detection today demands far more than isolated transaction scoring. It requires a system capable of recognizing patterns across vast transaction histories, drawing connections that are not immediately obvious, and adapting to constantly evolving fraudulent behaviors.
Transformers, with their self-attention capabilities, offer an exciting way to meet these challenges by analyzing transaction sequences holistically rather than piecemeal.
To illustrate the power and challenges of transformers in this space, I presented a case study.
Using a small set of simulated transactions, I walked the audience through how transformers embed transaction data, calculate attention matrices, and ultimately identify fraudulent patterns.
Interestingly, this was not just a conceptual exercise. I built the entire transformer workflow manually in Excel, including embeddings, attention score computations, and gradient descent backpropagation.
It took eight iterations for the model to start reliably distinguishing between fraudulent and normal transactions - a painstaking but rewarding demonstration of what goes on under the hood.
However, even as I showcased the potential of transformers, I stressed their limitations in real-world fraud detection. Fraud detection needs speed; decisions must be made in milliseconds, not minutes.
Transformers, unfortunately, are memory-hungry and computationally expensive. Their attention matrices grow quadratically with the number of inputs, making them ill-suited for the massive, high-velocity data streams that define real-time fraud prevention.
