As AI becomes embedded in financial services, organisations need to understand the full ecosystem involved in producing outputs, from language representation and embeddings to retrieval, reasoning and response generation. The article explores where trust can break down at each stage and why AI governance must extend beyond traditional model validation. For financial institutions, this means understanding the wider technology and information ecosystem that shapes AI outcomes.
Generative AI is evolving from experimental technology into an operational capability embedded within core business processes. The challenge is no longer simply deploying technology but building confidence in its outputs and understanding the ecosystem that supports them.
Discussions often focus on AI outputs while overlooking language representation, knowledge retrieval, model reasoning, and response generation. Trust depends on every stage involved in producing a response.
It All Begins with Language
Humans communicate through words, sentences and context. Computers require language to be converted into mathematical form before processing.
The first step is tokenization, which breaks text into smaller units called tokens. Depending on the model, a token may represent a word, sub-word or punctuation. Early NLP techniques used relatively simple representations. Bag of Words (BoW) counts word frequency:
BoW (word) = Count(word)
A variation known as Binary Bag of Words records only whether a word appears:
BinaryBoW (word) = {1/0 if word appears otherwise
To preserve some word relationships, NLP evolved to N-Grams, which capture neighboring words and short phrases:
= (
, 
,
)
Where represents a word (or token) and n represents the number of consecutive words.
Another advancement was Term Frequency–Inverse Document Frequency (TF-IDF), which assigns greater weight to more informative words:
TF = 
IDF = log 
TFIDF = TF x IDF
where N is the total number of documents and DF is the number containing the word.
These techniques enabled language to be represented numerically but shared a limitation: they represented words through counts and frequencies rather than meaning.
From Words to Meaning
Modern Generative AI represents meaning itself. Large Language Models convert words into dense numerical vectors known as embeddings:
Word 
Vector
Embedding (word) = [
, 
, 
, 
, 
]
Unlike earlier NLP techniques, embeddings capture semantic relationships, allowing words with similar meanings to be located near one another in high-dimensional vector space. Models therefore understand context, similarity and conceptual relationships rather than simply matching keywords.
Embeddings also underpin vector databases and modern retrieval systems, enabling searches based on meaning rather than exact word matches.
Risk Lens: Traditional NLP limitations primarily affect accuracy, whereas modern embeddings influence every downstream stage, from retrieval and reasoning to trust in the final response.
Augmenting Knowledge Beyond Training Data
One of the most significant developments in Generative AI is Retrieval-Augmented Generation (RAG).
Traditional language models rely largely on information learned during training, which may be outdated or lack organization-specific information.
RAG adds a retrieval step before generation, searching approved knowledge repositories, document stores, and vector databases for relevant information that is added to the model’s context. This enables more reliable responses using internal knowledge sources.
