Why Your Enterprise AI Project Stalled and How the Right Data Foundation Fixes It

A guide to Retrieval Augmented Generation, and why Data Vault 2 is the knowledge base that makes enterprise AI reliable.

There is a pattern playing out in organisations everywhere right now. A business leader champions an AI initiative. Budget is allocated. A large language model is selected such as GPT, Claude, Gemini, or one of their peers. A proof of concept is built, often using documents, PDFs, or a sample of data from the warehouse. It works beautifully in the demo.

Then it goes to production. And slowly, quietly, it starts to fail.

Not catastrophically. Not obviously. But the answers start to drift. The AI says a customer account is active when it was closed three months ago. It gives a different revenue figure than the finance report. It cannot explain why it reached a conclusion. It contradicts itself when asked the same question about two different time periods. Business users stop trusting it. The initiative stalls.

This is not a failure of AI technology. The large language models themselves are genuinely remarkable. It is a failure of data infrastructure, specifically, a failure to give the AI the right kind of knowledge base to reason from. And the fix is not a better model. It is a better data foundation.

This article explains the mechanics of why enterprise AI projects stall, introduces the technology that is designed to solve it: Retrieval Augmented Generation, or RAG and makes the case for why Data Vault 2 is the ideal knowledge base for enterprise RAG systems.

Enterprise AI does not stall because the models are not good enough. It stalls because the data the models are reasoning over is not good enough. That is a data architecture problem. Let us explain the solution.

How Large Language Models Actually Work and Why It Matters

To understand why data architecture is central to enterprise AI success, it helps to understand, at a high level, how large language models work and what their fundamental limitations are.

A large language model is trained on an enormous corpus of text, including books, websites, articles, code, and more and learns to predict and generate language with remarkable fluency. In the process, it absorbs a vast amount of general knowledge about the world. It understands language, context, reasoning patterns, and a huge range of topics.

What it does not know, and cannot know, is anything about your specific organisation. Your customers. Your products. Your transactions. Your business rules. Your definitions of revenue, risk, or churn. Your data, as it exists today.

When an LLM is asked a question about your business without any supporting context, it does one of two things: it may say it does not know, or more often and more dangerously, it generates a plausible-sounding answer based on general patterns, which may bear little resemblance to the truth. This is what is meant by hallucination: the model is not lying, it is pattern-matching in the absence of ground data.

The solution to hallucination is context, providing the model with accurate, relevant, current information about the specific question being asked, so that it can generate responses based on facts rather than inference. This is sometimes called “anchoring” the model to real data rather than letting it reason from general patterns alone.

Retrieval Augmented Generation, or RAG, is the leading technical approach for doing exactly this at enterprise scale.

What Is RAG and What Does It Need to Work?

RAG is an architecture pattern that connects a Large Language Model (LLM) to an external knowledge base. Rather than relying solely on the knowledge baked into the model during training, a RAG system retrieves relevant information from the knowledge base at query time and passes it to the model as context. The model then generates its response based on that retrieved context, not just its training data.

The process works in three steps:

1. A question is asked by a business user, an automated process, or another AI system.
2. The RAG engine searches the knowledge base finds the data, documents, or records most relevant to answering that question.
3. The retrieved context is passed to the LLM along with the question, giving the model the specific factual context it needs to generate a more accurate, trustworthy response.

This is a powerful, and genuinely transformative, pattern. It means an LLM can answer questions about your specific customers, your specific transactions, your specific business, if the knowledge base contains accurate, well-structured, semantically coherent data.

And that last clause is where most enterprise RAG implementations run into trouble.

The quality of a RAG system is directly and entirely dependent on the quality of its knowledge base. A RAG system built on top of a poorly structured, semantically inconsistent, historically incomplete data platform does not produce a trustworthy AI. It produces a highly fluent, very confident, occasionally wrong AI, which is considerably more dangerous than one that simply says it does not know.

RAG does not fix bad data. It amplifies it. A well-structured knowledge base makes RAG remarkably powerful. A poorly structured one makes it remarkably unreliable. 

Five Ways Enterprise RAG Fails and Why

Based on what we see in enterprise data environments, the following five failure modes account for most RAG implementations that stall or lose business trust. Each one traces directly back to a property of the underlying data that most conventional architectures do not provide. 

This failure mode is most acute in data platforms where entity coverage is inconsistent, where some customers have rich profiles and others have almost none, or where relationships between entities are implicit in join keys rather than explicitly encoded. The RAG engine retrieves what it can find; the LLM does not know what it could not retrieve.

The cause is almost always a data platform that does not preserve full temporal history. If the underlying data only reflects current state, as operational systems and most reporting layers do, the RAG system has no historical context to retrieve. It cannot answer questions about the past because the past does not exist in its knowledge base.

This is the cross-source consistency problem. Most enterprise data platforms integrate data from multiple source systems, but do not resolve the conflicts between them. The same entity, a customer, a product, an account, may be represented differently across systems, with different identifiers, different attribute values, and different relationship structures. A RAG system that retrieves from all of them simultaneously surfaces those conflicts in every answer.

Auditability requires that every piece of data in the knowledge base carries with it a clear record of where it came from, when it arrived, and what processing was applied to it. Most data platforms cannot provide this at the record level. The data has been transformed, aggregated, and moved through multiple layers, and the lineage has been lost along the way.

This is perhaps the most insidious failure mode because it is invisible. The AI is not obviously wrong; it is consistently wrong in a way that matches outdated business logic. Without explicit, versioned, machine-readable business definitions embedded in the knowledge base, there is no mechanism to detect or prevent this drift.

Why Data Vault 2 Is the Ideal RAG Knowledge Base

Every one of the five failure modes described above has a direct structural cause in the data platform, and Data Vault 2 was designed, explicitly and deliberately, to address each of those causes. But before going further, it is worth being precise about how the three components of an enterprise AI stack relate to each other because the relationship is often misunderstood. Data Vault stores and governs the data. The RAG engine retrieves the right slice of that data in response to a question. The LLM reasons over the retrieved context to produce a response. These are three distinct, separately orchestrated components. You do not simply “vectorize your Data Vault” and connect it to an LLM. The retrieval layer has its own design, what gets indexed, how embeddings are generated, what metadata travels with each chunk, how results are ranked and filtered. Getting this right is its own discipline, and it is not automatic. What Data Vault provides is the foundation that makes each of those three components work reliably.

Let us be specific.

Stable Entity Identity Eliminates Cross-Source Conflict

Hubs in a Data Vault contain Business Keys, the stable, system-agnostic identifiers that represent core business entities: Customer, Product, Account, Employee. A Business Key is the same regardless of which source system it came from. It persists across system migrations, re-platforming, and schema changes.

When a RAG system retrieves data about a customer, it can now retrieve a single, reconciled record with a stable identity, rather than three different representations from three different systems. Cross-source conflicts are resolved at the integration layer, not surfaced as contradictions (or missed) in the AI's answers.

Complete Temporal History Solves the Time Dimension

Satellites in a Data Vault record every change to every entity's attribute, with a load timestamp for every record. Nothing is overwritten. Nothing is deleted. The complete history of every entity, what it looked like at any point in time, is preserved indefinitely and query-able.

A RAG system grounded on a Data Vault can answer questions about any time period with equal confidence. What did this customer's profile look like in Q2 last year? How has this product's pricing changed over the past three years? What was the relationship between this supplier and these products before the merger? All these questions have answers because the history exists.

Explicit Relationships Provide the Knowledge Graph

Links in a Data Vault explicitly encode the relationships between entities. A customer places an order, an order contains a product, and a product is supplied by a vendor. These are not implicit in foreign keys or inferred from column names. They are first-class structural elements of the data model.

For a RAG system, this is the equivalent of a knowledge graph, a formal map of how the entities in your business connect to each other. When the AI is asked a question that requires traversing relationships, such as "which customers bought products from our at-risk supplier's last quarter?", the relationship structure is directly accessible in the knowledge base. The AI does not need to infer connections. They are explicitly encoded, and when done correctly, mapped to the relationship description in the enterprise ontology.

Business Vault Definitions Prevent Semantic Drift

The Business Vault layer sits above the Raw Vault and contains business-interpreted data, including the application of business rules, derived attributes, and formal definitions to the raw integrated data. Critically, these definitions can be versioned and time-stamped, just like any other attribute in the Data Vault.

This means that when the business changes its definition of 'active customer', the new definition can be implemented in the Business Vault with a clear effective date, and the RAG system can use the correct definition for any time period being queried. The AI does not drift silently onto outdated definitions. The versioning is structural, not a matter of documentation.

End-to-End Lineage Enables Auditability

Because Data Vault preserves every record's source, load timestamp, and processing history (Raw Vault, Business Vault & Information Mart), every piece of data in the knowledge base has a complete and traceable lineage. When the RAG system retrieves a record to answer a question, the provenance of that record is available, including which source system it came from, when it was loaded, and what Business Vault rules were applied.

This means an AI grounded on a Data Vault can not only answer questions, but it can also explain where its answers came from. In regulated industries, this is not a nice-to-have. It is the difference between AI that can be used in production and AI that cannot.

Data Vault does not just store your data. It structures it in a way that gives each component of the AI stack what it needs: the RAG engine gets stable, consistently keyed, semantically coherent records to index. The LLM gets retrieved context that is historically complete, definitionally consistent, and traceable. The orchestration layer gets clean lineage at every step. The three components are distinct, but they all depend on the same foundation. 

The Complete AI Stack: From Question to Trusted Answer

The table below shows the five layers of a complete enterprise AI stack. Below it, a single question is traced through every layer, from the moment a business user types it to the moment a trusted answer is returned. 

To make this concrete, consider a question a retention manager might ask: “Which customers renewed their contract in the last 90 days but have an open support case older than 14 days?”

Step 1: The question arrives at the RAG engine. The engine parses it and identifies the key concepts: Customer, Contract Renewal, Support Case, time bounds. It converts these into a vector query and searches the indexed Business Vault for the most semantically relevant records.

Step 2: The Business Vault responds. “Contract renewal” resolves to a versioned rule in the Business Vault that defines renewal events consistently across the CRM and the billing system. “Open support case” maps to a Satellite attribute on the Customer–Support Link with a status value and a timestamp. Both definitions are current and version controlled. The RAG engine retrieves the relevant customer records, filtered by the time bounds in the question.

Step 3: The Raw Vault provides the instances. The customer records come from Hubs with stable Business Keys, representing the same Customer entity regardless of whether it originated in Salesforce, the billing system, or the service desk. The Satellite history confirms which contracts renewed in the 90-day window and which support cases remain open, with exact timestamps. No duplicate customers. No ambiguous status values.

Step 4: The retrieved context is passed to the LLM. The model receives a coherent, correctly scoped, semantically consistent package: a list of matching customers, the definitions applied, the time window, and the source of each record. It does not receive three conflicting customer counts or a support case status that was overwritten last month.

Step 5: The LLM generates the answer. Because the context is clean, the response is specific, accurate, and explainable. The retention manager can see which customers are flagged, why each one qualifies, and which source systems the data came from. If a compliance team asks to audit the output, the lineage is available at every step.

This is why the foundation layer matters. The quality of the answer in Step 5 is entirely determined by the integrity of the data in Steps 2 and 3. IRiS ensures that integrity is structural, generated consistently, in adherence to Data Vault 2 standards, and maintained across every increment of the platform, rather than dependent on individual engineer discipline.

And because IRiS works one source table at a time, the AI-ready knowledge base grows incrementally, with each sprint adding new entities, new relationships, and new history to the foundation, delivering expanded AI capability in step with business priorities rather than at the end of a multi-year programme.

Designing the AI Serving Layer: What the LLM Actually Queries

One important distinction that is easy to miss in the architecture above: the LLM and RAG engine should not query the Raw Vault directly. The Raw Vault’s normalised structure, with Hubs, Links, and Satellites spread across multiple tables, is designed for loading, historization, and source separation. It is not designed for AI query performance. A simple question about a customer’s profile can require five or more joins across the Raw Vault before a usable answer is assembled. Research into LLM SQL generation accuracy (including MotherDuck’s analysis of the BIRD Benchmark, February 2026) consistently shows that join depth is one of the primary variables affecting accuracy: maximum two to three joins produce significantly better results than five or more.

The correct pattern is to build pre-joined Gold layer views from the Business Vault. These are purpose-built, descriptively named, flattened representations of the integrated and semantically enriched data. These views inherit all the structural benefits of the Data Vault foundation: stable entity identity, complete temporal history, versioned definitions, and end-to-end lineage. But they present that data in a form the RAG engine can index efficiently and the LLM can query accurately, with wide, flat, with descriptive column names and shallow join depth.

This is not a shortcut or a compromise. It is the intended architecture. The Raw Vault does the hard work of integration, historization, and source separation. The Business Vault applies the rules and definitions. The Gold layer surfaces the result in a form optimised for consumption, whether that consumption is a Power BI dashboard, an analyst’s SQL query, or an LLM reasoning over retrieved context. The Data Vault layered architecture was designed for exactly this separation of concerns.

What Good Looks Like in Practice

An enterprise AI system built on this stack, including the Data Vault integration layer, Business Vault semantic layer, RAG retrieval, LLM interface, behaves qualitatively differently from one built on a conventional data architecture. The difference is observable and measurable.

• Questions about current state are answered from reconciled, cross-source data with a stable entity model. The answer is the same regardless of which source system the user is used to looking at. (subject to permissions and access control)

• Questions about history are answered from complete temporal records. The AI can compare this quarter to last year, track trends, and identify when things changed and why.

• Questions about relationships traverse the explicit Link structure of the Data Vault. The AI understands how entities connect and can reason across multiple relationship hops.

• Questions involving business definitions draw from versioned Business Vault rules. The AI uses the correct definition for the time period being asked about.

• Every answer is auditable. The lineage of the data used to generate each response is available, traceable, and presentable to business users, auditors, or regulators.

This is what trusted enterprise AI looks like. Not a system that is sometimes right and sometimes wrong in ways that cannot be explained. A system that is reliably right, can explain its reasoning, and can be verified against its sources.

The Path Forward

If your enterprise AI initiative has stalled, or if you are planning one and want to avoid the patterns that cause stalling, the diagnosis is almost always the same: the knowledge base the AI is reasoning over does not have the structural properties that reliable AI reasoning requires.

The good news is that the solution is not a new technology. It is the disciplined application of a proven methodology, Data Vault 2, to the ‘Silver’ layer of your data platform, combined with a RAG architecture that connects your LLM to that knowledge base effectively.

We at Ignition have been building Data Vault platforms for enterprise clients across Australasia for over two decades. IRiS automates the most technically demanding part of that work, by generating the Raw Vault and Business Vault code consistently, at scale, and incrementally. The IRiS AI Assistant now extends that further, guiding teams from conceptual model to working Data Vault design in a single conversation.

The combination of a sound Data Vault knowledge base, a well-implemented RAG architecture, and a capable LLM is not a theoretical proposition. It is a production-ready pattern that delivers AI outputs your business can trust.

The question is not whether your organisation will implement enterprise AI. It is whether the data foundation underneath it will be good enough to make it work.

The organisations that get enterprise AI right will not be the ones with the best models. They will be the ones with the best data foundations. Build the foundation first. 

This article is part of the Data Vault Intelligence Series.