Agent Data Has Three Parts. Most Enterprises Only Build One.

Every enterprise agent pitch I've sat through this year opens the same way: connect a knowledge base, plug in a model, ship a pilot. Then the pilot answers a policy question beautifully and gets a customer's order status wrong, and nobody on the call can say why.

I keep seeing the same root cause. This applies to any enterprise team building or buying an AI agent platform in 2026, whether the agent handles support, sales, or internal ops. It's not a claim that better data guarantees success, only that bad data guarantees the specific kind of failure I keep watching teams walk into.

Enterprise AI agent platforms need three distinct data layers working together: structured data for precise answers, documents and RAG for broad knowledge, and knowledge/memory for continuity across sessions. Most teams invest in only one, usually RAG, and then wonder why their agent still gives wrong answers or forgets what a user told it yesterday.

Why do most enterprise AI agent pilots stall before production?

Gartner predicts that more than 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls (Gartner, June 2025). Separately, Gartner found that 63% of organizations don't have, or aren't sure they have, the data management practices AI requires, and predicts that through 2026 a lack of AI-ready data will cause organizations to abandon 60% of their AI projects (Gartner, February 2025).

The common explanation is that the model isn't good enough yet. I don't buy that, at least not for most of the projects behind those numbers. Not usually. The data underneath the agent was never made agent-ready. As much as 90% of enterprise data is unstructured, according to IDC research (IDC, via Box), and most of it has never been cleaned or classified for a system that acts on it directly.

Agent-ready data is a stricter bar than analytics-ready data. A dashboard tolerates an occasional bad row because a human reviews it before acting. An agent acts on the data directly. No human catches the error first, so the standard has to be continuous quality, not a one-time cleanup.

What is "Agent Data" and why isn't RAG enough on its own?

Agent Data is the combined set of structured records, documents, and persistent memory that an enterprise AI agent draws on to answer questions and take action. It's not one system. It's three, and each covers a gap the others can't.

• Structured data answers precise, factual questions from systems of record: order status, account balance, inventory count.
• Documents and RAG answer broad knowledge questions from unstructured content: policies, contracts, support articles, product manuals.
• Knowledge and memory carry context across time: what a user already told the agent, what decisions were made, what the agent has learned about a domain.

Most enterprise agent projects start and stop at RAG. It's the most mature tooling category, and it's the easiest line item to defend to a buying committee (a slide with "retrieval-augmented generation" on it clears procurement faster than "we also need to fix our schema"). That leaves an agent that can summarize a policy document well and still can't tell a customer their correct order status, because that answer lives in a database, not a document.

How do you get structured data agent-ready?

Structured data is a table, warehouse, or lakehouse record that an agent queries directly through SQL or a governed API rather than free text. This is usually the fastest of the three pillars to fix. The data already has a schema. But the schema was written for a database administrator, not a language model.

Column names lie by omission. A column named STATUS could mean [ACTIVE, PENDING, CLOSED] or [INPATIENT, OUTPATIENT, EMERGENCY], and the name alone tells the agent nothing. Sampling five to ten real values per column and attaching that context to the schema is the highest-leverage fix here. It's the same technique schema-discovery tools use before an agent attempts natural-language-to-SQL (Oracle, 2026).

Practical steps that make structured data agent-ready:

1. Attach business definitions and sample values to every column an agent might query, not just the ones that look obviously important.
2. Enforce row-level and column-level permissions before the query layer, not after. An agent should never see a row it would query around.
3. Version the schema and log every agent-issued query, so a wrong answer is traceable to a specific table, not a black box.
4. Cap what an agent can query directly. Give it a small number of governed views instead of raw table access.

How do you get documents and RAG data agent-ready?

Retrieval-augmented generation is a technique where an agent searches an indexed set of documents at query time and grounds its answer in the retrieved passages instead of relying only on what the model already knows. RAG quality is decided before the model ever sees a query: chunking, metadata, retrieval strategy.

One time, I was building an agent platform for a client in Singapore. The platform helped their marketing team benchmark creative work against government regulations the client had to comply with. The regulations ran a few hundred pages in a single PDF, and the agent kept giving inaccurate assessments because it was trying to reason over the whole document at once. We broke the rules into indexed chunks and had the agent retrieve only the section relevant to each question the marketing team asked. Both the accuracy and the speed of the results improved substantially.

A few practices separate agent-ready document stores from the rest:

• Break long documents into topic-scoped sections before ingestion. A 300-page policy manual chunked as one blob forces the retriever to guess which section is relevant.
• Attach metadata (product, region, audience, owner, review date) to every chunk, so retrieval can be filtered instead of searching the entire index every time.
• Combine keyword search (BM25) with dense vector retrieval, merged with reciprocal rank fusion. This consistently beats either method alone on both exact-match and semantic queries (Weaviate, 2026).
• Remove conflicting or outdated material before ingestion rather than after. An agent has no way to know a document is stale unless the metadata says so.

How do you get knowledge and memory agent-ready?

Agent memory is the layer that lets an agent retain context, facts, and outcomes across sessions instead of starting from zero every conversation. It splits into four types: semantic memory (facts about users and domains), episodic memory (records of past interactions), procedural memory (behavioral rules the agent follows), and working memory (the active context window in the current session).

This is the newest and least standardized of the three pillars. Oracle's AI Agent Memory, Google's Memory Bank, and Zep's context graphs are all racing to build the infrastructure layer for it in 2026. Some analysts expect contextual memory to overtake RAG as agentic AI's primary retrieval mechanism (VentureBeat, 2026 predictions). And none of that infrastructure fixes bad memory content on its own.

The data-quality version of this problem: what should the agent actually remember, for how long, and who can see it? A support agent that remembers a customer's account history across calls is valuable. A support agent that remembers and surfaces something a different customer said is a governance failure, not a feature. Memory needs the same tenancy and retention rules a database gets, even though it feels conversational.

Which pillar should you build first?

Start with structured data if your agents will mostly answer factual questions from systems you already govern. It's the fastest to fix, since the schema exists and you're adding context, not building from nothing.

Start with documents and RAG if the agent's job is answering "how does this work" or "what's our policy on X" questions spread across unstructured content with no other home.

Add memory last, and only once the first two are stable. Memory compounds whatever quality problems already exist in structured data and documents. Worst outcome: an agent that's confidently wrong and also remembers being wrong.

The case against this sequencing is real: budget and headcount are finite, and a memory layer can be the flashier demo for a steering committee. My current bet is that sequencing beats flash, but the evidence is thinner for organizations with a single, narrow agent use case where memory genuinely is the differentiator. If that's your situation, the order above is a starting point, not a rule.

My approach is to ask clients the right questions first, to understand all the data their future agent platform will actually need to touch. From there we classify, based on their daily workflows and priorities, which pillar to build first. So the order above isn't fixed. It shifts with each client's priorities and urgency.

Frequently asked questions

What is the difference between RAG and agent memory?

RAG retrieves documents at query time to answer a question from a knowledge base. Agent memory persists facts and context across sessions, so the agent remembers what happened last week without re-reading a document.

Do enterprises need a knowledge graph to build good agent data?

Not always. A knowledge graph helps when an agent needs to reason across many linked entities, but many enterprises get most of the value from clean structured data plus well-chunked documents before a graph is needed.

Which of the three agent data pillars should a company build first?

Start with structured data if agents will answer factual questions from your systems of record, since it is usually the fastest win and the most governed data you already have.

How much enterprise data is actually usable by agents today?

As much as 90% of enterprise data is unstructured (IDC, via Box), which means most of it needs cleaning, chunking, and metadata work before an agent can use it reliably.

Is agent-ready data different from analytics-ready data?

Yes. Analytics-ready data tolerates occasional errors because a human reviews the dashboard. Agent-ready data needs continuous quality and governance because the agent acts on it without that human check.

Key takeaways

• Agent Data has three parts: structured data, documents/RAG, and knowledge/memory. Most enterprise agent failures trace back to building only one.
• Structured data is usually the fastest fix. Attach business definitions and sample values to columns, not just names.
• RAG quality is decided before the model ever runs: chunking, metadata, and hybrid retrieval matter more than the model choice.
• Memory should come last. It compounds whatever quality problems already exist in the other two layers.
• Agent-ready data requires continuous quality and governance, not a one-time cleanup. That's what separates it from analytics-ready data.

I'd love to hear where this framework breaks in your context, especially if memory turned out to be the right layer to build first for you. What am I not seeing?