By Garrick Schermer, AI Data Strategy and Governance Lead
Agentic AI has moved quickly from concept to strategic priority across most industries. Behind nearly every promising pilot, however, sits a foundational question that gets far less attention than the AI itself: where does the data come from, and can you trust it?
That question comes down to two architectural approaches that are increasingly common in agentic AI projects: querying data directly through a Model Context Protocol (MCP) server, and querying data through a governed data platform such as an analytical data mart. Both have a legitimate place in an organization's AI strategy. The organizations that get the most value, without taking on unnecessary risk, are the ones that understand what each approach is actually good for.
Two paths to the same question
An MCP server is, in simple terms, a connector that lets a large language model reach into your data sources on the fly, in response to a natural-language prompt. Ask a chatbot "what are my top incidents by user?" and the MCP server can go straight to the underlying tables, join them, filter them, and return an answer in seconds.
A governed data platform takes a different route. Data is ingested from source systems, cleaned and transformed, organized into an analytical data mart with defined data models, documented relationships, and sanctioned metrics. Reports and AI agents draw from that mart rather than from raw operational systems.
Both approaches can technically answer the same question. They differ sharply in how they get there, what it costs to ask, and how much you can trust the answer.
The case for MCP: speed as a discovery tool
Used well, an MCP server pointed at raw or lightly modeled data is an experimentation engine. It lets teams test a hypothesis in minutes rather than months, since there is no data model to build before a question can be asked. Teams can prove out the value of a use case in a handful of prompt cycles, making it far easier to justify further investment before committing engineering resources. It also opens the door to discovering relationships in the data that no one had previously modeled or thought to look for, since the LLM is free to join and explore data on the fly, rather than being confined to a predefined schema.
This is genuinely valuable. It is also, by design, a sandbox capability rather than a production one.
What the speed doesn't show you
The trade-off for that speed shows up once you look past the first few prompts. Consider a deceptively simple question: "Tell me my top incidents by user." Run through an MCP server against raw tables, that single prompt can require the LLM to join and filter large volumes of data live, at a cost of millions of tokens for one answer. Multiply that across a team's daily usage, and the token bill becomes a real budget line, not a rounding error.
But cost is only the first issue. Because the MCP server is interpreting relationships in the data at the moment of the prompt, rather than relying on a pre-defined and tested data model, the same question asked two different ways, or by two different users, can produce two different answers. There is no single, official version of "top incidents" to check the result against.
That inconsistency compounds several other risks worth naming plainly. Without careful scoping, an MCP server can expose more of the underlying data source than a user should be able to see, raising real data security and visibility concerns. When an LLM assembles an answer dynamically, it is also difficult to reconstruct exactly how it got there after the fact, a real problem for any answer that ends up in a board report, a regulatory filing, or a customer-facing decision, since there is little audit trail or lineage to point to. And because an agent can be prompted in countless different phrasings, validating that it produces correct, repeatable results takes meaningfully more QA effort than validating a fixed report.
None of this means MCP servers are unsafe to use. It means they are best treated as a sandbox: ideal for proving out ideas, not yet ready to be the system of record.
The case for the governed data platform
The traditional data platform, an analytical data mart built on defined data models, documented foreign key relationships, and established governance, exists precisely to solve the problems above. Its benefits are the mirror image of the MCP server's weaknesses. Every dataset has a defined owner, a documented lineage, and a known, transparent relationship to every other dataset it connects to. The same question, asked any number of times by any number of people, returns the same answer, because the underlying joins and calculations were built and tested once rather than reinvented per prompt. And traditional query and reporting infrastructure is materially cheaper to run at volume than paying for AI tokens to re-derive the same relationships repeatedly.
The trade-off is time and flexibility. Building a proper data mart requires skilled data engineers and architects and takes longer to design and deploy than standing up an MCP connection. Because it is purpose-built around known questions, it may also not support a new line of inquiry without additional development work.
Finding the balance
The point of comparing these two approaches isn't to declare a winner. It's to match the tool to the task. A useful way to think about it is to use the MCP-connected sandbox to discover which questions are worth answering, and use the governed data platform to answer the questions that matter enough to be trusted, repeated, and audited.
In practice, that looks like a maturity path rather than a single decision. Organizations do well to start with low-risk, sandboxed MCP experimentation to test hypotheses quickly and see what patterns in the data are actually worth pursuing, then prioritize data platform development around the specific business questions that experimentation validates as valuable, rather than around building every possible model up front. Capabilities move into production as data trustworthiness, governance, and security controls catch up to the pace of experimentation.
This sequencing lets an organization capture the speed and creative upside of AI-driven discovery without asking a sandbox tool to carry production-grade weight it wasn't built for. Put simply: experiment fast, but govern before you scale.
Where SDI can help
SDI Presence works with clients across SLED, utilities, aviation and transportation, and commercial industries to build exactly this kind of balanced foundation: assessing data and AI maturity, standing up governed data platforms and analytical data marts, and designing agentic AI pilots that are structured to graduate responsibly from sandbox to production. Ready to build a trusted foundation for agentic AI?
Contact SDI to explore how we can help your organization turn AI experimentation into scalable, governed solutions.
.avif)
.avif)
