Building The AI-Ready Tech Stack: What CIOs Need to Get Right

On this episode of the “Gen AI Breakthrough” podcast, hosts Dan McCabe and Jay Ruffin discuss how organizations can build their technology stacks to fully leverage AI, emphasizing the importance of strong data governance, a robust semantic layer and flexible architecture over simply selecting advanced AI models. They highlight the growing trend of embedded AI capabilities within leading enterprise platforms and the role of model context protocol (MCP) in enabling flexibility and reducing vendor lock-in.

Welcome to The Hackett Group’s “Gen AI Breakthrough” podcast, where top experts give actionable artificial intelligence (AI) insights, expert advice and strategies to achieve breakthrough business performance. In this episode, hosts Dan McCabe and Jay Ruffin explore how organizations can future-proof their technology stacks to fully capitalize on AI’s potential.

First, Dan and Jay explain that AI is collapsing the traditional technology stack, making intelligence, data, infrastructure and applications increasingly interconnected. Rather than focusing on finding the perfect AI model, organizations should strengthen the architecture, semantic layer, data governance and master data management that determine whether AI can deliver reliable results. Enterprise systems are also evolving from systems of record into systems of action, increasing the need for stronger AI governance and real-time decision-making capabilities. Organizations that redesign processes around AI and establish strong data foundations are seeing significant gains in productivity and performance.

The discussion also highlights how AI is increasingly being embedded directly into platforms from vendors such as SAP, Oracle, Microsoft and Salesforce, lowering the barrier to adoption – particularly for midmarket organizations. Rather than building stand-alone AI programs, companies can leverage capabilities already included in their software while shifting investment toward change management, adoption and effective use. Dan and Jay recommend treating vendor roadmaps as a form of “free R&D,” aligning them with transformation plans, minimizing customization through a clean-core approach, and carefully evaluating new consumption-based AI costs. Finally, they introduce model context protocol (MCP) as a flexible connectivity layer that can link AI models with enterprise systems without hard-wired integrations. Because organizations cannot predict which models will dominate in the future, MCP provides a way to preserve flexibility while ensuring that access to high-quality organizational data remains at the center of an AI-ready architecture.

The discussion continues with MCP, which is described as a critical connectivity layer for future AI and agentic architectures. MCP allows organizations to switch between foundational models – such as OpenAI, Claude and Gemini – without repeatedly reengineering downstream integrations, helping reduce vendor lock-in as AI technology evolves. However, MCP is still maturing, and organizations must continue developing appropriate governance and security practices while recognizing that many existing applications are not yet MCP-ready. Dan and Jay also emphasize the importance of identifying AI capabilities already embedded in enterprise platforms; one client, for example, had more than 200-300 unused AI capabilities across systems such as Oracle, Blue Yonder, and Salesforce. Understanding and incorporating these capabilities into technology roadmaps can prevent unnecessary investment and accelerate AI adoption.

The discussion concludes that technology decisions made over the next 18 months will be critical in determining whether organizations successfully scale AI or end up with disconnected pilots that never deliver value. They recommend defining a strong semantic layer, establishing data governance before launching AI initiatives, maintaining clean data and architecture, and strategically managing enterprise resource planning (ERP), application programming interfaces, and integration layers. Organizations should view upcoming budget cycles as an opportunity to fix foundational architecture before AI expectations outpace their capabilities. Ultimately, the organizations that succeed will not necessarily have the best AI models, but the best underlying architecture, clean semantic layers, strong data governance and ERP systems that can serve as an intelligent core. The central message is to stop “shopping for models” and instead build the architecture and infrastructure those models depend upon.

Time stamps:

0:04 – AI is collapsing the traditional technology stack.

5:15 – Data governance and master data as AI enablers.

7:47 – Embedded AI is changing the economics of AI adoption.

9:22 – Embedded capabilities lower the barrier for midmarket companies.

13:41 – Moving from models to protocols: the rise of MCP.

15:02 – MCP enables flexibility and reduces vendor lock-in.

16:36 – Embedded AI capabilities and the need for strategic planning.

21:28 – The 18-month window to build the right foundation.