Unlocking AI: Importance of Proof of Concept

On this episode of the “Gen AI Breakthrough” podcast, Kyle McNabb hosts a discussion on the importance of proof of concept (POC) in testing AI solutions, highlighting its role in feasibility assessment and resource efficiency. Guests Kyle Robichaud and Jay Ruffin emphasize key factors such as data quality, risk mitigation, and alignment with organizational goals that influence the success of POCs. Additionally, it addresses the need for continuous education and effective communication to bridge the gap between executives and implementation teams.

Welcome to the “Gen AI Breakthrough” podcast, where top experts give actionable artificial intelligence (AI) insights, expert advice and strategies to achieve breakthrough business performance. In this episode, host Kyle McNabb, head of Research at The Hackett Group, is joined by Kyle Robichaud and Jay Ruffin – two experts in technology transformation. The discussion centers around how organizations are progressing from conceptual ideas to actual value realization in their AI efforts.

To begin, Jay explains a proof of concept (POC) as a small-scale test designed to validate the feasibility of an AI solution. Unlike a pilot or full product implementation, a POC is conducted in a controlled environment, often using synthetic or limited data. Its primary goal is to validate whether a concept works – not to develop a complete or production-ready solution. He warns against overcomplicating POCs by trying to test entire systems or integrating too many variables. The purpose of a POC is not to build a final product but to test parts of a solution. Multiple POCs may be conducted to test different capabilities, and failure at this stage is not only acceptable but expected as part of the learning process. Clear acceptance criteria must be defined before starting a POC. This ensures that the project remains goal-oriented and doesn’t become a resource drain.

Poor quality or siloed data can hinder AI implementation. The importance of using accurate, current and well-mapped data across systems is emphasized. Solutions like data lakes and master data management help improve the usability and availability of enterprise data. Without these, even strong AI concepts may fail. Ruffin adds that data security and privacy are also critical, and technical feasibility must include considerations like integration with other systems and scalability. Once a POC is successful, the next challenge is scaling it across the enterprise. Ruffin explains that data silos across different regions and systems can make this difficult. Additional training, especially for models, may be necessary. Solutions also vary – some are fully automated, while others require user interfaces and process changes. Therefore, change management becomes vital. Organizations may need internal marketing campaigns and executive education to promote understanding and adoption of AI tools. Executives must be aligned with teams on the potential and limitations of the AI being implemented.

Robichaud outlines key factors that organizations must consider before launching a POC. These include ensuring compliance with data security standards, understanding what data can be used and how it will be protected, and adhering to existing AI governance policies. Organizations are increasingly concerned about protecting their intellectual property and avoiding unintentional exposure of sensitive data. Compliance and governance thus become foundational pillars in evaluating readiness for POC implementation. The Hackett Group offers tools and expertise that streamline the POC process. Their methodology ensures that all critical questions are addressed efficiently from the outset, rather than requiring multiple rounds of clarification. Whether organizations should use synthetic or real data, or whether to rely on documents like PDFs, The Hackett Group can guide these decisions quickly and effectively. Jay notes that experience matters: if a team stumbles through early steps, it risks eroding management’s confidence in the entire initiative. A POC should be fast and focused. If it drags out, stakeholders may lose faith – even if the underlying data challenges are legitimate. The Hackett Group’s involvement can help maintain confidence and momentum.

Time stamps:

0:12 – Welcome to this episode hosted by Kyle McNabb.

1:31 – Defining the POC: purpose and use.

3:07 –  Key considerations for running a POC.

6:46 – Data challenges and importance in POCs.

9:21 –  Post-POC strategy: scaling and change management.

13:20 – Environment constraints and POC objectives.

14:52 – How The Hackett Group adds value: speed, tools and experience.