Maximizing ROI: Best Practices for Scaling Generative AI Across the Enterprise

By Arun Chandrasekaran

Generative artificial intelligence (GenAI) has the potential to revolutionize businesses in various industries. Most business and technology leaders are convinced that the advantages of GenAI outweigh any potential risks. However, lack of understanding about emerging industry best practices is constraining organization wide pilots and scalable production deployments.

Through 2025, Gartner predicts that at least 30% of GenAI projects will be abandoned after proof of concept (POC) due to poor data quality, inadequate risk controls, escalating costs or unclear business value.

To avoid obstacles to scaling GenAI, chief information officers (CIOs) must embrace the following emerging industry best practices.

Establish a Continuous Process to Prioritize Use Cases

The initial step in the GenAI journey is to establish the organization’s AI goals and engage in a preliminary discussion about what is achievable. The subsequent step involves gathering potential use cases that can be piloted with GenAI technologies. Prioritizing GenAI use cases is a strategic imperative for organizations. Such prioritization should not be driven solely by the appeal of technology, or the “flashiest demo,” but by a holistic assessment of its value proposition to the organization. While vendors may suggest discounted POCs reflecting their capabilities, the key is to identify use cases that deliver tangible business value and are the most technically feasible and avoid those that could lead to growing risks and costs when scaled in production. The task of prioritizing should be a collective decision, involving not only the technology teams but also the business lines that will utilize the GenAI application as well as security and risk teams.

Create a Decision Framework for Build Versus Buy

Scaling GenAI requires a systematic approach to build versus buy decisions for the many potential use cases in the organization. Ideally, businesses should consider building an AI product when it can provide a competitive advantage in their industry and when they have the necessary skills and knowledge for the process. In the context of GenAI, use cases where enterprises want to minimize risks for regulatory or brand equity reasons may also warrant a build approach. CIOs must evaluate all pros and cons of the approach before determining their build-versus-buy decisions for GenAI.

Pilot Use Cases for Scalability

Businesses must run pilots to try new ideas, build muscle memory within the organization on the art of the possible and learn by experimentation. They must ensure that pilots are built with scalability in mind by envisioning future data, privacy, security and usability needs. An agile mindset must be adopted before experimenting and testing the use cases to determine the next step — scale, refine or stop. A sandbox environment must be established to allow for safe experimentation throughout the organization. This should include appropriate security and privacy measures, as well as the availability of multiple GenAI models for experimentation and iteration within the sandbox. This allows developers to have the flexibility to select the most suitable models for each specific use case.

Design a Composable Generative AI Platform Architecture

The GenAI landscape consists of four critical layers — infrastructure, models, AI engineering tools and applications. Enterprises must ensure that their platform architecture is composable, scalable and embedded with governance upfront.

The GenAI model landscape is fast-paced and will constantly evolve, often in ways we cannot envision today (such as the rise of open-source models and domain models). Organizations must ensure there is enough flexibility in their architecture to swap models through composability.

Responsible AI Is at the Forefront of All Generative AI Efforts

GenAI creates not only new opportunities, but also new risks. Responsible AI is an umbrella term for all the different aspects of making appropriate business and ethical choices when adopting AI. Without a clear responsible AI framework, organizations will struggle to balance the benefits and risks of this technology. Organizations need to define and publicize a vision for responsible AI with clear principles and policies across focus areas like fairness, toxicity mitigation, ethics, risk management, privacy, sustainability and regulatory compliance.

Invest in Data and AI Literacy

Unlike traditional AI, GenAI is poised for active and direct use by a large segment of employees. This broad deployment requires a strong emphasis on AI literacy: the ability to utilize AI in context with competency to identify relevant use cases, as well as implement and operate corresponding AI applications. Enterprises must create and conduct personalized training programs targeting various business functions and training senior management on the data and AI literacy skills. Upskilling the technology teams with GenAI-specific skills in areas such as prompt engineering, model validation and tuning, infrastructure management and responsible AI is crucial.

Additional analysis on GenAI for enterprises will be presented during the Gartner Data & Analytics Summit, taking place April 24-25 in Mumbai, India.

(The author is Arun Chandrasekaran, Distinguished VP Analyst at Gartner, and the views expressed in this article are his personal)