Operationalising Gen AI at scale: Responsibility, reliability, and reusability as key to enterprise-grade integrations

Principal Consultant at GlobalLogic, Dr Caterina Constantinescu, shares her insights on the key challenges and how enterprises can overcome them when moving Gen AI from pilot projects to enterprise-scale deployment

As the world hurtles towards a future shaped by artificial intelligence (AI), it's becoming increasingly clear that using this technology requires more than just sending prompts to a large language model (LLM). Enterprises are grappling with the daunting task of scaling generative AI (Gen AI) from mere prototypes to fully-fledged, business-wide deployments, a challenge that demands their utmost attention and strategic solutions.

The promise of Gen AI is not just enticing—it's truly inspiring. It holds potentially transformative capabilities that can revolutionise business processes, enhance customer experiences, and drive innovation. So much so, that in a recent report, McKinsey estimates Gen AI could add trillions of dollars in value to the global economy. 

However, beneath the surface lies a complex landscape fraught with critical challenges and potential pitfalls that demand careful navigation and open conversations.

Challenges on the path to scale

Data dilemmas: Safeguarding intellectual property (IP) and protecting privacy

At the heart of Gen AI lies data—it is the lifeblood that powers its creative engines. Yet, this very data poses significant risks. Enterprises must grapple with the delicate balance between harnessing data for innovation and safeguarding IP and data privacy. Cisco’s 2024 Data Privacy Benchmark Study, for example, reveals that seven in ten enterprises cite threats to legal and IP rights (69 per cent) and the risk of disclosure to the public or competitors (68 per cent) as their top concerns. 

Research also indicates that customer experience management (CXM) leaders share similar concerns. Ninety-four per cent say it’s affecting their organisation’s ability to adopt and implement Gen AI solutions. It’s safe to say that the fear of proprietary information and data leakage looms large, necessitating robust mechanisms to safeguard IP and protect privacy. 

Navigating minefields: Governance, risk and compliance (GRC): 

Gen AI’s potential extends beyond business or technical metrics. It touches legal, ethical, and reputational realms. But, organisations must tread carefully, ensuring compliance with industry regulations and legal requirements. 

The cost of non-compliance is continuously growing in line with the introduction of legislation by various jurisdictions, such as the European Union’s EU AI Act. Such sanctions intend to be effective, proportionate, and dissuasive. In other words, the larger the enterprise, the greater the penalty. 

The lowest infringement in the EU AI Act is providing incorrect, incomplete, or misleading information. Yet, this carries administrative fines of up to 1 per cent of total worldwide turnover, or €7.5 million, whichever is higher. Thus, the challenge lies in leveraging AI’s power while avoiding GRC minefields. 

Customisation complexity: Integrating and aligning with existing infrastructure 

Achieving seamless coexistence between Gen AI and legacy systems or domain-specific models is a massive demand. Existing technology infrastructure, often the backbone of everyday operations, is designed for specific purposes only, meaning it can’t necessarily fulfil state-of-the-art Gen AI requirements. 

Implementing Gen AI tools and platforms that feed on modern, adaptable, and scalable architectures, like microservices around legacy systems based on antiquated frameworks and programming languages, is a substantial task. So is reconciling the variations in data formats and standards generated by existing infrastructure so that Gen AI applications can interpret and process data effectively. 

Change management: Minimising disruptions during transition

Transitioning from pilot projects to enterprise-wide Gen AI adoption requires considerable adaptation. Ensuring a smooth transition without compromising safety, productivity, user experience, or customer satisfaction is problematic. 

Gen AI necessitates reimagining an enterprise’s policies, processes, and workflows. Employees, in particular, bear the brunt of this shift. Of the 750 global chief information officers (CIO) surveyed by Lenovo, only four in ten felt their organisation had sufficient operational understanding beyond the IT function to be AI-ready. 

As noted by the International Monetary Fund (IMF) Gen-AI: Artificial Intelligence and the Future of Work report, older, less digitally savvy workers are particularly vulnerable to failure. Enterprises trying to implement change management that minimises disruption during transitions can, then, find themselves paddling against the tide. 

Business alignment: Controlling costs and driving returns on investment (ROI) 

The allure of Gen AI can blind organisations to practical considerations. ROI relies on controlling costs, maximising returns, and ensuring Gen AI initiatives positively contribute to strategic objectives. Yet, Lenovo also reports that 61 per cent of CIOs find it very or extremely difficult to demonstrate ROI from tech investments. 

Perhaps more noteworthy is that while almost all CIOs expect AI investments to increase, half say it is pulling attention and resources away from other key IT areas like cloud adoption (48 per cent), sustainability (38 per cent), and employee compensation (38 per cent). These findings suggest that many enterprises miscalculate costs, meaning they underestimate the impact on enterprise-wide ROI and fail to ensure total business alignment.

Successful architecture: Ensuring responsibility, reliability and reusability

Failing to overcome the challenges laid out above carries steep ethical, legal, and reputational risks. It’s no wonder that many enterprises are hesitant to move Gen AI applications into production. 

At GlobalLogic, we recently unveiled a first-of-its-kind Platform-of-Platforms architecture designed to support the deployment of enterprise-grade AI. To quote our Chief Operations Officer and Head of Engineering, Sumit Sood: “We understand what needs to be done to ensure responsibility, reliability and reusability in AI platforms.” 

My extended experience working in AI and machine learning has led me to agree. Integration at scales relies on platform architecture that supports these three factors. 

Responsibility

The responsibility tenet centres around ensuring Gen AI responses are accurate, lawful, and compliant with security and governance requirements. Transparency in data usage, explicit consent, and secure handling are non-negotiable. Responsibility isn’t just a buzzword; it’s imperative for building trust with users and stakeholders. 

Reliability

Gen AI models have a tendency to drift, hallucinate, and fall prey to security threats over time. Enterprises need mechanisms to monitor and correct these deviations. Balancing the need for model creativity against use case requirements and wider context ensures reliability. Regular audits, security protocols, and vigilance against vulnerabilities are essential. As is close adherence to MLOps best practices, which is a key feature of our Platform-of-Platforms.

Reusability

When organisations reuse standardised approaches and best practices for data ingestion, preparation, and model training mechanisms, the impact of Gen AI multiplies, and the overhead for adoption decreases. Shared consumption ensures not only consistency but also efficiency. The reusability principle posits that rather than reinventing the wheel, enterprises should leverage industry-leading models and algorithms, as well as modern architectures built with reusability in mind—for example, microservices—all supported by expert partnerships. 

The key to operationalising Gen AI at scale: Expert partnerships

It’s hardly surprising that over three-quarters of enterprises are considering outsourcing their Gen AI development to specialist third-party firms to mitigate the risks—but choosing the right partner is critical to success.

Embarking on a Gen AI odyssey is no easy feat. If transformative journeys are to be successful, enterprises need expert guides—seasoned specialists who understand the nuances of the terrain, know how to avoid the pitfalls and illuminate the path forward. 

What’s needed are partners with a mature understanding of the cutting-edge solution ecosystem who balance tech prowess with an ethical ethos to ensure safeguarding, privacy, and trust in order to guide enterprises to a future where Gen AI powers better business outcomes. Through such partnerships, enterprises thrive, operationalising Gen AI at scale to shape a future where innovation coexists with responsibility, reliability, and reusability. 

About Dr Caterina Constantinescu

Former doctoral researcher turned data scientist, Caterina now works as Principal Data Consultant at GlobalLogic, where she advises on GenAI, data observability and business impact issues, as well as the custom applications of data science models to industry use cases. 

About GlobalLogic

GlobalLogic is a leader in digital engineering. We help brands across the globe design and build innovative products, platforms, and digital experiences for the modern world. By integrating experience design, complex engineering, and data expertise—we help our clients imagine what’s possible and accelerate their transition into tomorrow’s digital businesses. 

Headquartered in Silicon Valley, GlobalLogic operates design studios and engineering centers around the world, extending our deep expertise to customers in the automotive, communications, financial services, healthcare and life sciences, manufacturing, media and entertainment, semiconductor, and technology industries. GlobalLogic is a Hitachi Group Company operating under Hitachi, Ltd. (TSE: 6501), which contributes to a sustainable society with a higher quality of life by driving innovation through data and technology as the Social Innovation Business.

Read the original article here.