McKinsey recently released a report on the current state of AI. While the whole report is worth a read, here are a few observations that I think are especially noteworthy.

Reimagining Workflows with Generative AI

Automating current processes, while providing some near-term cost savings, will not produce differentiated value opportunities for organizations. Leading companies are focusing on reimagining entire workflows with generative AI (gen AI) and analytical AI, rather than simply embedding these tools into existing processes. This approach is crucial for realizing the full potential of AI.

Recognizing and Managing Risks

As businesses begin to see the benefits of gen AI, they’re also recognizing its diverse risks. These include data management risks such as data privacy, bias, and intellectual property infringement; model management risks focusing on inaccurate output or lack of explainability; and security risks associated with incorrect use.

Therefore, successful implementation of gen AI requires drawing on leaders from across the organization. Achieving market impact and navigating the many inherent risks necessitates close partnerships with HR, finance, legal, and risk departments.

The Importance of Testing and Responsible AI

Gen AI solutions, being probabilistic models, can make mistakes or amplify biases in training data. Therefore, testing models before deployment is essential to ensure responsible AI. Without a robust testing approach, delivering on responsible AI is challenging. I think this is something we do not talk enough about. How do you test non-deterministic models? What are the use cases where not being 100% certain of what the model will output is OK and which use cases present unacceptable risk given current testing limitations?

Customization Over Off-the-Shelf Solutions

Roughly half of survey respondents are using readily available, off-the-shelf generative AI models. While understandable as the initial steps of new technology adoption, it’s not an approach that will provide long-term, sustainable value. As gen AI becomes more widespread, customization will be key to maintaining a competitive edge. This requires the development of significant expertise in creating and managing foundational models and large language models (LLMs).

The Full Solution: Beyond Technology

Gen AI models generally comprise only 15% of any given solution. To create value, organizations must have all the elements in place: domain reimagining abilities, relevant skill sets (including the upskilling of non-technical colleagues), a robust operating model, and proprietary data. Only with these factors can organizations unlock the full impact and move from experimentation to scale. I know from my attendance of the Organizational Design Forum’s 2024 conference in May that there is a good deal of thought being given to these topics by top notch organization design teams.

High Performers and Best Practices

Gen AI high performers are much more likely to follow risk-related best practices. For instance, they seek legal input from the beginning and assess risk through periodic reviews at the development stage - in essence, “shifting left.” High performers also report challenges such as implementing agile ways of working and effective data governance.

Moreover, high performers focus on testing and validation in the model release process and develop clear processes for iterative improvement. Over time, these practices will be crucial as highly customized solutions become the main source of differentiated value.