At the World Procurement Congress AI Forum, the ORO roundtable on “Making AI Real in Procurement” surfaced a clear theme: the technology is racing ahead, but operating model architecture, governance, people and trust are struggling to keep pace. In a room of senior procurement leaders, only a minority had agents live in production, and even fewer had built an orchestration layer around them to provide workflow context and governance. At the same time, the ambition was unmistakable: participants imagined agents that could tackle everything from tail spend to third-party risk, while acknowledging the need for much stronger foundations before they can safely scale.
When the Agent Learns the Wrong Lesson
Chris Vessey opened the session with two disarming stories about AI “drift” that felt uncomfortably familiar to many in the room. In one, a model used all week for formal work emails was suddenly asked to write a birthday message and produced something so stiff that the recipient called to check in on the sender. In another, after a burst of whimsical creative writing, the same system was asked to draft procurement strategy language, and responded with a fantastical tale about “worry monsters of maverick spend,” and “tangles of unstructured vendor data,” that was poetic, but unusable.
Both anecdotes made the same point: the agent had learned the wrong lesson. It took recent patterns and generalized them everywhere, regardless of context. “The problem isn’t intelligence anymore,” Chris noted. “It’s what the system thinks ‘good’ looks like based on the data and behaviour you feed it.” In a personal inbox, that is embarrassing but recoverable. In an enterprise procurement stack, the equivalent is an agent quietly applying skewed or outdated logic across thousands of transactions a month. The room quickly connected this to a real risk: if you can’t see how your agents are learning, you can’t see how they might be drifting.
The Maturity Check: Agents Without Explanation
A quick show of hands made the gap quite visible. Less than a quarter of organizations had one or more procurement agents in action today, and of those, two thirds of the live agents were working on fairly constrained use cases like purchase requisition checks. Most enterprises continue to experiment, piloting, or thinking in terms of point automation rather than orchestrated, multi-agent systems.
Even more telling was the response to questions on governance. When asked how their agents were being controlled today, participants cited tactics like locking down data, extensive testing, defining rules, and limiting access and data exposure. Those are necessary safeguards, but they sit around an agent, not inside its learning loop. Very few in the room were ready to raise a hand when asked whether they could confidently explain a specific agent decision; why a given transaction was approved, blacked, or routed the way it was. That silence pointed to a core issue: designing decision boundaries, accountability, and an operating model where intelligence can move quickly without enterprises losing control or adding risk.
Feedback Loops as Embedded Governance, not a Bolt-on
To move from anxiety to action, the discussion turned to a live example: ORO’s PR Review Agent. In this example an AI agent validates and amends 27 fields on a purchase requisition, escalating to a human or another agent where required. The interesting part is not just that it handles more than 15,000 transactions a month across multiple customers, but how it learns. A sample of individuals are regularly asked to review the agent’s decisions and either confirm or challenge them. When they disagree, they must explain why. Those explanations are treated as training data, feeding back to the agent team as structured course corrections.
This reframes governance not around distrust of agents but rather the assumption that the agent will learn and evolve from what it learns from. Chris summed up the ORO mindset simply: “Agents execute. Orchestration governs. People lead.” Certifications like ISO/IEC 42001 are necessary but not sufficient. Agent-level Governance is achieved via the feedback loop between people, agents, and orchestration, rather than a static set of rules or access controls bolted on after the fact. In a world where procurement leaders rightly worry about drift, this model resonated strongly: the feedback loop is critical to dynamic governance.
Ambition vs. Readiness: The AI Gap
Once the room had a concrete model of “good,” the conversation shifted to the art of the possible. If an agent could do one thing in procurement - no limits - what would it be? The answers came quickly: tackle third-party risk management end-to-end; maintain master data quality and integrity; drive demand forecasting; monitor and manage tail spend; deliver touchless P2P compliance; harmonize specifications; analyze bids and responses; continuously compare suppliers against diverse data sources; and even augment payment teams’ workflows.
Then came the harder question: for that vision to be safely deployed, what would need to be true? Here, the answers shift in tone with leaders pointing toward the need to simply get started, run pilots, and scale fast. Engaging IT early, proving ROI with solid business cases, and building quality deployment layers focused on control and governance. Others highlighted the importance of an operating model that can make decisions and adapt quickly, not just a technology stack that can execute. The distance between those two lists, the unconstrained wish list and the single prerequisite was the group’s AI readiness gap.
From Rule-Following Automation to Autonomous Orchestration
The discussion closed by zooming out to where procurement is heading. An analogy shared was self-driving cars: early features like lane assist and emergency braking were impressive, but fundamentally rule-following. Taking the leap to full autonomy requires more than a smarter algorithm; it demands better sensing, mapping, fail-safes, and crucially, a new regulatory and governance framework. Procurement is at a similar point, with most agents today still doing only 5-10% of what is possible; automating tasks with a hint of creativity rather than operating as truly autonomous, collaborative “colleagues.”
Lance Younger, drawing on decades of procurement experience, framed it from a broader ecosystem perspective. For him, the real shift is that ERP-era innovation cycles have given way to an AI era where “the cycle and pace of change is rapid, and what has become incredibly important is the view of the complete ecosystem. You have a backbone, best-of-breed solutions, and then orchestration that brings together processes, data, agents, and people into a seamless flow.” In other words, agents without orchestration simply create new islands of intelligence.
ORO has moved beyond single agents toward orchestrated, domain-specific agent teams and agentic workflows, spanning intake, PR review, pricing optimization, MRO, sourcing opportunity identification, contract risk, and third-party risk management. The roadmap discussed at the roundtable pointed toward natural language configuration, telling the system what to do, pre-deployment simulation of changes on historical data, and agents that not only execute but suggest process improvements based on patterns observed across the organization. As Lance shared, orchestration is the “special sauce” that makes this shift usable: linking layers together to enable speed without chaos.
For many leaders in the room, the key takeaway was simple but demanding: the organizations that win with AI in procurement will not just be those that deploy agents first, but those who build intentional orchestration and feedback loops that keep learning, avoid drift, and are held accountable.
Want to learn about how ORO can transform procurement at your organization? Book a demo with one of our experts.
The AI Forum was moderated by ORO procurement experts:
Chris Vessey, VP Innovation and Customer Value at ORO Labs. Background: 20+ years as procurement practitioner at P&G, Goldman Sachs and JPMorgan Chase with multiple global transformation lead roles for P2P, spend management, contingent workforce, TPRM, as well as managing payables, sourcing ops.
Lance Younger, EVP, EMEA GM and Global Alliances at ORO Labs. Background: 30+ year career with 15+ in enterprise procurement and supply chain including Founder of ProcureTech, Partner in the procurement practice at Inverto/BCG and Deloitte, and CEO/Co-founder at Statess (SRM). Extensive background in scaling procurement technology ecosystems across EMEA, focusing on intake-to-pay automation and strategic supplier alliances.
By Kate Jeter, Director of Global Field Marketing
Kate Jeter is a strategic B2B procurement tech marketing leader 25+ years of experience specializing in field marketing, events, and demand generation for SaaS and enterprise platforms. Before ORO, she was the Head of Marketing, Community, and Growth at ProcureTech. She is an expert in aligning marketing and sales for revenue acceleration, pipeline growth, and global brand positioning.