Most executives don’t think of their AI rollout as a project. They think of it as a purchase. Buy the tool, license the seats, announce it in an all-hands, and wait for the productivity numbers to show up on their own. Six months later, the numbers haven’t shown up, and nobody can say why, because nobody defined what “showing up” was supposed to look like in the first place.
Donna Medeiros, VP of AI and Data Advisory at Data Society, has watched this pattern repeat across industries long enough to name the root cause. It isn’t a talent gap or a tooling gap. It’s a management gap. Organizations are running AI like a software deployment when they should be running it like a project with a scope, a sponsor, a budget, and a set of numbers everyone agreed to before the work started.
Governance Is Not a Binder, It’s a Program
Ask ten companies what “AI governance” means and you’ll get ten answers involving policies, acceptable-use documents, and a committee that meets quarterly. Donna’s view cuts against that framing entirely.
“Governance becomes that use case, project implementation of something important for a business outcome, instead of just we’re putting policies, procedures, controls in place. There’s going to be some program management to it, like you’re running a project for use cases that are important for business outcomes.”
That distinction matters more than it sounds. A policy binder tells people what they can’t do. A program tells people what they’re trying to achieve, who owns it, and how progress gets measured along the way. One is a compliance artifact. The other is a management discipline, and it’s the difference between AI adoption that produces evidence and AI adoption that produces anecdotes.
Companies that have taken the AI use case prioritization framework seriously tend to arrive at this same conclusion from a different angle: you can’t prioritize what you haven’t scoped, and you can’t scope what you haven’t turned into an actual project with boundaries: https://datasociety.com/ai-use-case-prioritization-framework-2026/
Why This Gets Complicated Fast
If running AI like a project were simple, more companies would already be doing it well. Donna is candid about why it isn’t.
“The reason why it is, it’s complicated, it’s use case. And then the business line and those that are subject matter expertise for that business line use case has to develop the KPIs, and then there has to be the means to track it.”
Read that carefully and you’ll notice it’s not a technology problem. It’s an ownership problem. The people who understand whether a use case is working, really working, in the way that changes a business outcome, are not the AI team. They’re the subject matter experts sitting inside the business line where the use case lives. A claims processing improvement has to be judged by claims leadership. A sales enablement tool has to be judged by sales leadership. The central AI or IT function can provide infrastructure and support, but it cannot, and should not, be the sole author of what success means for someone else’s workflow.
This is exactly where a lot of executive teams get it backwards. They centralize AI decision-making in a technology function and wonder later why adoption in the field is thin. The fix isn’t more central control. It’s pushing KPI ownership out to the business line and holding that line accountable the same way you’d hold any project sponsor accountable for a capital investment. The organizations getting this right are the ones treating AI prioritization as a business discipline first, not a technology rollout that happens to involve the business: https://datasociety.com/ai-isnt-the-problem-prioritization-is-heres-where-most-organizations-get-it-wrong/
Benchmark Before You Build
If there is one instruction Donna repeats with the most force, it’s this one, stated as plainly as possible.
“You got to benchmark first.”
It is a short sentence, and it is also the single most skipped step in enterprise AI deployment. Teams get excited about a new capability, roll it out, and only start asking “did this help?” after the fact, at which point there’s no baseline to compare against. Was the process faster than before? Nobody measured before. Are customers happier? Nobody captured the prior satisfaction score. Did costs go down? Nobody isolated the cost structure before the tool arrived.
Benchmarking isn’t a bureaucratic delay tactic. It’s the only way an executive team can walk into a board meeting a year later and make a claim with evidence behind it instead of a feeling. Without a benchmark, every ROI conversation about AI degenerates into opinion. With one, it becomes a project status update, which is a conversation executives already know how to have.
The Toyota Proof Point
Donna doesn’t just argue for benchmarking in the abstract. She points to a real example of a company that did it in sequence, correctly.
“We had a case study at Toyota, but they benchmarked before they put a lot of governance platforms in place and said, this is what it’s taking now. And then they put some tracking mechanisms in place so that they could measure how AI affected things.”
Notice the order of operations. Toyota didn’t buy governance software and hope insight would follow. They measured what “now” looked like first, in concrete operational terms, before a single new platform or control was introduced. Only after that baseline existed did they layer in tracking mechanisms designed to isolate AI’s actual effect on the metric that mattered. That sequencing is what separates a defensible ROI story from a marketing slide.
This is the same discipline that shows up in Data Society’s own research into where organizations are losing money on AI investment. The blind spot isn’t usually the technology itself. It’s the absence of a measurement plan that existed before the technology arrived: https://datasociety.com/the-632-billion-blind-spot-in-ai/
What This Looks Like in Practice
For an executive translating this into action, the sequence Donna describes breaks into five concrete moves:
Pick one use case at a time. Resist the temptation to declare an “AI transformation” across the enterprise simultaneously. A use case is scoped, ownable, and measurable. A transformation is none of those things until it’s broken into use cases.
Assign a business-line owner with real subject matter expertise. Not a project coordinator. Someone who understands what “better” means for that specific workflow and can define the KPI without translation.
Define the KPI before the tool touches production. Time saved, cost reduced, adoption rate, customer or employee satisfaction, whatever fits the use case, but agree on it in writing before launch, not after.
Benchmark the current state. Capture the “this is what it’s taking now” number Donna describes, in the exact terms the KPI will later be measured against.
Track continuously, not just at the end. Program management means checking in throughout the project, the same way you’d check in on any initiative with a budget and a sponsor, not waiting a year to ask if it worked.
Executives who build this rhythm into their AI initiatives stop having faith-based conversations about AI value and start having the kind of conversations they already have about every other capital investment: grounded in a baseline, a target, and a track record.
Frequently Asked Questions
AI project management means treating each AI use case like a formal project, with a defined business outcome, a subject-matter-expert owner from the business line, agreed-upon KPIs, and a tracking mechanism, rather than treating AI as a tool rollout governed only by policy.
Benchmarking establishes the current-state baseline for time, cost, or quality before AI is introduced. Without it, there is no reliable way to measure whether AI actually changed the outcome, which makes ROI claims impossible to defend later.
The business line and subject matter experts closest to that specific use case should own the KPIs, not a centralized AI or IT team, because they understand what success actually looks like in that workflow.
Toyota benchmarked current performance before introducing governance platforms, documenting what the process took beforehand, and then implemented tracking mechanisms afterward to measure AI’s actual effect on that baseline.
AI governance is often reduced to policies, procedures, and controls. AI program management goes further, applying project discipline, ownership, KPIs, and benchmarking, to specific use cases tied to real business outcomes.
Ready to Put Structure Behind Your AI Roadmap?
If your AI initiatives are still running without a benchmark, a KPI, or a named business-line owner, that’s the gap to close before you scale anything further. Book time with Donna to map the program management structure your next AI use case actually needs: https://meetings.hubspot.com/donna-medeiros/meet-with-data-societys-ai-and-data-advisor
