Outcome at a Glance

• Aligned 15 product managers around one ranked list of 25 capabilities, used to plan the next year of development investment

• Validated the roadmap: the highest-ranked capabilities were already shipping or in flight

• Identified where to accelerate, and reframed go-to-market messaging in customers' own language

The Challenge

AI Control Tower launched in May 2025 to high expectations. The roadmap held far more candidate capabilities than the team could build, and 15 PMs each had a strong case for their own area. Leadership needed to know what customers valued most before committing a year of investment, and the team needed an answer that PMs from inventory, governance, risk, and value would all accept.

Mixed Methods at a Glance

• Quantitative: MaxDiff (best-worst) survey, N=417. Two segments: strategic AI decision makers (n=219) and AI product owners/managers (n=198). 25 capabilities, five per screen, 15 tasks per respondent, plus an anchor question.

• Qualitative: 25 in-depth interviews with admins, plus open-ended survey questions on challenges and integrations.

• Analysis: Choice modeling into a preference score per capability, segment comparison, thematic coding, and a mapping of results to the roadmap. A quant UXR peer-reviewed the analysis.

• Recruitment: Vendor-recruited and screened on AI role and governance involvement. I also workshopped capability wording and open questions with our quarterly Product Advisory Council.

How I Did It

1. Scoped the problem with the people who owned it. I ran a roadmap planning workshop with 15 PMs to collect candidate capabilities and narrow them to the 25 highest-priority, then rewrote each in plain, customer-facing language. Reaching one list took several rounds of negotiation, because every PM had a legitimate case.

2. Pressure-tested before spending budget. I took draft statements and survey questions to our Advisory Council to catch confusing wording, so the survey measured priorities rather than comprehension.

3. Quantified priorities with forced trade-offs. MaxDiff makes people choose, so "everything is important" isn't an option. I designed the survey, wrote the screener, managed the vendor, and modeled the results into preference scores for each segment.

4. Added depth with interviews. I conducted 25 interviews to learn why customers ranked things as they did and where governance breaks down in real workflows. Three themes emerged: fragmented visibility, manual governance bottlenecks, and no way to prove AI value. As one participant said, "We're flying blind."

5. Connected it to the roadmap. I mapped every capability against the roadmap and sorted each as a validated strength, an opportunity to accelerate, or a candidate to defer. I then ran a readout workshop with Product, Design, GTM, and leadership.

Where Qual and Quant Worked Together

• Interviews gave the "why," the survey gave the "how much." The interview themes pointed to the problems; the survey showed which solutions mattered most and whether both segments agreed. They did on four of their top five.

• Forced trade-offs corrected what people say. Open-ended answers stressed bias and ethical AI. When respondents had to choose, those items ranked below visibility and value tracking. We recommended watching them rather than building them first, a call open questions alone would have gotten wrong.

• Numbers sized the opportunity. A conversational insights interface ranked first in both segments. Among strategic leaders it earned 13% preference, about 3x the 4% expected if all 25 capabilities mattered equally.

• Segments differed in useful ways. Product owners pushed harder on automated approvals and governance built into developer tools, while strategic leaders emphasized KPI tracking and measurable value. That shaped who each recommendation was for.

Results and Business Impact

• One shared plan. A single ranked list replaced competing opinions across 15 PMs and became the basis for planning the next year of development investment.

• A de-risked roadmap. The highest-priority capabilities were already shipping or in flight, which confirmed existing investment.

• Clear places to speed up. I recommended accelerating governance automation inside developer tools, value dashboards and KPI tracking, and conversational insights, and flagged a multi-cloud integration gap from the integrations question.

• A sharper story for the market. I recommended moving messaging from "governance and control" to "unified visibility and measurable value," using the words customers used.

What I Owned

Everything end to end, independently: stakeholder alignment, Advisory Council sessions, study design, screener and MaxDiff design, vendor management, interviewing, analysis, synthesis, and readouts to Product, Design, GTM, and leadership. I collaborated with a vendor for recruitment and a quant UXR for analysis review.

Reflection

• Narrow harder, earlier. I let too many features into the study. Next time I'd set a firm cap up front, use explicit criteria to push back, and follow up afterward to see how the priorities were applied.

• Pair what people say with what they do. I'd bring in product usage data earlier to compare stated priorities with actual behavior.

• Scale it to the time available. This study was built for a high-stakes annual plan. With less time, I'd run the same logic in weeks: a shorter MaxDiff of 10-12 items, a quick panel check on wording, and a small set of interviews, so teams get a directional answer fast and can decide how much more rigor is worth it.