Product Sense for AI

I was building a RAG-based workflow system. The assumption driving every architectural decision was simple: more context produces better output. Bigger context window. More retrieved chunks. Richer system prompt. I packed it full.

It failed in a specific way that took me a while to articulate. Not catastrophically. Not obviously. The system would surface ten retrieved documents instead of three, and suddenly the model would start hedging, generating conflicting answers, or latching onto a marginally relevant passage and ignoring the clearly relevant one. Latency crept up.

Confidence scores dropped. Users started second-guessing outputs that should have been straightforward.

That experience is the clearest example I know of how AI product sense differs from classical product sense. In a classical system, more data usually helps. In a RAG system, more context can actively hurt if the retrieval quality does not match the quantity. The real challenge was never context maximization. It was relevance precision. I had confused a capability (large context window) with a strategy (more context is better). Those are not the same thing.

Product sense for AI requires you to understand not just what a model can do, but where it breaks, and to design systems that stay inside the capability envelope rather than constantly pushing against its edges.

Classical product sense was forged in a deterministic world. Click a button, get a predictable response. The failure modes were bugs: either the code ran or it did not. Trust came from consistency. A product that worked 99 times worked the 100th time too.

AI product sense operates in different physics.

The output is probabilistic. The same input produces different outputs at different temperatures, different model versions, different context lengths. The product is not a screen and a database. It is a model plus prompts plus tools plus evals plus UI plus guardrails plus rollout policy. Failure modes are not bugs to fix but statistical distributions to bound and shape. Trust is not a default state earned by uptime. It is something you earn every single interaction, and one bad output at the wrong moment can erase weeks of goodwill.

If your answer to “design an AI feature for X” could have been given five years ago by replacing “AI” with “an app,” you have not demonstrated AI product sense. The probabilistic, eval-driven, cost-aware nature of the system must shape every part of your reasoning.

The strongest interviewers at OpenAI, Anthropic, Google DeepMind, and Sierra probe three areas that classical PM loops largely ignore: Model behavior intuition. What is the model good at today? Where does it break? What is its failure distribution in this specific job-to-be-done?

Probabilistic UX. How do you design for a system whose output is not guaranteed?

Citations, confidence indicators, abstention, undo, verify-mode, escalation — these are not nice-to-haves. They are the product.

Eval thinking. How do you measure “good”? What is your offline eval set? How do you bridge offline to online? How do you prevent Goodhart drift on a generative metric?

The differences are structural, not cosmetic. Let me go through the ones that matter most.

The product definition itself changes. In classical PM, the product is screens, flows, and database state. In AI PM, the product includes the model, the prompts, the tools, the eval suite, the guardrail policies, the rollout gates, and the fallback behavior. A change to any of these is a product change. Most candidates understand this in the abstract. Very few design it in practice.

Prioritization needs new dimensions. Classical reach-times-impact-times-confidence-over-effort still applies. But you need to add: cost-of-being-wrong (what happens when the model is wrong?), reversibility (can the user or system undo a bad output?), containment (is a failure bounded to one user or does it propagate globally?), eval coverage (do you have a credible way to measure quality?), and cost-per-call (do the unit economics work at scale?).

Success metrics expand. Engagement and retention are necessary but not sufficient.

You need hallucination rate, abstention rate, task completion rate, eval-suite accuracy, acceptance rate, regenerate rate, and citation click-through. The engagement trap is real: time spent in an AI product can rise because the model is bad and users keep retrying.

Always pair engagement with task completion.

Safety is a daily product surface. In classical PM, safety is mostly a compliance checkbox. In AI PM, refusal behavior, harmful content boundaries, jailbreak resistance, and vulnerable-user handling are product surfaces with metrics, trade-offs, and eval sets. You do not get to mention safety in one sentence at the end of your answer.

Roadmaps change character. Feature-based quarterly roadmaps break when the underlying platform shifts every six months. AI roadmaps must be outcome-anchored with explicit capability re-evaluation cadence.

The five questions an AI PM must always be able to answer, whether asked directly or not: Why does this need AI and not a rules engine? What does the model do well and badly in this specific job? How will the user know when to trust the output? How will you measure success offline before launch and online after? What goes wrong, and what is the user’s experience when it does?

Frameworks are scaffolding, not scripts. Use them to organize your thinking, then go beyond them with specifics. Here are the four most useful for AI PM product sense rounds.

GUPSE-R (Goal, User, Pain, Solution, Eval, Risk) is a six-step structure adapted from CIRCLES for AI-native problems. Best for greenfield “design X” prompts. Goal: clarify the prompt and pick a sharp objective. User: pick a specific segment with a real, painful job-to-be-done. Pain: one or two concrete pains prioritized by frequency and severity.

Solution: two or three AI-native solutions; pick one and design it. Eval: how will you know if it works, offline and online? Risk: what can go wrong, and what is the mitigation?

triangulation tool. For every solution you propose, check all three corners explicitly. Is the pain frequent and severe enough to justify the build? Can current models do this reliably, and what is the failure mode? Do the cost-per-call and willingness-to-pay work?

around who, what, why now, why us, how do we know it worked. Better for senior PM rounds where freshness matters more than structure.

Marily Nika’s AI Product Lens applies three lenses to any AI feature: capability lens (what can the model do), trust lens (how will users believe it), and integration lens (how does it fit into the user’s existing workflow). Useful for deep-dive phases.

In the room, do not announce the framework. Walk the interviewer through your structure conversationally: “Let me start by getting the goal right, then I’ll pick a user, dig into their pain, propose a few solutions, and discuss how I’d measure and de-risk this.” Most interviewers care less about which framework you use than whether you cover Eval and Risk explicitly.

Beyond frameworks, AI PMs need a mental library of design patterns: reusable solutions to recurring AI product problems. These appear in almost every strong product sense answer.

Decide upfront where the human sits relative to the model’s output. Human-before means AI suggests, human accepts — autocomplete, draft emails. Human-after means AI acts, human reviews — auto-categorize, then audit. Human-around means AI runs autonomously but is escalated on edge cases — Sierra-style support agents. Fully autonomous means AI acts without human oversight, and this belongs only in low-stakes, reversible actions.

Most candidates default to fully autonomous because it sounds more impressive. The better answer usually involves human-around with clearly defined escalation triggers.

This is the single biggest differentiator between AI products users love and AI products users abandon. Five tools in your trust stack: citations and sources (the highest trust ROI in any RAG product), confidence indicators (explicit “I’m not sure” or implicit color and font weight signals), abstention (“I don’t know” beats a confident hallucination every time), undo and edit (reversibility removes the cost of being wrong), and verify-mode (a dedicated UX where every claim is grounded and checkable).

When you propose any AI output in an interview, immediately follow it with the trust mechanism. “The model proposes X. Citations to the source document are inline, and the user can accept, edit, or regenerate.” That one habit separates strong AI PM candidates from average ones.

The blank prompt box is a discoverability disaster for non-power users. Better patterns: examples-first (show three to five starter prompts with one click), task gallery (organize entry points by job-to-be-done), just-in-time tutorials (triggered by user friction, not at launch), progressive disclosure (hide power features until the user is ready). Most AI features underperform because users never discover what the system can do for them.

Retrieval-augmented generation is the dominant pattern in enterprise AI. The UX choices that matter: citation density per paragraph versus per claim, source previews that let users see the source snippet rather than just the title, disambiguation UX that asks the user when retrieval is weak rather than guessing, and search bar persistence that lets users see and edit what was retrieved.

My RAG lesson applies here directly. Relevance precision matters more than context maximization. Design for a system that knows when to say “I found three highly relevant documents” rather than one that always returns ten with diminishing returns.

Design the failure UX before designing the success UX. Most candidates only design the success path. Real AI products spend more effort on failure paths than on success paths.

Common failure UX patterns: fallback to deterministic (“here is the most relevant result I found” instead of “I don’t know”), escalation (route to human or a more capable but slower model), reset (give the user a clear “start over” path), and transparency (surface that something went wrong rather than faking success).

Agent Design vs. Chat Design vs. Inline AI Three macro form factors. Chat is for open-ended exploration and multi-turn dialogue.

Inline AI sits inside an existing workflow — Copilot in Word, ghost text in an IDE — with lower switching cost but lower discoverability. Agents plan and act on the user’s behalf over multiple steps: best for clearly bounded workflows, and they demand strong guardrails because the cost of a bad action compounds across steps.

Cost per call varies by 100x across models. This shapes the product: route small-and-fast by default and escalate to frontier when needed, apply token budgets that cap context per session, cache identical queries, use asynchronous mode for work where latency is not critical. This is not an engineering footnote. Every routing decision is a user experience decision in disguise.

Memory tiers and persistence policies: session memory lives within a single conversation with no persistence, project memory is scoped to a workspace and persists with explicit opt-in, user memory crosses sessions and must be inspectable and erasable. Every memory tier raises a new privacy question. Design for it explicitly.

A common pacing problem in AI product sense interviews: candidates spend 20 minutes on user and pain and run out of time before evals and risks. Use this default time budget.

clarifying questions, pick a user segment

pains, pick the sharpest

score them

failure modes, design choices

guardrail metrics

switch

next step If you are running over, cut your solution count, not your eval. The most common scoring deduction is “no clear measurement plan.” Strong clarifying questions demonstrate AI fluency: “Are we assuming current model capabilities or future ones?” signals your assumption set. “What is the cost ceiling per interaction?” signals you understand AI economics. “Do we have eval data, or are we

starting from scratch?” signals eval-first thinking. Avoid “who is the user?” — it is too generic and not AI-specific.

Strong AI PMs generate solutions across the full AI surface area, not just “a chatbot.” Build the habit of brainstorming across eight AI surfaces: inline AI inside an existing workflow, chat or conversational interface, agent that acts on the user’s behalf, RAG over a knowledge base, multimodal (voice, vision, mixed), personalization via memory, background AI (proactive, batch, async), and verify-mode (grounded, citable, slower).

For each surface, consider low-friction, medium-friction, and high-trust variants. In the room you only have time to discuss three or four. Always include the obvious solution first — it anchors the interviewer. Then show your thinking by improving on it.

Every solution sentence should explicitly name what AI is doing — retrieval, generation, classification, agentic action — and why a non-AI approach would fail. If your solution could have been built five years ago, you are not showing AI product sense.

Classical reach-times-impact-times-confidence-over-effort is necessary but not sufficient.

The extended scoring dimensions:

solved?

output?

quality?

time?

Quick example: an AI email auto-replier versus an AI calendar booker. The auto-replier has high reach, medium impact, low cost-of-being-wrong (user reviews before sending), and is reversible. Ship it. The calendar booker has medium reach, high impact, high cost-of-being-wrong (double-booked CEO call), and is partially reversible. Ship it — but with a strong human-in-the-loop confirmation step and explicit trust UX.

Example 1: Improve ChatGPT Segment: serious knowledge workers using ChatGPT for research and writing five or more times per week. The top pain is verification — they love the draft speed but distrust the facts, so they spend time re-checking every factual claim. Secondary pain: long conversations lose context.

Solution pick: Research Mode plus a Critic second model. Flow — user opts into Research Mode for serious work. Every factual claim is post-processed by a Critic that checks grounding, flags low-confidence claims, and inserts citations. UX: claims are color-coded (grounded, inferred, uncertain). Hover reveals source. One tap to verify externally. Failure UX: when no source is found, the Critic forces abstention.

Eval: 500 factual prompts with ground-truth answers. Measure hallucination rate, abstention precision, citation accuracy. North star: “verified claims kept by user” per session. Guardrails: do not ship if abstention rate exceeds 25% or hallucination rate exceeds 5% on eval. Risks: latency adds 2-4 seconds (stream per-claim rather than upfront), over-abstention (sampled audit), cost increase of 30-50% (restrict to paid tier initially).

Segment: PMs and operators who live in Notion at their desk but barely open it on mobile because typing on a phone is slow. Pain: capturing ideas and meeting notes when away from a desk. The keyboard is the bottleneck; voice and camera are underused.

Solution: voice-to-structured-page. Tap mic on the home screen widget, speak naturally, AI structures into headings, bullets, action items, and tags. User confirms with one tap. Trust UX: editable preview before saving, full transcript stored alongside, one-tap undo.

Eval: 200 voice recordings of typical PM monologues, judged on transcript accuracy and structural fidelity. North star: “pages created from mobile that are still accessed within 7 days.” Risks: noisy environment (robust audio preprocessing), privacy concerns over voice (on-device transcription where possible), incorrect database routing (default to a single Mobile Inbox if uncertain).

Segment: knowledge workers in Operations who live in Outlook, Teams, and Excel. Top pain: Copilot suggestions feel disconnected from actual work because the system does not know what the user is doing across the suite. Secondary pain: trust — the user is not sure whether Copilot hallucinated.

Solution: ship enterprise-grade citations and a workflow agent together. Citations: every Copilot response cites the source — a SharePoint doc, an Outlook thread, a Teams message — with permission-aware previews. Workflow agent: predefined “plays” (meeting follow-up) triggered by the user, with explicit confirmation at each step. Trust UX: admin audit of every Copilot action, “show your work” mode before any tool call.

Eval: 300 enterprise scenarios labeled by SMEs across departments. North star: “Copilot tasks accepted by users that they would have otherwise done manually” per week per seat.

permissions when retrieving sources.

Two users: policyholders who want fast resolution and support agents who want to focus on complex cases. Agent owns tier-1 contacts (policy lookup, claim status, billing, document upload). Tools include: read policy DB, read claims system, update payment method with explicit confirmation, upload documents, escalate to human.

Trust UX: every action shown to the user before execution, explicit confirmation for any change, audit log accessible to compliance, user can request a human at any point. Eval: 1,000 real anonymized historical contacts labeled by senior agents. North star: “cases auto-resolved correctly per week” verified by sampled audit, not just CSAT. Guardrail: false-resolution audit rate below 1.5%. Critical risks: regulatory misstatement, PII exposure, distress detection for edge cases like death notifications arriving via chat.

Ambient AI listens to the visit with explicit patient consent. Produces a structured SOAP note draft and billing code suggestions immediately after. Highlights uncertain fields for physician review. Trust UX: verbatim transcript alongside structured note, confidence highlighting, mandatory physician review and sign-off, adverse-event reporting workflow.

Eval: 200 simulated plus 200 real consented visits scored by senior physicians. North star: “minutes of charting saved per physician per day.” Long-tail risk: physician over-trust (“rubber stamping”). UX nudges: rotate which fields are highlighted, periodic attention checks.

Verify-mode-only product — every claim is grounded in case law with inline citations.

Hybrid retrieval: vector plus keyword plus jurisdiction-aware filters. Citation engine: validates that cases exist, are current, and have not been overruled.

Zero-hallucination posture: if a citation cannot be verified, the model abstains or asks for clarification. Guardrail: any hallucinated citation in production is a P0. Deploys blocked until the eval suite proves the fix. North star: “hours saved per associate per week with zero hallucinated citations in production.”

Example 7: Voice Cooking Assistant Voice-first with optional camera for “look at my pan.” State-aware: tracks where the user is in the recipe and resumes after interruptions. Failure UX: if uncertain about a food-safety question, the assistant asks rather than guesses.

Safety-critical eval: allergy miss rate must be zero. Any miss is a P0 failure. North star: “recipes completed start-to-finish with assistant.” ASR accuracy target: under 8% word error rate in clean environments, under 18% in noisy.

User provides a research goal. Agent opens browser tabs, gathers public information, summarizes per source, drafts a memo with citations. Human-in-the-loop: agent pauses at key decision points. Trust UX: user watches live actions in a side panel, pause or intervene at any time, no write actions without confirmation, audit log stored and replayable.

Critical risk: prompt injection from web content takes over the agent. Strict tool boundaries and content firewall between retrieved text and instructions. North star: “memos produced per analyst per week with senior-analyst approval rate at or above 80%.” Scoring Rubric

positioned to solve.

chatbot.” Multiple solutions considered.

Failure UX designed.

abstention, undo named explicitly. Safety risks acknowledged.

north star, guardrail metrics. Goodhart-aware.

tradeoffs explicit. Phased rollout with kill switch.

Weak (likely no-hire signal): Generic persona. One vague solution that could have been built five years ago. No mention of evals or measurement. Trust UX absent or hand-waved.

No risks named, no rollout plan.

Good (likely lean-yes): Specific persona, clear pain. Multiple solutions with one picked and detailed. Trust UX named at high level. Metrics mentioned but generic. Risks named generically.

Great (strong hire signal): Sharp persona tied to AI’s structural strengths. AI-native solution with explicit failure UX. Trust UX integrated throughout, not bolted on. Specific eval set with size, source, slices, guardrail metrics, and Goodhart guard. Phased rollout with named percentages and kill switch. At least one novel insight or surprising tradeoff.

“Classical PM with AI sprinkled on.” Describing a feature that could have been built five years ago, then noting that AI “powers” it. Every solution sentence should explicitly name what AI is doing and why a non-AI approach would fail.

No evals. The single most common scoring deduction. Reserve at least five minutes for evals. State the offline eval set, the online north star, and one guardrail metric.

Hand-waving cost and latency. At least once in your answer, mention model routing, token budget, or latency-versus-quality tradeoff.

No failure UX. Designing only the success path. Explicitly describe what happens when the model is wrong, uncertain, or refuses.

Safety as a checkbox. Mentioning safety in one sentence at the end. Integrate safety throughout — in refusal behavior, trust UX, abuse-resistance in rollout, and vulnerable-user handling.

Vanity metrics. Pair every engagement metric with a quality or trust metric. Engagement plus task completion. Retention plus thumbs ratio.

No tradeoff. Proposing a solution that is allegedly better on all dimensions. Name the tradeoff explicitly: “This costs 30% more in inference but reduces hallucination by 50%.” Solution-first thinking. Always do user and pain first, even if you have a brilliant solution in mind.

The single thing that separates a good answer from a great one: the candidate designs the eval before the rollout, not after. When you say “before I ship this, I’d build an eval set of 300 representative cases across these five intent slices” in the first half of your answer, you have already demonstrated something most candidates never show — that you treat probabilistic systems as needing proof of quality, not just proof of concept.

• ☐I can articulate why AI product sense is structurally different from classical product sense in at least three specific dimensions.
• ☐I have a mental library of 12+ AI-specific design patterns and can name the right one for a given problem in 30 seconds.
• ☐I know the GUPSE-R framework and can deploy it conversationally without announcing it.
• ☐My solutions include the trust mechanism immediately after describing the output, not as an afterthought.
• ☐I reserve at least five minutes for evals in every 45-minute answer.
• ☐I can name a specific eval set (size, source, slices) for any solution I propose.
• ☐I name at least one cost, latency, or routing tradeoff in every answer.
• ☐I design the failure UX alongside the success UX, not after it.

Product sense for AI is not about being clever. It is about being honest about what your system cannot do — and designing around that honestly.

The blank context window is not a sign of capability. Relevance precision is. The confident answer is not a sign of trust. The citable answer is. The feature that works in the demo is not a sign of quality. The feature that works in production, on the long tail of user inputs, at 3am on a Tuesday — that is the sign.

Design for that.