AI for Product Discovery and New Demand
AI widens the search for unmet need; only source-linked struggle, costly commitment, adoption and repeated value can turn a generated possibility into demand evidence.
TLDR
- The discovery bottleneck is evidence conversion, not idea volume.
- Lead users and anomalous cases reveal mechanisms, not automatic mass markets.
- Synthetic customers, quotations and market sizes belong in quarantine.
- The next step is the smallest probe that can separate rival demand claims.
- A disciplined discovery system improves abandonment as well as selection.
The immaculate concept with no customer behind it
A product team asks a model for growth opportunities in small business finance. By lunch it has twelve polished concepts, synthetic personas, apparent interview quotations and a market-size range. One proposal receives executive attention because the narrative is unusually coherent. Nobody can identify an observed workflow, customer who built a workaround, existing alternative, costly consequence or person willing to change behaviour.
AI can make discovery faster by connecting scattered evidence, translating across domains, finding anomalies and generating rival problem frames. It can also supply an unlimited volume of plausible need. When every possibility arrives with fluent customer language, competitive analysis and a prototype brief, the organisation can no longer see which claims exist outside the model.
The bottleneck is not ideation. It is conversion of observed struggle into a falsifiable demand claim, followed by the smallest test that can distinguish value from narrative fluency. A model cannot experience a workaround, consent for a customer, commit a budget, adopt a product or demonstrate repeat value.
The AI-Augmented Demand Discovery Operating System developed here has twelve connected parts: Discovery Question and Search Charter; Opportunity Evidence Contract; Customer and Context Evidence Graph; Lead-User and Edge Signal Finder; Observation-to-Hypothesis Translator; Synthetic Hypothesis Quarantine; Problem, Job and Alternative Map; Concept and Mechanism Studio; Demand Evidence Ladder; Bounded Market Probe Registry; Opportunity Portfolio and Dependency Map; and Discovery Assurance and Learning Docket.
Begin with a decision, not a broad prompt
A Discovery Question and Search Charter names the strategic domain, customer and non-customer boundaries, decision that evidence must inform, structural change or customer progress under investigation, known facts, assumptions and open questions. It specifies lead-user, extreme and adjacent search paths; permitted sources; consent, privacy, confidentiality and intellectual-property limits; delivery constraints; time and research budget; rivals; expected evidence rung; owner and stop condition.
“What AI products can we build?” privileges a solution before the problem exists. A better question is where a consequential job is poorly served, what observation would establish that, which alternative currently wins and whether intelligence changes the mechanism relative to a simpler service, process or deterministic tool.
Search the market edges without declaring them the market
The Customer and Context Evidence Graph links field observation, interview, declared need, actual workflow, service records, purchase, usage, cancellation, non-consumption, search, community signal, telemetry, user-built workaround, sales observation, substitute, regulation, technology change, internal capability, generated hypothesis, probe, commitment, contradiction and superseded claim. Each node retains source, date, scope, permission, quality, uncertainty and downstream use.
Generated clusters link back to individual observations. Minority and contradictory cases remain visible. Frequency in customer text is not market magnitude: a few high-consequence needs may support a valuable niche, while thousands of mentions may reflect curiosity, complaint amplification or duplicated content rather than intent to change.
Lead users reveal mechanisms and future conditions
Search for people who face needs earlier than the market, gain unusually high benefit from a solution, build modifications, combine products in unintended ways or absorb hidden coordination work. Include non-customers excluded by complexity or trust, former customers, rejected prospects, extreme-scale users, service teams, partners and adjacent domains where future conditions already exist.
Lead-user research provides a mechanism for discovering novel concepts, and empirical work examines commercial attractiveness in particular settings. It does not license a leap from edge case to total addressable market. The edge reveals a problem mechanism and a possible solution. Transferability requires evidence from a defined population with different constraints.
| Source | What it can support | Bias | Permission | Expiry | Next validation |
|---|---|---|---|---|---|
| Field observation | Workflow, struggle and workaround in context | Small, selected setting | Purpose and participant boundaries | Process or context change | Discriminating observation elsewhere |
| Lead-user modification | Mechanism under extreme need | Not representative; expert capability | Rights and attribution | Alternative becomes standard | Transferability probe |
| Service or support record | Repeated exception and consequence | Only customers who contact | Permitted secondary use | Policy or product change | Observe silent users and non-customers |
| Search, review or community signal | Language, curiosity and candidate issue | Duplication, loud users and platform effects | Terms and privacy | Fast-moving discourse | Verify behaviour and costly priority |
| Customer commitment | Willingness to spend time, access, budget or change | Probe and recruitment context | Informed terms | Decision window | Delivery, adoption and repeat value |
| Generated hypothesis | Alternative frame or research question | Training data and prompt | No customer consent inferred | Model and context change | Replace with observed evidence |
Quarantine invention before it acquires a customer
A synthetic hypothesis is useful when its status is unmistakable. It can suggest a segment to observe, a mechanism to challenge, language to test or an alternative explanation for an anomaly. It becomes dangerous when the generated persona is counted as a participant, the simulated quotation is presented as voice-of-customer evidence, or a model-produced market range is treated as a measured population. Repetition across prompts does not create independence. Multiple models may reproduce the same training-data pattern.
The quarantine record therefore labels every generated statement with its model, prompt, date, human owner and intended research use. It forbids synthetic material from supplying frequency, prevalence, willingness to pay, behavioural consequence, population size or consent. Promotion requires an external source and a precise link between what was observed and what is now claimed. Contradictory evidence remains attached rather than being summarised away.
| Material | Permitted use | Prohibited inference | Promotion evidence |
|---|---|---|---|
| Generated persona | Recruitment screener draft | A segment exists or shares these priorities | Defined population and observed participants |
| Simulated interview | Interviewer training and rival questions | Customer language, prevalence or consent | Source-linked field record |
| Generated concept | Mechanism alternative | Desirability or differentiation | Behavioural probe against a comparator |
| Model market estimate | List of denominator questions | Market size or reachable demand | Population data, eligibility and adoption assumptions |
| Synthetic data | Pipeline testing and privacy-preserving rehearsal | Real distribution or rare-case frequency | Governed representative measurement |
Quarantine also protects customer evidence from a subtler failure: premature synthesis. Clustering hundreds of fragments can erase the one contradiction that changes the decision. Store the raw source, analytic transformation and claim separately. Let reviewers travel backwards from opportunity to evidence and forwards from evidence to the action it supports. If they cannot, the claim returns to quarantine regardless of how plausible it sounds.
Evidence freshness matters as much as provenance. A workaround observed before a policy change, competitor launch or price shock may no longer describe the decision customers face. Each contract names an expiry trigger and the owner who must refresh it. Refreshing does not mean asking a model to restate the old source. It means checking the external condition, sampling disconfirming cases and deciding whether the claim still warrants exposure. This is especially important when generated summaries circulate more widely than their underlying records. The summary inherits the shortest relevant expiry, not a new life of its own.
Map the struggle, job and existing alternative before the concept
Customers rarely wait in an empty market. They postpone, delegate, improvise, use a spreadsheet, employ a specialist, tolerate an error, combine tools or decide the result is not worth pursuing. Product discovery must explain that existing allocation of time, money, risk and attention. A concept competes with those alternatives and with non-consumption, not just with products in a category grid.
The Problem, Job and Alternative Map records the triggering context, progress sought, current sequence, workaround, consequence, people affected, switching constraint and evidence source. The observation- to-hypothesis translator then turns the record into a falsifiable statement: for a named population in a named situation, removing a named constraint should cause a measurable change relative to a specified alternative within a stated time. AI can propose several translations; the team chooses which observation each preserves.
Concepts are mechanisms, not commitments
The Concept and Mechanism Studio varies how a verified constraint might be removed without pretending that every variation deserves a build. One mechanism might automate a step, another coordinate two parties, another provide an assurance signal, and another eliminate the task. Each concept inherits the same opportunity evidence contract and names the assumption it uniquely tests. This allows divergence without manufacturing ten separate markets.
Climb an evidence ladder, one discriminating test at a time
Evidence for demand accumulates through different behaviours. A complaint can establish friction; it cannot establish a reachable population or a willingness to switch. A registration can establish curiosity; it cannot establish sustained use. A paid pilot can show authority and budget; it may still conceal extraordinary support, novelty effects or a one-off compliance deadline. Discovery fails when these rungs are treated as interchangeable.
The Demand Evidence Ladder has nine positions: generated question, source-linked signal, contextual observation, repeated pattern, verified consequence, alternative and switching map, costly customer commitment, adoption under normal conditions, and repeated value. Every opportunity stores its current rung, the strongest contradiction and the next evidence that would change the decision. Teams may stop on any rung; they may not describe a lower rung using the language of a higher one.
Make rival explanations compete
A discovery claim becomes useful when it could be wrong. For every favoured explanation, the rival hypothesis tournament records at least two credible alternatives and a negative control. Slow invoice approval might reflect missing information, but it might instead reflect budget scarcity or a deliberately cautious policy. A landing page can distinguish message interest; it cannot distinguish those operating causes. The next probe should maximise decision-relevant information per unit of customer burden, cost and irreversible work.
A negative control is not merely a failed treatment. It is an outcome, population or period that should remain unchanged if the proposed mechanism is correct. If a new invoice explanation is meant to reduce delays caused by ambiguity, it should change questions about missing documentation but not approvals frozen by an organisation-wide budget hold. Movement in both places suggests seasonality, selection or another shared cause. The control protects the mechanism claim.
Register bounded probes before exposing the market
The Bounded Market Probe Registry names the decision, population, proposition, channel, comparator, allocation, exposure cap, success and harm measures, delivery promise, data rights, accessibility, owner, expiry and stop rule before launch. It prevents an attractive click-through rate from expanding into an unapproved product promise. A probe must be truthful enough that a participant is not deceived about availability, price, use of data or human involvement.
| Unknown | Smallest useful probe | Behaviour | Guardrail | Do not claim |
|---|---|---|---|---|
| Problem meaning | Contextual observation and counter-example search | Repeated workaround | Consent and data minimisation | Prevalence |
| Message relevance | Truthful comparison or wait-list page | Qualified follow-through | No false scarcity or availability | Adoption |
| Workflow value | Concierge or Wizard-of-Oz service | Task completion and return | Disclose material human involvement | Scalable economics |
| Willingness to commit | Refundable deposit, letter of intent or scheduled pilot | Costly authorised action | Clear terms and easy withdrawal | Repeat value |
| Normal-condition value | Limited production pilot with comparator | Adoption, retention and outcome | Support load, safety and remedy | Population transfer |
Manage opportunities as a learning portfolio
A discovery portfolio should reveal evidence dependencies, not rank ideas with one blended score. Novelty, problem strength, reach, commitment, feasibility, rights, safety and strategic option value remain separate columns. A high average can otherwise hide a fatal zero, such as no lawful data access or no delivery mechanism. The Opportunity Portfolio and Dependency Map links each claim to shared evidence, enabling teams to see when one field study changes several concepts and when ten concepts merely duplicate one need.
| Opportunity | Strongest evidence | Rival or dependency | Next action | Stop condition |
|---|---|---|---|---|
| Maintenance fault isolation | Repeated expert workaround and downtime consequence | Asset-record quality may be the cause | Bounded cited-answer test | No safer or faster diagnosis |
| Micro-retailer cash visibility | Non-consumption caused by setup burden | Data access and trust | Consent-based concierge probe | No continued use after reconciliation |
| Automated compliance narrative | One paid deadline-led pilot | One-off need and manual delivery | Return to observation | No repeat workflow |
| Synthetic procurement persona | Generated story only | No external participant or population | Stop and quarantine | Already met |
| Duplicate invoice assistant | Problem evidence exists | Current product already resolves it | Stop duplicate | No distinct mechanism |
Eight cases reveal where plausible demand breaks
Case 1, the verified workaround. A lead user has built a spreadsheet that combines three exports before every weekly planning meeting. Observation confirms two hours of work, version errors and a delayed purchasing decision. Two comparable operators face the same mechanism, and one commits data access for a bounded prototype. This opportunity can advance to a concept test, but the claim remains limited to the observed operating context. It has not established a national market or proved that automation will be trusted under normal conditions.
Case 2, the loud feedback cluster. Thirty support messages request a dashboard. AI groups the language and makes the cluster look consistent, yet the underlying customers are reacting to three different problems: missing exports, delayed source data and an unclear definition. A dashboard may solve none. The record returns to observation, sampling customers who did not contact support as well as those who did. Message count is evidence of attention, not a design instruction or a population estimate.
Case 3, the truthful interest probe. A team has observed a recurring hand-off failure and mapped the current manual alternative, but willingness to change remains unknown. A page accurately describes a limited concierge service, its availability date and the human work involved. Qualified visitors may schedule a workflow review without surrendering unnecessary data. The probe can test costly follow-through against the existing process. It cannot yet support claims about scalable margin, retention or automated delivery.
Case 4, the governed hold. Clinicians repeatedly reconstruct medication histories across incompatible systems, and the safety consequence is well documented. The proposed mechanism depends on data that the discovery team is not authorised to combine, and the source system cannot yet provide reliable provenance. Strong problem evidence does not neutralise that dependency. The opportunity is held with named owners for rights, interoperability and clinical assurance. The team may study the workflow further but cannot expose patients to the concept as if delivery were ready.
Case 5, the synthetic enterprise buyer. A model produces a procurement persona, objections, tender language and an apparent budget range. Competitor pages confirm that the category has vendors, but no source establishes this buyer's workflow, authority or unmet consequence. The portfolio stops the demand claim and keeps only a recruitment hypothesis. If later interviews find the same language, each statement is linked to the real source rather than credited retrospectively to the simulation.
Case 6, the denominator-free market. Search volume and social discussion rise around a new regulation. A generated report multiplies the number of registered firms by an assumed price and labels the result an addressable market. It ignores which firms are affected, already compliant, reachable, able to switch or willing to pay. The market-size claim stops. A valid next study would measure eligibility and current response in the defined population before any revenue calculation.
Case 7, the solved problem under a new label. Field research verifies a frustrating approval step, but the company's existing product already removes it when configured correctly. A new AI concept attracts internal attention because its interface looks novel. The dependency map reveals identical evidence and mechanism. The new opportunity stops as a duplicate; product education or configuration repair proceeds under the existing owner. Portfolio discipline prevents novelty from competing with a working remedy.
Case 8, commitment without deliverability. Three customers sign letters of intent and offer paid pilots for a multilingual review service. Testing shows that rare-language cases require expert judgement unavailable within the promised response time. The team holds the opportunity rather than treating signatures as permission to improvise unsafe delivery. It can narrow the population, build qualified capacity or change the promise. If none makes the mechanism feasible, the stop rule protects customers and converts an attractive pipeline into a documented learning result.
Run discovery assurance as a decision cadence
A weekly review should ask which opportunity changed rung, what new observation caused the change, which rival strengthened, whose exposure increased and which stop rule approached. It should not reward teams for the number of interviews, prompts or concepts. A monthly portfolio review examines shared dependencies and releases capacity from duplicates and indefinite holds. Material customer harm, unlawful processing, fabricated evidence or an uncontrolled promise triggers immediate review rather than waiting for the cadence.
Decision quality can be audited through reversals. Review whether an opportunity advanced on evidence available at the time, whether a stopped claim later reappeared without new evidence and whether negative findings changed resource allocation. Record forecast ranges rather than precise fiction, and compare expected with observed adoption, support load and repeat value. The objective is calibrated learning: faster movement when evidence is strong, faster abandonment when it is not, and visible uncertainty everywhere in between.
The Discovery Assurance and Learning Docket records decision history, evidence lineage, permissions, model and prompt versions, challenge review, probe exposure, adverse effects, accessibility findings, commitments, delivery receipts, portfolio change and closure reason. A stopped opportunity remains searchable. This prevents a discarded synthetic claim from returning under a new name and lets future teams reuse verified observations without reusing expired conclusions.
Demand Discovery Decision Console
The console routes a live opportunity to exactly five outcomes. It does not score imaginative quality or estimate market size. It makes the decisive evidence boundary visible, identifies the next learning action and preserves an explicit stop. Choose one of ten named scenarios, then alter a field to test how the decision changes.
Run a low-cost demand probe before building
Use the smallest truthful exposure that distinguishes the opportunity from its strongest rival.
The strongest rival is abundant simulation plus fast market exposure
The strongest case against this system is practical. Traditional interviews are slow, selected and often poor at eliciting latent needs. Customers describe familiar solutions, researchers ask leading questions and committees use process as a reason not to act. Modern models can simulate many contexts, combine weak signals and produce testable concepts cheaply. Why not generate broadly, launch landing pages quickly and let revealed behaviour decide?
That rival is right about two things. Simulation is valuable for enlarging the hypothesis set, and market behaviour is often more informative than stated preference. A good operating system should therefore shorten the path from anomaly to bounded probe. It should not demand months of ceremonial research before every reversible test. Lead-user evidence is also intentionally unrepresentative: its value lies in revealing mechanisms and future conditions, not in estimating the average customer.
The rival fails when cheap exposure is treated as costless or self-interpreting. Clicks respond to framing, placement and curiosity. Deceptive scarcity or fictional availability can create behaviour while damaging trust. A model-generated population may omit those excluded by access, language or risk. Without a rival claim, delivery record and subsequent value measure, a conversion metric cannot say which problem was solved. Speed is compatible with discipline only when the probe is honest, bounded and designed to change a decision.
Governance is proportional to the use. Data protection obligations apply when discovery processes personal data; data minimisation, purpose, lawful basis, access control and retention remain necessary. AI risk controls should record intended use, evidence limits, evaluation, human responsibility and incident response. Accessibility should be tested in research materials, prototypes and probes, not postponed until production. Consumer-facing tests should avoid dark patterns, hidden conditions, fabricated endorsement and obstructive withdrawal. These controls improve the evidence by reducing the chance that observed behaviour is manufactured by pressure or exclusion.
AI changes the economics of search. It allows a team to inspect more domains, preserve more contradictions and generate more mechanisms than a linear workshop could. The strategic advantage, however, is not the number of concepts produced. It is the rate at which the organisation turns market-edge evidence into discriminating tests, abandons unsupported claims and compounds what it learns about real demand. Discovery becomes reliable when every promising narrative can answer a simple question: what happened outside the model that would not have happened if this opportunity were false?
References
- von Hippel, E. (1986), “Lead Users: A Source of Novel Product Concepts”. Management Science, 32(7), 791–805.
- Griffin, A. and Hauser, J. R. (1993), “The Voice of the Customer”. Marketing Science, 12(1), 1–27.
- Franke, N., von Hippel, E. and Schreier, M. (2006), “Finding Commercially Attractive User Innovations”. Journal of Product Innovation Management, 23(4), 301–315.
- Poetz, M. K. and Schreier, M. (2012), “The Value of Crowdsourcing”. Journal of Product Innovation Management, 29(2), 245–256.
- Dahlander, L. and Piezunka, H. (2014), “Open to Suggestions: How Organizations Elicit Suggestions Through Proactive and Reactive Attention”. Research Policy, 43(5), 812–827.
- Shane, S. (2000), “Prior Knowledge and the Discovery of Entrepreneurial Opportunities”. Organization Science, 11(4), 448–469.
- Camuffo, A. et al. (2020), “A Scientific Approach to Entrepreneurial Decision Making”. Management Science, 66(2), 564–586.
- Camuffo, A. et al. (2024), “The Role of the Scientific Approach in Entrepreneurial Decision Making”. Strategic Management Journal.
- von Hippel, E. (2005), Democratizing Innovation. MIT Press.
- W3C (2023), Web Content Accessibility Guidelines (WCAG) 2.2.
- US Federal Trade Commission (2022), Bringing Dark Patterns to Light.
- European Union (2016), General Data Protection Regulation.
- European Union (2024), Artificial Intelligence Act.
- NIST (2023), Artificial Intelligence Risk Management Framework.
- NIST (2024), Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile.
- NIST (2020), Privacy Framework.
- OECD, OECD AI Principles.
- UK Competition and Markets Authority (2022), Online Choice Architecture.