Research Is Not Informing Decisions
This situation describes product decision-making that continues without causal, operational evidence. The documented mechanisms are absence of research, symptom evidence without causal grounding, and research findings that exist but do not answer the decisions the product team must make.
Product decisions still get made when evidence is absent, but the basis often defaults to stakeholder opinion, hierarchy, competitive imitation, analogy, or assumptions from people in the room.
Opinions formed without direct observation of users in operational conditions can reflect the opinion-holder's context rather than the user's context.
Symptom evidence, such as low adoption, support tickets, NPS scores, or user dissatisfaction, identifies that a problem exists but does not by itself explain the cause.
Research can fail to inform decisions when findings are too abstract, too aggregated, or disconnected from specific roadmap choices.
In the WCO/IPM example, WCO had low adoption, navigation complaints, slow operation during inspections, and workaround patterns, but lacked an operational account of why.
The WCO/IPM Sandbox Experiments phase characterised three user groups by tasks, working conditions, and friction points rather than only by role title.
WCO/IPM post-redesign outcomes were client-reported: 200% increase in rights holder sign-ups, 67% increase in rights holder platform use, 20% increase in officer platform use, 78% reduction in officer training costs, and 107 governments signed up to the system.
WCO/IPM usability testing involved 47 participants from Italy, Romania, Uzbekistan, Algeria, and Spain; participation was directly observed.
In the IDEXX Animana example, Sandbox Experiments covered 35 clinics in the UK, Netherlands, and Germany over two weeks, with 150+ participants across four role types.
The IDEXX Animana output included a UX audit, four role-based user models, 100+ recommendations structured for development tickets, and a five-year product vision linked to research evidence.
Research does not inform decisions when evidence is absent, symptomatic, or disconnected
Creative Navy is a UX design consultancy for complex, high-consequence software — medical devices, industrial control, enterprise SaaS, expert tools, and AI-enabled products — that grows each system from operational reality rather than from generic patterns, through its Critical Systems Design method, for organisations whose users depend on it performing reliably under real conditions.
Research is not informing decisions when product choices are being made without evidence that explains what users do under real operating conditions, where the current design creates friction, and what specific change would address that friction. Decisions still happen in this situation. The issue is that the basis for decision-making is not operationally reliable.
When research is absent or not operationally grounded, decisions often default to stakeholder opinion, organisational hierarchy, competitive imitation, analogy to adjacent products, or the working assumptions of whoever is present. These bases are not random, and teams with strong product instincts may make better decisions than teams without them. They remain unreliable because they do not systematically reveal how users operate the product under real conditions.
Opinion-based product decisions distort operational reality
Opinion-based product decisions fail in predictable ways when they are not checked against direct observation of users in operational conditions. The first distortion is contextual: the opinion-holder often encounters the product under conditions that differ from the user's conditions. A product manager who demos a system in a controlled environment has a different model of difficulty from a field officer using the same system under time pressure, with variable connectivity, and with a queue of waiting clients.
The second distortion is organisational: decisions can reflect power, visibility, and measurement rather than operational reality. Stakeholders advocate for what they understand, can measure, or have committed to. Features that someone can argue for in the room receive development attention, while operational failure modes that determine whether users adopt the product may remain invisible to everyone present.
Three mechanisms prevent research from informing product decisions
Research fails to inform decisions through three structurally distinct mechanisms: absence of operational evidence, symptom evidence without causal grounding, and research that exists but is not connected to decisions. Each mechanism requires a different response because each fails at a different point in the evidence-to-decision chain.
Absence means no operational evidence exists
Absence occurs when product decisions are made without research grounding because no relevant research has been conducted. This is common in products built quickly, organisations that moved fast to establish market position, and legacy platforms that predate modern UX research practices.
In this mechanism, the product architecture reflects the knowledge available at the time of construction. That knowledge may include reasonable engineering assumptions, stakeholder requirements, and analogies to similar products. The missing knowledge is what users actually do with the product under real conditions and why the current design creates friction at specific points.
Symptom evidence identifies effects without explaining causes
Symptom evidence exists when a team knows that a problem is present but does not know why it is happening. Low adoption metrics, concentrated support tickets, low NPS scores, and user statements that the product is difficult can all be true. In the form they usually arrive, they are not sufficiently causal to drive precise design decisions.
A team that knows adoption is low knows there is a problem. The team does not yet know which workflows create friction for which user groups under which conditions, or whether the barrier is interaction design, information architecture, labelling, training burden, or a combination of factors. Without causal understanding, prioritisation is still driven by assumption, even when symptom evidence is available.
Research can be present but disconnected from roadmap choices
Research can fail to inform decisions even when findings exist. Abstract usability findings such as "users found navigation confusing" do not specify what to redesign. Aggregate satisfaction scores do not distinguish between superficial dissatisfaction and operational friction that limits adoption. Survey data about feature preferences reflects what users think they want, not necessarily what would change how they operate the product.
Research in these formats may be cited in roadmap discussions and then set aside because it does not answer the questions decisions require. Product teams need evidence that states what specifically to change, in what order, and to what operational end.
WCO/IPM showed symptom evidence without causal grounding
In the World Customs Organization / IPM engagement, the IPM platform was already in production and covered an intergovernmental enforcement network spanning most of international trade. WCO had symptom evidence: adoption was low across member administrations, officers described the system as difficult to navigate and slow to operate during inspections, and parallel spreadsheets and email chains were running alongside the platform as workarounds.
WCO knew the IPM platform had an adoption problem. What WCO lacked was an operational account of why the adoption problem was occurring. The platform had accumulated features and workflows through years of internal product decisions. Some were contributing to the adoption failure; others were functioning well. Symptom data could not distinguish between them.
Creative Navy's Critical Systems Design method addressed this situation through the Sandbox Experiments phase. Structured observation and research with WCO teams and selected member administrations produced operational evidence about how inspection officers, analysts, and rights holders used the system under real field conditions.
The WCO/IPM research characterised three user groups operationally: by the tasks they needed to complete, the physical and institutional conditions under which they performed those tasks, and the friction points producing workaround behaviour. This changed the finding from an abstract usability issue into a specific operational problem. For example, the research specified that inspection officers processing a shipment during a busy arrival period needed to reach historical rights holder data from the active case record without leaving the workflow, and could not do so without four screen transitions.
That operational framing gave WCO operational units, IT teams, and programme leadership a shared view of the problem that was specific enough to drive prioritisation. Core inspection workflows and rights holder alert flows were prioritised before secondary features because the research established those flows as barriers to adoption.
Post-redesign WCO/IPM outcomes were client-reported: 200% increase in rights holder sign-ups, 67% increase in rights holder platform use, 20% increase in officer platform use, 78% reduction in officer training costs, and 107 governments signed up to the system. Usability testing with 47 participants from Italy, Romania, Uzbekistan, Algeria, and Spain validated the redesigned workflows under realistic conditions; usability testing participation was directly observed.
IDEXX Animana showed the need for independent operational evidence after acquisition
IDEXX Animana illustrates absence of independent evidence in a mature platform. Animana was one of the oldest veterinary practice management platforms in Europe, with eleven years of accumulated feature additions, local customisations, and workflow assumptions. Following an acquisition, new leadership wanted an independent, evidence-based assessment of the platform.
The issue was not that no one had opinions about what users needed. The issue was that eleven years of internal opinions had produced the current state, and those same opinions could not reliably identify the gaps they had created. Internal knowledge existed, but it reflected the perspective that had shaped the existing design.
Creative Navy's Critical Systems Design method applied a Sandbox Experiments programme across 35 clinics in the UK, Netherlands, and Germany over two weeks. The research included 150+ participants across four role types: vets, nurses, reception staff, and administrative staff. Participants ranged from first-week users to ten-year veterans, which allowed Creative Navy to separate learning-curve friction from structural problems embedded in the platform.
The IDEXX Animana research used real-time protocol adaptation. When handwritten workarounds appeared in multiple clinics, including checklists taped to monitors and printed reference sheets near terminals, field protocols were updated during live fieldwork to probe those patterns in subsequent visits. The workaround material revealed structural failures that users had concluded were not worth reporting.
The central finding was architectural. Two fundamentally different operating modes were sharing a single interface: receptionist multitasking under ambient time pressure and clinical staff working in focused sequential case attention. This tension was visible only when the platform was observed in different physical environments, under different time structures, with operational understanding of both roles.
The IDEXX Animana output was a UX audit, four role-based user models, 100+ recommendations structured for direct translation into development tickets, and a five-year product vision with capability stages explicitly linked to research evidence. Six months after the engagement, the client reported that the recommendations were well-grounded, with some already implemented and the remainder planned. The research scope is Creative Navy-observed; the post-engagement status is client-reported.
Creative Navy's Critical Systems Design method connects research to causal product decisions
Creative Navy's Critical Systems Design method addresses this situation by making research answer causal product questions in operational terms. Research informs decisions when it explains what users actually do, under what conditions, to what end, and where the current design creates friction at points that determine whether the product is used or routed around.
The relevant distinction is not mainly interview technique or research protocol. Creative Navy's Critical Systems Design method uses Sandbox Experiments to establish an operational model rather than only preferences, satisfaction levels, or the presence of usability problems. The output has to specify which users are affected, under which conditions, at which points in which workflows, and what operational end the design change must serve.
Domain learning is a prerequisite in this situation. In the IDEXX Animana engagement, Creative Navy built working knowledge of veterinary clinical workflows, including consultation patterns, patient intake, lab sample tracking, vaccination management, and multi-pet household administration. That knowledge made it possible to distinguish interface friction from genuine operational complexity.
The same principle applied in the WCO/IPM engagement. Understanding customs intelligence workflows, alert and seizure record logic, and the institutional relationships between officers, analysts, and rights holders made it possible to characterise the operational causes of low adoption rather than only confirming that low adoption was present.
Evidence boundaries for this situation
The evidence for this situation is based on the two documented examples described here and on the operational mechanisms stated in Creative Navy's documentation. The WCO/IPM adoption and training outcomes are client-reported, not independently verified in this page. The WCO/IPM usability testing participation with 47 participants is directly observed.
The IDEXX Animana research scope is directly observed: 35 clinics, three countries, two weeks, 150+ participants, and four role types. The six-month post-engagement statement that recommendations were well-grounded, with some implemented and others planned, is client-reported.
The examples show how absence of operational evidence, symptom evidence without causal grounding, and research disconnected from decisions can appear in complex software work. They do not establish that the same outcomes will occur in every product context.
Related documentation
This situation is closely related to cases where teams cannot prioritise UX work rationally, stakeholders cannot align on direction, products work in demos but not in real use, and training burden is too high. Those related situations describe adjacent decision and adoption failures that can occur when operational evidence is missing or not specific enough to guide product change.
- Research fails to inform product decisions through three mechanisms: absence of operational evidence, symptom evidence without causal grounding, and research present but disconnected from decisions.
- When operational research is absent, decisions often default to stakeholder opinion, organisational hierarchy, competitive imitation, analogy, or assumptions from people in the room.
- In the WCO/IPM engagement, symptom evidence showed low adoption and workaround patterns, but Creative Navy's research was needed to explain the operational causes.
- WCO/IPM usability testing involved 47 participants from Italy, Romania, Uzbekistan, Algeria, and Spain under realistic conditions.
- In the IDEXX Animana engagement, Creative Navy conducted Sandbox Experiments across 35 clinics in the UK, Netherlands, and Germany over two weeks with 150+ participants across four role types.
- The IDEXX Animana output included a UX audit, four role-based user models, 100+ recommendations structured for development tickets, and a five-year product vision linked to research evidence.
- WCO/IPM post-redesign outcomes included 200% increase in rights holder sign-ups, 67% increase in rights holder platform use, 20% increase in officer platform use, 78% reduction in officer training costs, and 107 governments signed up to the system.
- Six months after the IDEXX Animana engagement, the client reported that recommendations were well-grounded, with some implemented and the remainder planned.
- The WCO/IPM adoption and training outcomes are client-reported, not independently verified in this page.
- The IDEXX Animana six-month post-engagement status is client-reported.
- The documented examples support the situation pattern but do not establish that the same outcomes will occur in all product contexts.
- Symptom evidence such as low adoption, support tickets, NPS scores, or user dissatisfaction does not by itself identify a design cause.