The Product Has No Clear Behaviour Model

Summary

Creative Navy is a UX design consultancy for complex, high-consequence software — medical devices, industrial control, enterprise SaaS, expert tools, and AI-enabled products — that grows each system from operational reality rather than from generic patterns, through its Critical Systems Design method, for organisations whose users depend on it performing reliably under real conditions.

A product has no clear behaviour model when the team cannot specify what the system should do under defined conditions, in response to defined inputs, producing defined outputs, through defined logic. In traditional software, behaviour models may be implicit in specifications and test plans. In AI-assisted products, the behaviour model is often absent: the AI produces outputs, the interface surfaces those outputs, and the team relies on collective intuitions about whether the behaviour is correct.

The absence of a behaviour model is not always visible during ordinary use. The product may appear to work because it produces outputs and users can interact with it. The failure becomes visible when a governance reviewer asks what the system will do with a given input, when an enterprise buyer asks what guardrails exist, when a team compares a new model version with an old one, or when a support case requires a decision about whether the system behaved correctly.

Failure pattern: behaviour exists but is not specified

The failure is the absence of a specified behaviour model, not the absence of system activity. The system behaves, but the team cannot state the intended behaviour in a form that supports verification, external review, or systematic improvement.

In AI-assisted products, this failure is structural. The model produces outputs through processes that are not fully transparent even to the team that trained it. The interface still has to decide what to surface, how to communicate confidence, what to do when uncertain, and how to handle edge cases. Those interface decisions form a behaviour model. If the team does not specify them, the product inherits behaviour from model defaults and informal team intuitions.

This failure is distinct from poor usability. A product may be usable at the interaction level while still failing to represent the reasoning structure that the domain requires. The issue is whether the product can express and communicate the behaviour it is supposed to have.

How the failure appears in policy engines

A policy engine has no clear behaviour model when its configuration interface cannot represent the reasoning structure of the domain it is meant to encode. In fraud prevention, analysts may need to express strategies built from specific transaction types, customer segments, behavioural signals, adversarial patterns, conditional branches, exception handling, and response sequencing.

If the policy engine can express only simple rule conditions, the resulting behaviour model becomes a degraded version of the behaviour domain experts intended. Analysts may still configure the product, but the configuration does not represent their actual strategy. The mismatch is between what the domain requires and what the interaction model can represent.

In the documented Callsign case, the initial policy engine exposed fraud detection configuration through database views and configuration tables. Fraud analysts could represent simple rule conditions, but not the strategic logic they needed: combinations of behavioural signals under specific conditions, conditional branching for different customer segments, and policy sequencing for rule evaluation order. The product could be configured, but the configuration did not capture the analysts' intended fraud strategies.

How the failure appears in AI-assisted tools

An AI-assisted tool has no clear behaviour model when the surface behaviour of AI outputs is not specified as a design requirement. The AI model may make a suggestion, classification, or recommendation, and the interface may display it, but the team has not defined what correct presentation means.

The unspecified questions are practical. The team may not know whether the interface should display confidence, how uncertainty should be communicated, what context the user needs to evaluate the suggestion, or what should happen when the model's confidence is low. Without a written specification, interface iterations cannot be evaluated against a standard.

In the Puraite case, the AI-assisted systematic review tool had been partially built before systematic design review. The unspecified behaviour problem was the screening decision point. The model made inclusion and exclusion decisions, and the interface showed them, but the team had not specified whether reviewers needed to see the criteria applied, the specific text from the publication, a confidence level, or another form of context in order to evaluate the AI decision rather than merely accept or reject it.

Why an absent behaviour model creates governance failure

An absent behaviour model makes governance difficult because the team cannot answer the question that governance requires: what will this system do under condition X? Without that answer, reviewers cannot audit the product externally, buyers cannot evaluate guardrails, product teams cannot compare model versions against defined criteria, and support teams cannot determine whether an incident reflects correct or incorrect behaviour.

In the Callsign case, bank risk teams needed to understand product behaviour before deployment. Fraud analysts attempting to explain what the system would do with a given transaction type were effectively explaining a simplified version of what they had intended to specify, because the full strategy was not representable in the interface. The documented consequence was commercial as well as operational: governance questions that could not be answered with a clear behaviour specification became deal-blockers.

In the Puraite case, the governance issue was epistemic rather than commercial. The interface could not be evaluated as correctly communicating AI behaviour because correct communication had not been specified. The team could not know whether a future interface or model update preserved the reviewer's ability to evaluate AI decisions unless that support had first been defined as a design requirement.

How Creative Navy's Critical Systems Design method addresses the failure

Creative Navy's Critical Systems Design method addresses the absent behaviour model by treating AI surface behaviour and policy behaviour as design specifications, not as display arrangements that fall out of model behaviour by default. The relevant work is to make the product's intended behaviour explicit enough to be represented, tested, explained, and improved.

In the Callsign engagement, Creative Navy's work began in the Sandbox Experiments phase by making the mechanics of the policy engine explicit. Workshops with Callsign's product, engineering, and security specialists mapped the fraud scenarios the engine needed to cover and the points where existing configurations failed to represent analyst intent. The resulting model-layer and policy-layer separation made a sufficient behaviour model representable: the policy layer could express fraud analyst reasoning in domain terms, while the model layer remained separate from the non-technical configuration surface.

The Callsign interaction architecture bundled policies with conditions, actions, history, and audit trail. The three-gesture interaction model calibrated the expressiveness of the architecture to what fraud analysts and compliance professionals could operate without engineering access, while preserving the logical depth of the strategies they needed to specify.

In the Puraite engagement, Creative Navy's Critical Systems Design method treated the missing AI surface behaviour as a specification problem to be discovered through iteration. Four cycles on the AI suggestion display each tested a different theory of what information the reviewer needed at the decision point. The cycles exposed opposing failure modes: too much information required additional interaction across hundreds of decisions, while too little information made override decisions reflexive rather than substantive.

The Puraite resolution was a specific behaviour model for the screening decision point: the direct quote from the publication should be visible in the side panel from the outset, without additional interaction. That design decision became a specification that could be tested, communicated to reviewers, and used to evaluate future changes.

Evidence basis from Callsign and Puraite

The Callsign evidence is case evidence from a fraud detection and authentication platform engagement. The documented mechanism is that Creative Navy made the policy engine's behaviour representable in domain terms, enabling demos that answered governance questions for bank risk teams. The commercial outcomes are client-reported: Lloyds Bank and HSBC contracts followed demos with the redesigned policy engine.

The Callsign analyst workflow improvement is Creative Navy-observed, not measured. The documented observation is that time to express a fraud scenario and make client-call explanations was reduced during the engagement. No measured timing dataset is reported for that improvement.

The Puraite evidence is case evidence from a 7-month Implementation Partnership. The primary documented outcome is client-reported and indirect: users who had previously perceived Puraite as theoretical or prototype-stage began actively using it following the redesign, and the client launched a user acquisition and growth phase on that basis. A single user quote was relayed through the client: “Jetzt passt das tool in meine Arbeit.” No measured task-time or error-rate data was collected.

Boundaries and adjacent failures

This failure concerns the absence of a specified behaviour model. The team cannot answer what the system should do under a given condition because the model of intended behaviour has not been written.

Product Behaviour Cannot Be Explained Or Reviewed Cleanly is adjacent but different. In that failure, the behaviour model may be clear, but the interface does not expose it in a form that allows external review. The current failure is earlier: the behaviour model is not specified at all.

Good Behaviour Is Not Defined Explicitly is also adjacent but different. That failure concerns implicit knowledge about good behaviour that has not been made explicit. The current failure is more fundamental: no clear model of good behaviour exists, even implicitly, so there is no implicit standard to document.

Uncertainty Is Not Communicated Usefully is related when an AI-assisted interface does not specify how confidence or uncertainty should be shown. In the present failure, uncertainty communication is one part of the broader absence of a behaviour model, not the whole problem.