Teams Cannot Define Good AI Behaviour

Summary: AI behaviour at the interface is not the same as model performance

Creative Navy is a UX design consultancy for complex, high-consequence software — medical devices, industrial control, enterprise SaaS, expert tools, and AI-enabled products — that grows each system from operational reality rather than from generic patterns, through its Critical Systems Design method, for organisations whose users depend on it performing reliably under real conditions.

Teams cannot define good AI behaviour when they know what the model can do but have not specified what good behaviour looks like at the point of use. The team may have training data, evaluation metrics, benchmark performance figures, accuracy data, and known degradation points. Those facts do not define how the AI should present outputs to the people who must interpret and act on them.

Good AI behaviour in the user context depends on what the model produces and on how the output is shaped, framed, sequenced, and presented. A model can perform well on a benchmark task while producing interface behaviour that misleads users, communicates confidence in an operationally unsuitable way, prevents meaningful evaluation, or appears at a point in the workflow where action on the output is counterproductive.

Undefined AI behaviour makes the interface inherit model structure

Undefined AI behaviour commonly appears when the interface presents what the model produces in the form the model produces it. Confidence scores appear because the model produces them. Criteria are listed because the model applies them. Output formats reflect model data structures. The AI behaviour at the interface becomes implicit in implementation rather than explicit in a design specification.

This pattern is different from a poorly performing model. The model may be capable, accurate, and well evaluated, while the product still behaves poorly for users because the interface has not translated model output into domain logic. Domain experts such as analysts, clinicians, and researchers need to evaluate AI output against their domain expertise, not against the model's internal vocabulary or feature structure.

Consequence: the interface exposes model structure rather than domain logic

When no behaviour specification exists, interface decisions default to the model's internal organisation. The interface may show features, confidence distributions, classification categories, or matched criteria in the model's own vocabulary.

This makes professional evaluation harder. Domain experts do not normally reason in the same terms as the model. They need to understand whether the AI output is meaningful in their domain, whether the evidence supports the recommendation, and whether acting on the recommendation fits the operational context.

Consequence: behaviour decisions accumulate as implementation choices

When AI behaviour has not been formally specified, individual behaviour decisions are made incrementally during implementation. These decisions include what to surface, what to suppress, which confidence levels map to which display treatments, where human override applies, and how AI outputs should be sequenced relative to human judgment.

The issue is not that these decisions are necessarily made badly. The issue is that they are made without a shared behaviour model. Locally sensible implementation choices may not add up to coherent AI behaviour. The product cannot then be evaluated against a standard because no standard was set.

Consequence: the product cannot improve deliberately

A product whose AI behaviour was not specified cannot be improved against a target. Teams may make local changes and observe what happens, but they lack a counterfactual definition of the behaviour the product should approach.

This produces iteration without convergence. Changes that make the product worse can be difficult to identify because there is no specified answer to the core question: for the users who will act on this AI's outputs in this operational context, what should the AI do?

Puraite example: behaviour specification through iterative design

Puraite is an AI-assisted systematic review platform for conducting rigorous literature reviews used in academic, clinical, and pharmaceutical research. The AI integrates into multiple stages of the review process: automating initial screening decisions, suggesting inclusion and exclusion criteria, and extracting structured data from qualifying publications. At each stage, a reviewer or project manager must accept, override, or modify AI outputs.

When Creative Navy joined the Puraite engagement, the AI behaviour model had not been fully specified as a design problem. Three core behaviour questions were unresolved: what the AI should surface at each stage of the review process, how confidence should be communicated to reviewers, and where human override should sit in relation to AI recommendations.

The starting interface presented AI decisions without giving reviewers a basis for evaluating them. The screening view showed inclusion and exclusion decisions but did not show the criteria the AI had applied or the text from the publication the AI had used to reach its conclusion. Reviewers could override, but without supporting information the override was a judgment about the decision rather than a judgment about the evidence.

Puraite suggestion display: four design cycles defined what the AI should surface

The Puraite AI suggestion display required four design cycles to resolve. Reviewers needed to see the criteria applied by the AI and the specific text from the publication that the AI used to reach its decision. That information had to be present without additional interaction steps, because extra steps compound across hundreds of decisions in a single review session.

Each iteration tested a different way to balance evidence completeness and screening rhythm. Designs that provided full criteria evidence required expansion or navigation, which broke the screening rhythm. Designs that compressed the display for scan speed removed the evidence needed for substantive override decisions.

The resolved design placed the direct quote from the publication in the side panel from the outset of the decision interaction. The reviewer could see the AI decision, the criteria applied, and the text evidence simultaneously. This design decision specified what the AI should surface at the decision point and what epistemic position the reviewer should have relative to the AI recommendation.

Puraite confidence display: confidence became a first-class interface element

The Puraite data extraction flow required a behaviour decision about whether and how AI confidence should be surfaced to project managers and reviewers scanning extracted data. The model produced confidence estimates, but the design question was how those estimates should function in the interface.

The design communicated AI confidence as an explicit percentage with colour-coded scanning support. Lower-confidence extractions became visible at a glance, allowing reviewers to allocate verification effort to the extractions most likely to require it.

This was a behaviour specification decision, not only a visual treatment. The interface made confidence a first-class element of the AI output presentation rather than a background property available only through investigation.

Puraite blinded mode: human override required protection from anchoring

Puraite raised a further behaviour question: whether AI recommendations should be visible to reviewers during initial screening, or whether initial screening should be conducted independently with AI recommendations revealed only afterwards.

Blinded screening mode addressed an epistemic concern. If AI recommendations are shown during screening, they may anchor the reviewer's independent judgment. In that condition, the human-in-the-loop can become confirmation of AI decisions rather than independent assessment.

Creative Navy identified blinded mode as a product requirement during the engagement. The detailed design was outside the engagement scope, but the mode was retained in the product. Its identification illustrates that rigorous AI behaviour definition can surface additional product requirements as the problem becomes better understood.

Puraite navigation restructuring: behaviour definition exposed wider workflow issues

The Puraite engagement began with a defined scope focused on AI screening screens. During the work, Creative Navy identified that the product navigation was also a significant barrier. The navigation contained 13 top-level items organised around the product's internal structure, which made it difficult for users to find workflows relevant to their stage in the systematic review process.

Creative Navy restructured the navigation from 13 top-level items to 4 by organising it around the systematic review process rather than the product architecture. This navigation change was not AI behaviour in itself, but it reflected the same principle as the AI suggestion display: design should follow how users move through work, not how the system is organised internally.

The outcome evidence is limited. The redesign produced a client-reported shift in user perception: users who had previously perceived Puraite as theoretical or prototype-stage began actively using it following the redesign, and the client launched into a user acquisition and growth phase. A single user quote was relayed by the client: "Jetzt passt das tool in meine Arbeit" — "Now the tool fits my work." No measured task-time data, error-rate data, or baseline research was recorded before the engagement.

Creative Navy's Critical Systems Design method treats behaviour definition as design work

Creative Navy's Critical Systems Design method addressed the Puraite behaviour specification gap through operational grounding and design-as-research. The work did not implement a predefined AI behaviour model. The behaviour model emerged through candidate designs, observed tradeoffs, and iteration.

Sandbox Experiments established the operational model under a constraint: direct user research access was not available within the timeline. A member of the Creative Navy team with firsthand experience conducting systematic reviews served as the domain learning proxy. Creative Navy presented this substitution and its tradeoffs explicitly to the client rather than using it silently.

Design-as-research was the appropriate mode because Puraite sat in a product category with minimal prior art. The engagement could not simply draw on established patterns for AI-assisted systematic review screening, confidence display, criteria presentation, or reviewer decision context. The four iterations on the suggestion display were the process through which the behaviour specification was produced.

Evidence basis and limits for the Puraite example

The Puraite example provides case evidence for how undefined AI behaviour can be specified through iterative design. Creative Navy-recorded engagement facts include the unresolved behaviour questions, the four design cycles on the AI suggestion display, the confidence display decision, the identification of blinded mode, and the navigation restructuring from 13 top-level items to 4.

The strongest evidence in the Puraite example concerns the design reasoning and the design artefact decisions documented during the engagement. Outcome evidence is weaker. The shift in user perception, active use after redesign, the growth phase, and the quoted user reaction were client-reported and indirect. They were not independently measured, and no baseline research was conducted before the engagement.