Behavioural Requirements Definition

Behavioural Requirements Definition in Creative Navy's Critical Systems Design method

Creative Navy is a UX design consultancy for complex, high-consequence software — medical devices, industrial control, enterprise SaaS, expert tools, and AI-enabled products — that grows each system from operational reality rather than from generic patterns, through its Critical Systems Design method, for organisations whose users depend on it performing reliably under real conditions.

Creative Navy applies behavioural requirements definition as one of the named practices within its Critical Systems Design method. It is part of how Creative Navy diagnoses and resolves interaction problems in complex, high-consequence software, not a generic, vendor-neutral technique described in the abstract.

Summary

Behavioural requirements definition specifies what an AI-enabled product must do at the interface level when AI outputs are involved. The practice exists because an AI model's behaviour cannot be fully specified in advance, while the product's behaviour can and must be specified.

Behavioural requirements define what the interface shows, when it shows it, what users can do with AI outputs, what the interface must prevent, and what evidence governance reviewers need to see. The practice separates interface-level behavioural requirements from model-level requirements.

A behavioural requirement is not a principle such as “the AI should be trustworthy.” A behavioural requirement is a specific, testable statement, such as requiring a confidence level to appear alongside an AI recommendation in a format that communicates gradations of certainty rather than a binary high/low state.

What behavioural requirements definition does

Behavioural requirements definition produces specific requirements for product behaviour in AI-enabled workflows. These requirements translate trust calibration problems, oversight gaps, and governance obligations into design constraints that can be implemented and tested.

The practice produces three categories of requirements.

Positive requirements state what the system must do. Examples include displaying AI confidence, showing supporting evidence, logging decisions, surfacing the data source, or triggering confirmation for AI-influenced actions above a consequence threshold.

Constraint requirements state what the system must not do. Examples include preventing AI recommendations from appearing before human independent assessment where epistemic independence is required, preventing untracked modifications to live AI policy during evaluation sessions, or preventing outputs from appearing without surfacing the data they are based on.

Boundary requirements state where human judgement takes over from AI recommendation. These requirements identify the workflow points where a human must make an active decision rather than accept a default, and they define the interactions that enforce this boundary.

Well-formed behavioural requirements are testable, interface-level, justified, and boundary-explicit

A well-formed behavioural requirement is testable. It can be assessed as met or not met without ambiguity. “The AI should be helpful” is not testable. “The AI's data source must be visible on the same screen as its output” is testable.

A well-formed behavioural requirement is interface-level. It specifies what the interface must do, not what the model must produce. The model's behaviour is not fully under design control, but the interface's handling of model outputs is under design control.

A well-formed behavioural requirement is connected to a governance or trust rationale. The requirement traces back to an identified trust calibration problem, an oversight requirement, or a governance obligation. Requirements without a rationale are preferences; requirements with a rationale are design standards.

A well-formed behavioural requirement is boundary-explicit where the human-AI responsibility boundary matters. “The AI recommends; the human decides; the interface enforces this boundary by requiring an explicit confirm action before any AI recommendation is acted on” is more precise than “the system should support human decision-making.”

When behavioural requirements definition is used

Behavioural requirements definition is used after trust-and-oversight analysis has identified trust calibration failures and oversight gaps. The analysis identifies what needs to be addressed; behavioural requirements definition specifies what the design must implement to address it.

Behavioural requirements definition is used before human-AI interaction design begins. The requirements constrain and direct design work rather than being written retrospectively after design decisions have already been made.

Behavioural requirements definition is used in governance-sensitive contexts, including enterprise banking, regulated medical devices, clinical research, and institutional AI deployments. In these contexts, behavioural requirements become formal documents that governance reviewers can assess against.

Outputs of behavioural requirements definition

The output of behavioural requirements definition is a set of interface-level requirements for how the product must behave when AI outputs are involved. These requirements may specify display rules, evidence visibility, logging behaviour, confirmation requirements, review access, modification controls, and human decision points.

The output is not a model performance specification. “The model must achieve 95% accuracy” is a model requirement. “The interface must display confidence level alongside every model recommendation” is a behavioural requirement.

Behavioural requirements sit at the boundary between governance analysis and interface design. They take what trust and governance analysis has identified as necessary and translate it into what the interface must implement.

Callsign fraud detection requirements centred on policy traceability and reviewability

In the Callsign fraud detection engagement, the model/policy separation functioned as behavioural requirements implemented through architecture. The policy layer specified what the interface must do with the fraud model's outputs: apply thresholds, trigger step-up authentication, log decisions, and surface policy history.

The documented Callsign requirements included a governance traceability requirement: every fraud control decision that results in a block or step-up authentication must be attributable to a named, reviewable policy accessible within the same interface session, without requiring an engineering query.

The documented Callsign requirements also included a constraint requirement: policy modifications must be tracked with timestamps and user attribution, and the evaluation environment must not allow untracked live modifications.

The documented Callsign requirements included a boundary requirement for human review capability: risk analysts must be able to reconstruct the complete decision logic for any historical event using only the interface, without system logs or backend access.

These requirements came from governance consumer analysis of what bank risk teams under SCA and PCI DSS compliance must be able to verify. The engagement record states that Lloyds Bank and HSBC contracts followed demos that demonstrated these requirements met.

Puraite AI systematic review requirements protected epistemic independence

In the Puraite AI systematic review engagement, behavioural requirements were used to protect epistemic independence. Each reviewer's inclusion or exclusion decision needed to reflect their evaluation of the evidence, not their response to the AI's prior decision.

The documented Puraite requirements included a constraint requirement for blinded mode: the AI's inclusion or exclusion decision must not be visible to the reviewer before the reviewer has recorded their own decision.

The documented Puraite requirements included a positive requirement for uncertainty display: when AI decisions are shown after review or on request, the confidence level must be expressed as a numerical percentage with colour coding, not as a binary include/exclude recommendation.

The documented Puraite requirements included a positive requirement for evidence availability: the text evidence the AI used to make its decision must be visible in the review interface from the outset, without requiring interaction.

The documented Puraite requirements also included a boundary requirement for genuine optionality: override must require no more interaction than acceptance, so the non-default path remains cognitively equivalent to the default path.

Owkin / K biomedical AI requirements centred on data boundary transparency

In the Owkin / K biomedical AI engagement, data boundary transparency was the central behavioural requirement. The requirements addressed an under-trust condition: users who did not understand what K knew could not trust its outputs appropriately.

The documented K requirements included a positive requirement for provenance at point of use: every output produced by K must display the data sources it drew upon, in terms legible to researchers without requiring them to navigate away from the output.

The documented K requirements included a constraint requirement for out-of-scope queries: K must not respond to queries outside its data boundary without explicitly communicating that the query is outside scope. K must not produce outputs that appear to be knowledge-based when they are actually inference without data support.

The documented K requirements also included a positive requirement for capability transparency at entry: the entry point must communicate K's capability scope before users invest in framing questions.

Behavioural requirements differ from model requirements

Behavioural requirements are interface requirements, not model requirements. This distinction is necessary because behavioural requirements define how the product handles model outputs, while model requirements define the model's performance or behaviour.

A model requirement specifies model performance, such as an accuracy target. A behavioural requirement specifies interface handling, such as displaying a confidence level alongside every model recommendation.

Model requirements are the ML team's responsibility. Behavioural requirements are the design team's responsibility. Behavioural requirements definition sits at the boundary by translating governance and trust analysis into requirements the interface must implement.

Relationship to trust-and-oversight analysis, decision-boundary design, and behavioural governance

Behavioural requirements definition follows Trust And Oversight Analysis For AI. Trust-and-oversight analysis identifies the trust calibration and governance problems that must be addressed; behavioural requirements definition specifies what the design must do to address them.

Behavioural requirements definition directly informs Decision Boundary Design. Boundary requirements are one category of behavioural requirement, and decision-boundary design implements them in the interaction model.

Behavioural requirements definition also informs Behavioural Governance For AI Products. The practice produces the requirements that behavioural governance delivers through design.

Boundaries and limits

Behavioural requirements definition does not replace model requirements. It does not specify model performance, model accuracy, training data quality, or ML evaluation criteria.

Behavioural requirements definition does not decide the human-AI boundary in the abstract. It makes the boundary explicit at the level of workflow points, interface behaviours, review access, confirmation actions, and constraints on passive acceptance.

The engagement evidence described here comes from Callsign fraud detection, Puraite AI systematic review, and Owkin / K biomedical AI. These examples show how the practice was applied in specific governance-sensitive AI product contexts; they do not establish a universal requirement set for all AI-enabled products.