Constraint, State, Decision, Response. Ben Faircloth

Language-model applications become difficult to reason about when distinct responsibilities disappear inside a model call.

The same pattern shows up elsewhere in AI systems. In FN-001, Small Is Not Open, capability, access, control, and evidence become misleading when they are treated as one question. In BN-001, Separating Measurement From Interpretation, a working system becomes easier to reason about once numerical measurement is kept separate from model interpretation.

Application logic has the same problem.

A prompt can quietly absorb policy, memory, workflow logic, evidence requirements, tool selection, and response generation. The application may still work, but its behavior becomes difficult to inspect. When something goes wrong, the explanation collapses into a familiar question:

Why did the model do that?

A more useful question is:

Which part of the system was responsible for the behavior?

One way to answer that is to separate four concerns:

Constraint. State. Decision. Response.

The model may participate in each. It should not automatically own all four.

Start with the system, not the prompt

Consider a simple assistant that answers questions from a collection of policy documents.

A first implementation might look like this:

answer = llm(
    f"""
    Answer the user's question using the documents below.

    Documents:
    {context}

    Question:
    {question}
    """
)

This is enough to demonstrate retrieval and generation.

It also hides most of the application.

The code does not make explicit whether the retrieved material is sufficient, whether the request is within scope, whether the user is allowed to receive the information, whether the system should retrieve again, or whether it should decline to answer.

Those responsibilities still exist. They have simply been pushed into the prompt.

A more useful model is to treat an application at time \(t\) as a system with state \(S_t\), constraints \(C\), a decision policy \(\pi\), and a response process \(G\).

D_t = \pi(S_t, C)\]\[R_t = G(S_t, D_t)

The first equation asks what the system should do.

The second asks what the system should return.

Those are not necessarily the same problem, and they do not necessarily belong to the same model.

Constraint

Constraints define the boundaries of permitted behavior.

They may come from security policy, product requirements, evidence thresholds, authorization rules, regulatory requirements, or simple application logic.

Suppose the system is only allowed to answer when retrieval quality exceeds some threshold \(\tau\).

We can represent that requirement as:

C_{\text{evidence}}(S_t) = \begin{cases} 1 & \text{if } q_t \geq \tau \\ 0 & \text{otherwise} \end{cases}

where \(q_t\) is the retrieval-quality signal available in the current state.

In code, this does not need to be sophisticated:

def evidence_is_sufficient(
    score: float,
    threshold: float = 0.72,
) -> bool:
    return score >= threshold

The threshold here is illustrative. In a real system, the evidence signal and threshold would need to be defined and validated for the retrieval method and workload. A raw similarity score is not a universal measure of answerability.

The important distinction is not mathematical complexity. It is where the responsibility lives.

Compare that with putting the following sentence in a prompt:

Only answer when the available evidence is sufficient.

The sentence expresses an intention.

The function enforces a condition.

Both may be useful, but they are not equivalent.

This is the same architectural distinction that appears in FN-001. Control is not established merely because we instruct a model to behave a certain way. Control comes from the surrounding system retaining responsibility for consequential decisions.

State

State is the information available to the application at a particular moment.

For a simple grounded assistant, state might include the current question, retrieved documents, a retrieval score, user permissions, prior tool results, and any earlier decisions.

We can write:

S_t = \{x_t, E_t, H_t, P_t, O_t\}

where:

\(x_t\) is the current user input,
\(E_t\) is available evidence,
\(H_t\) is interaction history,
\(P_t\) represents permissions or policy state, and
\(O_t\) contains observations from tools or prior system actions.

A minimal implementation might look like this:

from dataclasses import dataclass, field


@dataclass
class AppState:
    question: str
    retrieved_docs: list[str] = field(default_factory=list)
    retrieval_score: float = 0.0
    user_role: str = "user"
    decision: str | None = None

This looks almost trivial.

That is useful.

State should be boring enough that we can inspect it.

As an application grows, the alternative is often implicit state scattered across prompt history, tool outputs, hidden application variables, and model-generated summaries. At that point, debugging becomes an exercise in reconstructing what the system believed when a decision was made.

Explicit state gives us something to examine.

Decision

A decision determines what should happen next.

This is one of the responsibilities most commonly delegated to a model simply because a model is available.

Imagine that our assistant has three possible actions:

ANSWER
RETRIEVE_MORE
ABSTAIN

The decision policy can be represented as:

D_t = \pi(S_t)

The policy does not need to be generative.

from enum import Enum


class Decision(str, Enum):
    ANSWER = "answer"
    RETRIEVE_MORE = "retrieve_more"
    ABSTAIN = "abstain"


def decide(state: AppState) -> Decision:
    if state.retrieval_score < 0.50:
        return Decision.ABSTAIN

    if state.retrieval_score < 0.72:
        return Decision.RETRIEVE_MORE

    return Decision.ANSWER

This does not mean every decision should be encoded as an if statement.

Some decisions are genuinely ambiguous. A model may be the right component for routing, classification, planning, or interpretation.

The point is narrower:

Using a model should be a design decision, not the default location for application logic.

A useful question is not:

Can the model make this decision?

Modern models can make many decisions surprisingly well.

The better question is:

What properties must this decision have?

Does it need to be repeatable? Auditable? Permission-aware? Bounded by policy? Explainable after the fact? Tolerant of variation?

Those requirements determine where the decision belongs.

Response

The response is what the system exposes to the user.

This is where generative models are often strongest.

Once the application has established that answering is permitted and enough evidence exists, a model can synthesize the material into a useful response.

def respond(state: AppState) -> str:
    if state.decision == Decision.ABSTAIN:
        return "I don't have enough evidence to answer that reliably."

    if state.decision == Decision.RETRIEVE_MORE:
        return "I need additional evidence before answering."

    return generate_answer(
        question=state.question,
        documents=state.retrieved_docs,
    )

The complete flow is now visible:

def run(question: str) -> str:
    state = AppState(question=question)

    docs, score = retrieve(question)

    state.retrieved_docs = docs
    state.retrieval_score = score

    state.decision = decide(state)

    return respond(state)

This is intentionally small.

The value is not that this code represents a production architecture.

The value is that each responsibility now has somewhere to live.

If the system produces a bad answer, we can ask whether retrieval failed, whether state was incomplete, whether the decision policy allowed generation when it should not have, or whether generation itself failed.

That is more actionable than treating the entire application as one probabilistic black box.

Where should the probability live?

Language-model generation is probabilistic.

For model parameters \(\theta\),

y \sim p_\theta(y \mid x, c)

The model produces an output \(y\) conditioned on input \(x\) and context \(c\).

That variability is often desirable.

Two good explanations of the same concept do not need to use identical wording. A summary can vary. A teaching example can vary. A conversational response can vary.

But not every system behavior should inherit that variability.

Whether a user possesses a permission should probably not vary from run to run.

Whether the application has crossed a defined evidence threshold should not depend on phrasing.

Whether a transaction is authorized should not change because sampling produced a different answer.

The design question is therefore not whether an application uses probabilistic models.

It is:

Where is probabilistic behavior acceptable, and where does the system need explicit control?

That is the architectural boundary.

A small experiment

The easiest way to see the difference is to build the same application three ways.

Use a small document collection and ten questions for which the evidence varies in quality.

In the first architecture, give the model the retrieved context and allow it to decide whether to answer.

result = llm(question, context)

In the second, make answerability an application-level decision.

if retrieval_score >= threshold:
    result = llm(question, context)
else:
    result = abstain()

In the third, keep the same decision boundary but require the generated response to identify the evidence supporting its claims.

Run each question several times.

The interesting observations are not simply whether the answers are correct.

Look for whether the system changes its decision across repeated runs, whether unsupported claims appear, whether abstention behavior remains stable, and whether the available evidence can be connected to the claims the system makes.

This turns variability from an abstract property of language models into something observable at the application level.

The connection to measurement and interpretation

This separation is closely related to the problem explored in BN-001: Separating Measurement From Interpretation.

In xCycleForm, joint coordinates and geometric angles can be calculated deterministically from observed landmarks.

The system can measure:

\theta_{\text{knee}} = \cos^{-1}\left(\frac{(H-K)\cdot(A-K)}{\|H-K\|\|A-K\|}\right)

where \(H\), \(K\), and \(A\) represent the hip, knee, and ankle coordinates.

That calculation and the statement:

Your saddle may be slightly low.

are not the same kind of output.

The first is measurement.

The second is interpretation.

Keeping those responsibilities separate makes the system easier to inspect.

Constraint, state, decision, and response apply the same principle at the application level.

We are not trying to remove intelligence from the model.

We are trying to make clear which intelligence belongs where.

Exercise

Build a small grounded question-answering system with the following decision space:

ANSWER
RETRIEVE_MORE
ABSTAIN

Create two versions.

In the first, allow the model to choose the action and generate the response.

In the second, make the decision in application code and use the model only after ANSWER has been selected.

Then run the same questions repeatedly.

Record the decision and response for every run.

A simple experiment table is enough:

question_id
architecture
run
retrieval_score
decision
supported_claims
unsupported_claims

You are not trying to prove that deterministic code is always better than model reasoning.

You are trying to identify where variability enters the system and whether that variability is acceptable.

That distinction is much more useful.

Test the boundary

Some system behavior should be testable without invoking a model at all.

For example:

def test_low_evidence_abstains():
    state = AppState(
        question="What is the policy?",
        retrieval_score=0.31,
    )

    assert decide(state) == Decision.ABSTAIN


def test_strong_evidence_answers():
    state = AppState(
        question="What is the policy?",
        retrieval_score=0.91,
    )

    assert decide(state) == Decision.ANSWER

These tests do not tell us whether the generated answer will be good.

They tell us something equally important:

The application retained control over when generation is allowed to occur.

That is a property we can test directly.

The takeaway

Language models are powerful reasoning and generation components.

They are not substitutes for application architecture.

When a system becomes difficult to reason about, separate the responsibilities:

Constraint

What is allowed?

State

What does the system currently know?

Decision

What should happen next?

Response

What should the user receive?

If those questions cannot be answered independently, there is a good chance the prompt is carrying responsibilities that belong elsewhere in the system.

The goal is not to make every part deterministic.

The goal is to know which parts are not.

Connected Notes

TN-002When Is the Evidence Enough?Moves inside the decision itself and asks when accumulated evidence is sufficient for the system to stop gathering and act.FN-001Small Is Not OpenSeparates capability, access, control, and evidence so that model-selection decisions are not collapsed into a single question.BN-001Separating Measurement From InterpretationShows the same separation-of-responsibility principle in a working multimodal system, keeping numerical measurement distinct from generative interpretation.

PublishedSeptember 2026

AuthorBen Faircloth

Codegithub.com/benfaircloth/tn-001-constraint-state-decision-response

Cite asFaircloth, B. (2026). "Constraint, State, Decision, Response." BenFaircloth.com, Teaching Note 001.

All teaching notes