An enterprise AI agent rarely fails because the underlying model cannot generate a response. It fails when the system around the model cannot control what happens next.

A planner selects the wrong tool. A tool returns incomplete context. Memory preserves irrelevant state. An evaluator checks the final answer but misses a dangerous intermediate action. An observability layer records latency but cannot explain why the agent made a decision. Six months later, the organization has an AI system in production that nobody can fully trace.

The architecture matters more than the model.

A production-grade enterprise AI agent needs explicit boundaries between reasoning, action, context, evaluation, and operations. This article describes the reference architecture we use to think about those boundaries: what each component does, where it fails, and what should be instrumented from the first deployment.

The Agent Is Not the Architecture

The model is one component inside the system. It is not the system itself.

The current generation of agentic systems has moved beyond prompt-response applications. Agents can select tools, retrieve information, maintain state, execute actions, evaluate intermediate results, and adapt their process based on what they observe.

Anthropic's engineering work describes the distinction clearly: workflows follow predefined paths, while agents dynamically direct their own processes and tool use. The same work also recommends increasing architectural complexity only when the task requires it. That distinction matters in enterprise environments. Consider an agent responsible for handling a maintenance escalation.

The request might begin with:

"Investigate the repeated failure on Line 4 and recommend the next action."

The agent may need to:

1. Identify the relevant asset.
2. Retrieve maintenance history.
3. Query sensor data.
4. Compare previous failure patterns.
5. Check inventory.
6. Determine whether a maintenance procedure applies.
7. Generate a recommendation.
8. Request human approval.
9. Create a work order.

The model is involved throughout the process, but it is not responsible for every function.The system needs a planner to determine the next step.

• It needs tools with clearly defined permissions.
• It needs memory to preserve relevant state.
• It needs evaluators to determine whether outputs and actions meet defined criteria.
• It needs observability to record what happened.
• It needs governance around the entire execution path.

This is where many enterprise implementations go wrong. They begin with the model. The better starting point is the system boundary.

The reference architecture

At Prestine, we think about an enterprise AI agent as a set of connected control layers:

User / Enterprise System → Agent Interface → Planner → Tools & Enterprise Systems → Memory & Context → Evaluator → Observability & Governance

These layers are not necessarily implemented as separate products or services. They are architectural responsibilities. The implementation can vary. The responsibilities cannot disappear.

Reframing Enterprise AI Agent Architecture

The central architectural problem is not how an agent reasons. It is how an enterprise controls what the agent is allowed to do.

That changes the design conversation. A model can produce a technically impressive response and still create an unacceptable enterprise system.

For example, imagine an agent that can update an ERP record.

The model may correctly understand the user's intent. It may select the correct ERP function. It may generate the correct parameters.

But what happens if:

• the user does not have permission to perform the action
• the ERP system is unavailable
• the tool returns stale information
• the model calls the tool twice
• the returned data contradicts another enterprise system
• the action cannot be reversed
• the model continues after an unexpected tool response
• nobody can reconstruct the execution six weeks later

These are architecture problems. They cannot be solved by selecting a more capable model alone. This is why enterprise AI agent architecture needs to be treated closer to distributed systems engineering than conversational interface design.

The control plane around the model

The model provides reasoning capability. The architecture provides control. That control should exist at several levels.

Before execution: identity, authorization, context, task boundaries, and policy.

During execution: tool permissions, state management, retries, timeouts, validation, and human intervention.

After execution: evaluation, tracing, audit records, performance measurement, and continuous improvement.

National Institute of Standards and Technology(NIST) AI Risk Management Framework and its Generative AI Profile reinforce this lifecycle perspective. The GenAI profile identifies 13 categories of risk and more than 400 suggested actions for managing those risks, emphasizing that trustworthy AI requires practices across the system lifecycle rather than a single model-level control.

The implication for architecture is direct:

Governance cannot be added after deployment. It needs an address in the architecture.

This is also why agentic systems need stronger evaluation than conventional AI applications. Agents operate across multiple steps, make tool calls, modify state, and adapt based on intermediate results. Anthropic's 2026 work on agent evaluations notes that these capabilities make agents harder to evaluate and that evaluations need to make behavioral changes visible before they reach production.

The architecture therefore needs to answer two questions simultaneously:

Can the agent complete the task?

And:

Can the enterprise understand and control how it completed the task?

The second question is where production systems are separated from demonstrations.

The Reference Architecture We Deploy

A useful enterprise AI agent architecture can be organized around seven core responsibilities. The exact technology stack can change. The responsibilities remain stable.

1. Planner and Orchestrator

The planner determines what should happen next.

It interprets the task, decomposes it when necessary, selects the appropriate workflow or tool, and determines when the task should stop or request human input. This is the agent's control logic.

For a procurement agent, the planner might determine:

Request → Supplier lookup → Contract validation → Price comparison → Policy check → Approval → Purchase request

For a maintenance agent:

Asset identification → Failure history → Sensor analysis → Diagnosis → Parts availability → Recommendation → Human approval → Work order

The planner should not have unrestricted access to every enterprise capability. That creates an uncontrolled execution surface. Instead, planning should operate within defined capabilities and policies.

Failure mode

The most common failure is planning drift.

The agent begins with a valid objective but takes unnecessary steps, loops between tools, or pursues a locally reasonable action that does not advance the business objective.

Instrumentation pattern

Record:

• task ID
• plan version
• step sequence
• selected tool
• reason for tool selection where available
• execution time per step
• retries
• abandoned paths
• escalation events
• final outcome

The goal is not to record every token. The goal is to reconstruct the execution path.

2. Tools and Enterprise Actions

Tools are where an agent touches the enterprise.

They may connect to:

• ERP systems
• CRMs
• databases
• document repositories
• IoT platforms
• maintenance systems
• identity systems
• ticketing platforms
• internal APIs
• external services

This layer deserves particular engineering attention. A tool is not merely an API endpoint exposed to an LLM. It is an action boundary.

Anthropic's recent engineering guidance emphasizes that tools for agents need clear interfaces, meaningful context, appropriate boundaries, and evaluation because agents interact with tools differently from deterministic software systems.

Failure mode

The major failure modes are:

• ambiguous tool definitions
• excessive permissions
• poor input validation
• insufficient context
• inconsistent return formats
• irreversible actions without approval
• duplicate execution

A tool that allows an agent to "update customer information" is too broad. A tool that allows the agent to update a specific field under defined conditions is easier to govern.

Instrumentation pattern

Every tool invocation should produce a structured record containing:

Agent ID → User ID → Tool ID → Input schema → Authorization decision → Execution status → Response metadata → Duration → Error code → Result

Sensitive payloads should be handled according to enterprise security and privacy requirements. The important principle is straightforward:

Every meaningful agent action should be attributable.

3. Memory and Context

Memory determines what the agent knows about the current and previous state of the task.

But memory is frequently misunderstood. More memory does not automatically produce a better agent. Enterprise agents need controlled context.

There are usually several distinct forms:

Working context

Information required to complete the current task.

Persistent memory

Information that should remain available across interactions.

Enterprise knowledge

Documents, records, policies, procedures, and structured data retrieved when needed.

Execution state

The machine-readable state of the workflow itself. These should not be treated as one undifferentiated memory store.

For example, a maintenance agent may need the current machine condition, historical work orders, manufacturer documentation, spare-parts availability, and the current approval state. Each has a different lifecycle and trust boundary.

Failure mode

The failure is usually context contamination.

Irrelevant, stale, contradictory, or unauthorized information enters the agent's context and influences its decision.

Instrumentation pattern

Track:

• context sources
• retrieval identifiers
• document versions
• timestamps
• relevance scores where applicable
• memory reads and writes
• context size
• failed retrievals
• stale-data conditions
• authorization status of retrieved information

Memory should be treated as a controlled data dependency, not as an invisible feature.

4. Evaluator

The evaluator determines whether the agent's behavior was acceptable. This is one of the most important components in a production architecture. An evaluator can operate at multiple levels.

Output evaluation

Was the final answer correct, complete, grounded, and compliant?

Tool evaluation

Did the agent select the right tool?

Step evaluation

Was each intermediate action valid?

Outcome evaluation

Did the execution actually achieve the business objective? These are different measurements. An agent can produce an accurate explanation and still fail operationally because it never completed the required transaction.

NIST's work on evaluating machine-generated reports similarly emphasizes completeness, accuracy, and verifiability rather than treating fluent generation as sufficient evidence of quality.

Failure mode

The common failure is evaluating only the final response. That hides the path that produced it.

An agent may arrive at a correct final answer after making an unauthorized tool call or retrieving incorrect information.

Instrumentation pattern

Create evaluation records for:

• task success
• factual accuracy
• tool selection
• policy compliance
• groundedness
• escalation correctness
• latency
• cost
• failure category
• human correction

Evaluations should run before deployment and continue after deployment. The evaluator is part of the learning system.

5. Observability

Observability answers a different question:

What actually happened?

Traditional application monitoring often focuses on uptime, latency, errors, and infrastructure health. Agentic systems require more. An agent can return a technically successful HTTP response while making a poor business decision.

You need visibility into the execution chain. A useful trace might look like:

Task received → Context retrieved → Plan generated → Tool selected → Authorization passed → Tool executed → Result evaluated → Plan updated → Human approval requested → Action completed

That trace becomes the operational record.

Failure mode

The most expensive failure is often invisible failure. The system works most of the time, but a small percentage of tasks fail in ways nobody can explain.

Without traces, engineers inspect prompts.

• Then models.
• Then APIs.
• Then databases.
• The investigation becomes guesswork.

Instrumentation pattern

At minimum, instrument:

• request ID
• user/session ID
• agent version
• model version
• prompt or policy version
• tool calls
• retrieval events
• state transitions
• evaluator results
• latency
• token usage
• cost
• errors
• escalation
• final outcome

This creates a production trace rather than a black box.

6. Identity, Policy, and Governance

Governance should not sit beside the architecture as a policy document. It should sit inside the execution path.

The agent needs to know:

• who initiated the task
• what they are authorized to access
• what the agent itself is authorized to do
• which data it can retrieve
• which actions require approval
• which actions are prohibited
• what must be logged
• when execution must stop

This becomes especially important as agents move from information retrieval toward action.

Reading a purchase order and approving a purchase order are not equivalent permissions.

Retrieving maintenance history and closing a maintenance work order are not equivalent permissions.

NIST's AI RMF is explicitly designed to support organizations in managing AI risks throughout the design, development, deployment, and use of AI systems.

Failure mode

The most common governance failure is treating the agent as a user with broad permissions.

Instrumentation pattern

Log:

Identity → Policy → Requested action → Authorization decision → Approval requirement → Decision → Action

When an enterprise cannot reconstruct this chain, it has an accountability gap.

7. Human Control

The final architectural component is not technical. It is the human decision boundary. Not every task should be fully autonomous.

A production agent should know when to continue, when to stop, and when to request intervention.

The threshold should depend on:

• financial impact
• operational risk
• reversibility
• regulatory requirements
• data sensitivity
• confidence
• business criticality

For example:

Retrieve maintenance history: autonomous.
Generate maintenance recommendation: autonomous with evaluation.
Create draft work order: autonomous.
Authorize high-value equipment shutdown: human approval.

This is not a weakness in the architecture. It is a deliberate control mechanism.

Recent research from Anthropic on trustworthy agents similarly emphasizes keeping humans in control, securing agent interactions, maintaining transparency, and protecting privacy as agent capabilities expand.

The Architecture in Production

The components become useful when they operate as one system.

A production execution might look like this:

1. Request

A plant manager asks the agent to investigate repeated failures on a production asset.

2. Identity

The system validates the user's identity and operational permissions.

3. Planner

The agent decomposes the task into asset identification, historical analysis, sensor retrieval, diagnosis, and recommendation.

4. Context

The system retrieves the asset record, maintenance history, relevant procedures, and current sensor data.

5. Tools

The agent calls approved enterprise systems through bounded tool interfaces.

6. Evaluation

Each significant output is checked against defined criteria.

7. Human Control

If the recommended action involves a high-risk operational decision, the system requests approval.

8. Execution

Once approved, the relevant enterprise action is executed.

9. Observability

The entire execution trace is recorded.

10. Learning

The outcome becomes evaluation data for future system improvement. This is the difference between an agent demo and an enterprise system.

The demo asks:

Can the model complete the task?

The production architecture asks:

Can the system complete the task reliably, safely, measurably, and repeatedly?

What Not to Build

There are several architectural shortcuts that appear attractive early and become expensive later.

One giant agent

Give one agent access to every database, API, document store, and business function. The result is a large execution surface with unclear responsibility.

Tool access without boundaries

If every tool is available for every task, authorization becomes difficult to reason about. Tools should represent defined capabilities, not unrestricted system access.

Memory without lifecycle rules

Persisting everything creates context, privacy, and data-quality problems. Memory needs ownership, retention, access rules, and relevance controls.

Evaluation only in staging

Agent behavior changes as models, prompts, tools, data, and policies change. Evaluation needs to remain part of the production lifecycle.

Observability as an afterthought

If tracing is added after failures occur, critical execution context may already be missing. Instrumentation belongs in the architecture from the first production deployment.

The production boundary is where intelligence becomes an engineering responsibility.

A Reference Pattern for Enterprise AI

The architecture can be reduced to one operating principle:

Separate reasoning from authority.

The model can reason. The planner can coordinate. Tools can act. Memory can provide context. Evaluators can measure. Governance can constrain. Observability can explain. Humans can intervene.

Each responsibility has a defined boundary.

That makes the system easier to test, operate, change, and scale.

It also prevents a common architectural mistake: placing too much responsibility inside the model itself.

The model should not become the database.

• It should not become the authorization layer.
• It should not become the audit trail.
• It should not become the workflow engine.
• It should not become the monitoring system.

Those are system responsibilities. The model is one component within that system.

From Agent Architecture to AI Transformation

Enterprise AI agents are becoming part of larger operational systems.

They will increasingly interact with ERP platforms, industrial systems, customer workflows, knowledge repositories, IoT infrastructure, and human decision processes.

That means the architecture cannot stop at the agent. The agent must become part of the enterprise operating model. This is where the P.A.I.L.O.T™ lifecycle becomes relevant.

Problem Discovery determines whether an agent is actually required.

AI Opportunity Mapping identifies where agentic execution creates measurable value.

Implementation establishes the technical system.

Learning Systems create the evaluation and feedback loop.

Operational Integration connects the agent to real enterprise workflows.

Transformation at Scale turns an isolated implementation into a repeatable capability.

The reference architecture sits primarily inside Implementation, Learning Systems, and Operational Integration. But the architecture should never be designed independently of the business problem. The strongest agent is still the wrong system if it solves the wrong operational problem.

Conclusion

Enterprise AI agents are moving from conversational interfaces into real workflows. That shift changes the engineering requirement.

A production agent needs more than model capability. It needs bounded tools, controlled context, explicit memory, measurable evaluation, traceable execution, policy enforcement, and clear human decision boundaries.

The model provides intelligence.

The architecture makes that intelligence operational.

If you're not sure where your AI initiatives stand today, our AI Maturity Assessment maps your position across the P.A.I.L.O.T lifecycle in under 3 minutes. Start at prestine.ai/ai-assessment