Chapter 4: The Agentic AI Stack

Understanding who provides what — and where your effort should focus.

Building an agentic system involves three distinct layers. Clarity on these layers helps you make good "build vs buy" decisions and focus your effort where it creates the most value.

Layer 1: The Model (The Brain)

The large language model provides the core reasoning capability — the ability to understand language, think through problems, and generate responses.

Examples: Claude (Anthropic), GPT-4 (OpenAI), Gemini (Google), Llama (Meta)

You don't build this. You access it via API, paying per use. The model providers have invested billions in training these systems. Your job is to use them effectively, not recreate them.

Layer 2: The Framework/Platform (The Nervous System)

The framework handles everything that turns a model into a functioning agent:

• The agent loop (receive input → reason → act → observe → repeat)
• Tool definitions and execution
• Memory management
• Integration with external systems
• Error handling and retries

Examples of existing platforms:

• Azure AI Foundry (Microsoft)
• Amazon Bedrock Agents (AWS)
• Google Vertex AI Agents
• Mastra (open-source TypeScript)
• LangChain/LangGraph (open-source Python)
• CrewAI (multi-agent focused)

Or you can build your own. Some organisations write custom code to handle the agent loop, tool execution, and memory. This gives maximum control but requires significant development and maintenance effort.

If you choose to build your own framework — or want to understand what's inside the platforms you're evaluating — the logical architecture typically includes four layers: Interface (how users interact), Orchestration (the agent loop), Core Services (model, tools, memory), and External Systems (what the agent connects to).

→ See Addendum: Agentic AI Framework Architecture

Layer 3: The DNA (The Identity)

The DNA — instructions plus knowledge — is what transforms a generic agent into a specialist. This is where your unique value lives.

This is what you design and configure. Regardless of which model or platform you use, the DNA is yours. It's the IP that makes your agent valuable for your specific business context.

Where Should You Focus?

For most organisations, the answer is clear:

The model providers are spending billions on Layer 1. Platform providers are competing fiercely on Layer 2. Neither of these is where you'll differentiate.

Note: the framework architecture described in this chapter — and illustrated in the addendum — reflects a single-agent baseline. When you introduce multiple specialist agents collaborating on a task, Layer 2 expands significantly. The Orchestration Layer grows to include confidence aggregation and inter-agent coordination; the Memory Service expands to include shared state accessible across the agent team. These multi-agent extensions are covered in Part 2, particularly Chapters 9 (Multi-Agent Patterns) and 11 (Memory Patterns).

Runtime Architectures

Not all agents need the same runtime environment. A customer service advisor answering questions requires a very different infrastructure to a software delivery agent that clones repositories, writes code, runs builds, and deploys artifacts. Understanding the runtime options helps you make better infrastructure decisions and avoid over- or under-engineering your solution.

There are three broad runtime architectures for agentic systems, each suited to different use cases.

Model 1: Managed Platform Runtime

Managed platforms like Azure AI Foundry, Amazon Bedrock Agents, and Google Vertex AI Agents handle the runtime for you. Your agent logic runs within the platform's managed infrastructure. You configure the agent's DNA (instructions and knowledge), define tool connections, and the platform manages compute, scaling, and the agent loop.

Best suited for: Advisory and conversational agents, business process automation, customer-facing assistants, and any agent whose primary activity is reasoning and tool calls rather than direct system manipulation. These agents receive a prompt, reason through the problem, possibly call some APIs or retrieve information, and return a response. They don't need file systems, development tooling, or long-running compute.

Trade-offs: Fastest path to production with minimal infrastructure management. However, you're constrained by what the platform supports and locked into that provider's ecosystem. Customisation of the agent loop, memory management, and orchestration patterns is limited to what the platform exposes.

Example: The Pragmatix Advisory Portal runs specialist advisors on Azure AI Foundry. Each advisor is a stateless reasoning engine with tool access — the managed platform handles all runtime concerns.

Model 2: Shared Infrastructure Runtime

You build your own Layer 2 framework and deploy it on shared infrastructure — a server, a container, or a set of containers. All agents share the same runtime environment, with the framework managing the agent loop, tool execution, and memory.

At its simplest, an agent in this model is: a server that receives a user message, code that sends that message to an LLM API along with system instructions and tool definitions, code that handles the response (executing tool calls and passing results back to the model), and a loop that repeats until the model produces a final answer. Your code is the framework — the glue that manages this back-and-forth, handles errors, manages memory, and exposes it through an API or interface.

Best suited for: Organisations that need more control over the agent loop, custom orchestration patterns, or portability across cloud providers. Also appropriate when you need to integrate with systems or protocols that managed platforms don't support, or when you want to avoid vendor lock-in.

Trade-offs: Full control and portability, but you own the infrastructure, scaling, and operational burden. Agents share resources, so a misbehaving agent could affect others. Simpler to operate than container-per-agent models, but less isolated.

Example: A custom agentic framework running on Azure Container Apps, with multiple specialist agents sharing the same deployment. The framework uses PostgreSQL for memory, Redis for caching, and calls Claude or GPT-4o for reasoning.

Model 3: Ephemeral Agent Runtimes (Container-per-Agent)

In this model, each agent instance spawns in its own isolated, ephemeral runtime environment — typically a container or managed development environment. An orchestrator decides what agents are needed, provisions the runtime, the agent does its work, and the environment is destroyed when the task is complete.

Platforms like Daytona, GitHub Codespaces, and Gitpod provide the infrastructure for this pattern. They offer API-driven provisioning of standardised environments with full development tooling — precisely what code-writing agents need.

Best suited for: Software delivery agents, code generation teams, testing and QA automation, data pipeline execution, or any use case where agents need to manipulate files, execute code, run builds, or operate with full development tooling. Particularly valuable for multi-agent systems where parallel execution and strict isolation between agents are important.

Trade-offs: Maximum isolation and scalability. Each agent gets a clean environment, preventing interference between tasks. Resources scale with workload, and you only pay for compute when agents are active. However, this is the most complex model to build and operate. Environment provisioning adds latency, and coordinating state across ephemeral containers requires careful design.

Example: An agentic software delivery platform where a supervisor agent receives a feature request, then spawns specialist agents in isolated Daytona environments — a discovery agent analyses requirements, an architecture agent designs the solution, coding agents write and test the code in parallel containers, and a deployment agent handles release. Each agent has its own workspace with git, compilers, and testing frameworks available.

Choosing the Right Runtime

The right runtime architecture depends on what your agents actually do, not on what sounds most sophisticated. Match the architecture to the use case.

The Pragmatic Approach

As with all technology decisions, start with the simplest architecture that meets your needs:

• If your agents reason and call tools — a managed platform is likely sufficient. Don't build infrastructure you don't need.
• If you need control or portability — build your own framework on shared infrastructure. This gives you flexibility without the complexity of managing ephemeral environments.
• If your agents execute code or manipulate systems — consider ephemeral runtimes. The isolation and clean-state benefits justify the additional complexity.

Many organisations will use more than one model. Advisory agents might run on a managed platform while software delivery agents use ephemeral runtimes — all coordinated through a shared orchestration layer. The key is matching the runtime to the workload, not defaulting to the most complex option.