Agent runtimes, orchestration and harnesses

On this page

Chapter 6 of The Hitchhiker's Guide to the Agentic Economy

Report hub · Previous: context, memory and data · In this chapter: the controlled execution path · run-state machine · market map · sandboxes · run contract · Next: tools, browsers and service access

The mechanics of durable software execution

When an agent workflow runs for longer than a single HTTP request, it requires an execution environment capable of surviving infrastructure failures without duplicating external actions or losing task state.

A software framework makes it trivial to invoke a model and pass its response into a tool. Moving that interaction into enterprise production requires managing a much wider lifecycle: preserving intermediate progress across worker crashes, pausing execution while waiting for human authorization, applying safe retry limits to network timeouts, and verifying whether an external write operation actually committed before replaying an action.

In production architectures, commercial defensibility lies in the controlled execution path rather than the specific orchestration framework. Five operational properties govern reliability:

1. State persistence: The execution run can pause and resume across distributed workers without losing task variables or execution history.
2. Idempotent retry mechanics: Network timeouts and transient errors are retried using unique transaction keys that prevent duplicate external side-effects.
3. Crash resilience: Process crashes, container evictions, or node restarts do not silently drop in-flight jobs.
4. Inspectable transitions: Every state change, tool call, and policy check is logged with monotonic identifiers accessible to system operators.
5. Reconciliation boundaries: When an external tool call fails ambiguously, the runtime pauses execution until the real-world effect can be reconciled, rather than assuming the action failed.

Distinguishing these components requires separating three related layers:

• Runtime: The underlying processes, worker pools, and compute infrastructure that execute code, enforce CPU/memory quotas, and handle network I/O.
• Orchestration: The state machine logic, scheduling queues, dependency graphs, approval gates, and compensation routines coordinating the workflow.
• Harness: The operational perimeter surrounding the foundation model, managing context ingestion, persistent memory, tool access credentials, and approval policies.

Guidance from the Australian Signals Directorate highlights that an agentic AI harness—encompassing the runtime, memory, tools, and permissions—is the decisive control surface governing enterprise security and operational reliability (ASD, "Agentic AI harnesses").

Key Takeaways

• A model response becomes a durable unit of work only when the system gives it a run identity, a state record, a recovery path and an acceptance condition.
• Checkpointing the orchestration state does not prove that a network side effect happened once. The side effect needs its own intent, idempotency and receipt or reconciliation record.
• Waiting for a person, retrying a transient error, cancelling a run and compensating an earlier effect are different states. Combining them into failed hides operational decisions.
• A sandbox can isolate files, processes and network access while the orchestrator loses the job, or the orchestrator can survive while the sandbox expires. The contract must join both lifecycles.
• Documented runtime capability is abundant across providers. Comparable evidence of production reliability across them is not.
• The right commercial comparison is cost per accepted job, including model calls, orchestration, sandbox resources, storage, review, recovery and unresolved effects.

Runtimes decouple from workflow orchestration

The market is separating into four distinct layers: agent development frameworks (which chain model calls and tool definitions), managed cloud runtimes (which host and supervise agent processes), durable workflow engines (which persist long-running task state), and isolated execution sandboxes (which isolate untrusted code and web browsers).

These boundaries are actively shifting. Model creators are extending their platforms into managed execution environments. Workflow automation vendors are adapting enterprise state engines to non-deterministic agent workloads. Sandbox providers are exposing programmatic session APIs that allow runtimes to spin up short-lived microVMs on demand.

OpenAI’s platform trajectory emphasizes this convergence: “Useful agents need a powerful harness that manages context, uses tools efficiently, and coordinates subagents” (OpenAI, "Introducing the Agents API"). Hyperscalers and model providers are actively expanding beyond raw token generation into the operating software layer that supervises execution.

A single model call is not a unit of work

A single model invocation is a synchronous request and response. A commercial unit of work encompasses a multi-stage operational lifecycle:

accept objective and authority
  -> create job and run identity
  -> select versioned context and policy
  -> execute bounded steps
  -> record checkpoints and effects
  -> wait, ask for approval or retry when required
  -> reconcile external state
  -> deliver and obtain acceptance

A foundation model can complete its text generation while the underlying business job remains unresolved. An external API can time out after processing an order. A human manager may take two days to approve a purchase request. A cloud container may be evicted by an autoscaler after committing a database change but before logging the receipt. A secure sandbox can reach its execution timeout while an analytical script is mid-calculation. Each event requires an explicit state transition and an operational recovery procedure.

Separation of operational responsibilities

Building a reliable execution pipeline requires clear division of labor across three layers:

Architectural layer	Core operational role	Key questions answered	What it cannot guarantee alone
Execution runtime	Manages OS processes, allocates worker threads, streams I/O, enforces timeouts	Where is the process running? Is its worker lease valid? What memory is consumed?	Whether the broader business deliverable succeeded or external writes committed
Workflow orchestrator	Manages state machines, schedules graph steps, handles pauses, retries, and rollbacks	What step executes next? What checkpoint is saved? What state must replay?	Whether an external third-party API committed a write during a network disconnect
Agent harness	Integrates model inference, context memory, tool access, and policy guardrails	What data may the model see? What tools can it invoke? What action requires approval?	Whether a syntactically valid tool execution produced the intended business result

The application and its underlying systems of record maintain authoritative commercial truth. The runtime and orchestration layer provides the operational rails that allow those systems to be updated safely.

The agent run-state machine

Reliable execution requires modeling agent behavior as an explicit finite state machine:

admitted
   |
   v
running -------------------------------> completed
   |                                          ^
   |                                          |
   +--> waiting_for_input --------------------+
   |          |
   |          +--> running
   |          +--> cancelled
   |
   +--> awaiting_approval -------------------> running
   |          |
   |          +--> cancelled
   |
   +--> retry_wait --------------------------> running
   |          |
   |          +--> failed
   |
   +--> compensating --> compensated
   |          |
   |          +--> failed
   |          +--> needs_reconciliation
   |
   +--> cancelling --> cancelled
   |          |
   |          +--> needs_reconciliation
   |
   +--> needs_reconciliation --> reconciling
                                  |
                                  +--> completed
                                  +--> compensated
                                  +--> failed
                                  +--> cancelled

These states reflect operational realities:

• admitted confirms the job has entered the queue; it does not indicate business acceptance.
• waiting_for_input pauses execution pending missing user data.
• awaiting_approval suspends execution before committing a consequential action until an authorized human signs off; an expired approval terminates the step rather than defaulting to execution.
• retry_wait pauses the run under exponential backoff with a hard attempt limit.
• compensating executes explicit rollback routines to reverse earlier external side-effects where supported.
• needs_reconciliation flags that an external action completed ambiguously (e.g., an HTTP 504 Gateway Timeout during an order post) and prohibits automated retries until the real-world state is verified.

Every transition logs an immutable record: run ID, monotonic sequence number, timestamp, initiating identity, previous state, next state, and reason code.

Retries as a risk-governed decision

Automated retries are safe only under strict technical conditions: restoring an in-memory worker from a durable checkpoint prior to an external call, handling a transient 429 rate limit with a clear Retry-After header, or reconnecting to a read-only database.

Retries are dangerous when the runtime cannot confirm whether an external write operation executed. If a payment gateway accepts a charge but the connection drops before returning an HTTP 200, blindly retrying the request duplicates the charge. Production systems require a persistent idempotency key tied to the logical business effect rather than the network attempt. If the target system cannot confirm whether the key was processed, the run must transition to needs_reconciliation for manual review.

The double-durability seam

The most critical operational seam in agent systems sits between orchestration durability and external effect durability:

orchestration durability                    effect durability
-------------------------                   ---------------------------
run identity                                logical effect identity
checkpoint / event history                  intent record
lease / worker ownership                   idempotency key
pause / resume                              provider receipt or status
retry / cancellation                        reconciliation and compensation

Orchestration durability tracks internal state: What does the agent believe, and what instruction executes next? Effect durability tracks external reality: What action did the external third-party API actually commit?

A durable workflow engine can replay an execution graph perfectly from its event log. But if an intermediate activity invoked a non-idempotent external API, replaying that step repeats the real-world side effect. "Exactly once" execution within an orchestrator does not guarantee exactly-once execution across distributed web APIs. Systems must record explicit intent logs and idempotency keys before invoking external endpoints.

What durable execution records must capture

An auditable run record captures ten operational dimensions:

1. Sponsoring business owner, task objective, and acceptance criteria.
2. Version identifiers for the workflow graph, system prompt, model weights, tool schemas, and policy definitions.
3. Checkpointed state history, including the latest verified safe recovery point.
4. Input parameters, retrieved context hashes, and raw tool output payloads.
5. Unique effect identifiers, idempotency keys, intent records, and returned provider receipts.
6. Human approval requests, approver credentials, timestamps, and explicit authorization scope.
7. Retry counts, backoff parameters, timeout ceilings, and cancellation records.
8. Execution sandbox ID, container image hash, egress network policy, and volume snapshot references.
9. Final delivery artifacts, error logs, compensation traces, and manual reconciliation summaries.
10. Data retention schedules, deletion verification, and incident escalation owners.

Who supplies runtimes and orchestration infrastructure

The market spans development frameworks, managed cloud runtimes, durable workflow platforms, and isolated compute sandboxes:

Infrastructure layer	Representative vendors	Documented capability	Commercial model	Operational boundary
Graph framework & control plane	LangGraph and LangSmith	Checkpoints graph state by thread, supports interrupts, resume and retry; LangSmith Agent Server adds persistent queues, PostgreSQL storage, and Redis signaling (Agent Server; deployment)	Open-source framework paired with hosted/self-hosted observability and deployment licensing	Framework persistence abstracts state; application must still manage external API idempotency and business acceptance
Agent SDK & runner	OpenAI Agents SDK	Manages multi-turn loops, tool execution, sub-agent handoffs, and guardrails; RunState serializes paused runs for human approval (running agents; human in the loop); documents integrations with Temporal and Dapr	Open-source client library driving OpenAI model and platform consumption	Client runner serializes local state; durable cross-node execution requires external infrastructure (Temporal)
Managed agent runtime	Google ADK and Agent Runtime	Multi-agent workflow orchestration with managed containerized runtime (runtime); resumes workflow tasks from an invocation ID, but notes side-effecting tools may execute more than once (resumability)	Open-source framework paired with Google Cloud managed compute consumption	Managed hosting abstracts container operations; side-effecting tools still require custom duplicate-prevention controls
Durable framework extension	Microsoft Agent Framework	Executes graph supersteps with state checkpoints; Durable Extension integrates with Azure Durable Functions for long-running state and human waits (checkpoints; durable extension)	Cloud consumption across Azure Functions, Foundry Agent Service, or enterprise compute	Checkpoints capture executor state; external legacy database updates require separate transaction logging
Durable workflow engine	Temporal	Crash-resilient event-sourced workflows that persist state across worker, network, and host outages; manages long-running agent loops and retryable pipelines (durable AI)	Open-source core with managed Temporal Cloud consumption and storage pricing	Guarantees deterministic workflow replay; external activities interacting with third-party APIs still require custom idempotency keys
Interactive workflow engine	Prefect	Explicit task state tracking, automatic retries, caching, and concurrency limits; flows can suspend for typed human input (interactive workflows; states)	Open-source engine paired with Prefect Cloud control-plane plans	Workflow primitives track task lifecycles; does not evaluate model reasoning validity or factual accuracy
Data asset orchestrator	Dagster	Op retry policies, run recovery following worker failure, and execution re-runs reusing cached assets (ops; run retries; re-execution)	Open-source orchestration paired with Dagster Plus enterprise hosting	Optimized for data pipeline assets; external non-idempotent tool side-effects require custom handling
MicroVM agent sandbox	E2B	Isolated sandboxes for executing code and browser tools, custom template builds, and fast filesystem snapshots (docs; pricing)	Hosted microVM runtime billed by compute second, with enterprise VPC options	Provides execution isolation; does not manage outer workflow orchestration or external payment receipts
Serverless compute sandbox	Modal Sandboxes	Serverless container execution with reconnectable session IDs, filesystem snapshotting, and metered GPU/CPU (sandboxes; resources)	Usage-based compute metering based on requested CPU, memory, and wall-clock time	Ephemeral containers provide execution boundaries, not persistent workflow event histories
Isolated workspace platform	Daytona Sandboxes	Dedicated container and VM environments with network namespace isolation, GPU support, pause/resume, and snapshots (architecture; billing)	Managed SaaS or self-hosted control plane with resource-based pricing	Workspaces survive process pauses, but recovering external API side-effects requires application-level ledgers
Edge serverless sandbox	Cloudflare Sandbox	Runs isolated containers via Cloudflare Workers, Durable Objects, and Containers, supporting commands, files, and background daemons (security; pricing)	Metered compute and storage under Cloudflare Workers paid subscription plans	Sandbox SDK 1.0 currently designated preview; provides execution isolation at the network edge

These technologies compose into layered systems: an OpenAI runner can execute commands inside an E2B microVM, orchestrated by a Temporal workflow that checkpoints state into PostgreSQL. System reliability is determined by how cleanly state is handed off across these boundaries.

Evaluating runtime claims

Evidence type	What it establishes	Unresolved operational question
Provider documentation	Stated feature sets, API parameters, and deployment models	Real-world reliability across complex multi-step failure modes
Open-source repository	Publicly inspectable code, commit history, and issue trackers	Production performance of managed, proprietary cloud versions
Commercial pricing schedule	Metering units for compute duration, storage, and control planes	Net operating cost per successfully completed business task
Partner case study	A documented deployment by a named commercial partner	Generalizability across diverse enterprise legacy IT environments
Production trace logs	Verifiable data on task recovery, duplicate rates, and latency	Performance under unpredicted third-party API outages
Independent benchmark	Comparative performance under declared stress protocols	Long-term software maintainability and vendor lock-in risks

Commercial models in execution infrastructure

Four primary commercial models support the execution layer:

1. Open-source framework with commercial control plane: LangGraph, OpenAI Agents SDK, and Google ADK provide open development libraries while monetizing hosted deployment, telemetry tracing, seat licenses, and enterprise security features.
2. Managed agent runtimes: Platforms like Google Agent Runtime and Microsoft Foundry host worker processes, manage container lifecycles, and scale compute automatically, charging for underlying cloud resources while reducing infrastructure management overhead.
3. Durable workflow engines: Temporal and Prefect monetize centralized control-plane capacity, workflow event history storage, and high-availability enterprise clustering.
4. Metered compute sandboxes: E2B, Modal, Daytona, and Cloudflare bill for active CPU cores, memory allocation, disk I/O, and container runtime seconds.

Calculating the unit economics of an agent system requires aggregating all infrastructure costs against verified business deliverables:

cost per accepted job =
  (model and inference
   + orchestration and state
   + sandbox and compute
   + storage and network
   + human approval and correction
   + recovery, duplicate and reconciliation work)
  / accepted jobs

Sandboxes as an isolation perimeter

When an agent executes code, runs shell commands, renders documents, or browses the web, it must be isolated to prevent prompt injection or malicious code from compromising enterprise infrastructure. Sandboxes provide containerized, virtualized environments that isolate compute, disk storage, and network access:

Sandbox requirement	What the sandbox provider manages	What the orchestrator must record
Execution isolation	Ephemeral microVM or container boundary restricting filesystem and process escape	Image digest, tenant ID, injected credential scope, and resource limits
Workspace state	Filesystem snapshots, persistent volumes, and memory dumps	Snapshot ID, creation timestamp, and state validation upon restoration
Process lifecycle	Container startup, remote command execution, timeouts, and resource reclamation	Sponsoring job ID, active execution step, and orphan process cleanup rules
Long-running work	Extended container lifetimes, volume snapshots, and state resumption	Verification that user approvals and spending authority remain valid upon resume
External actions	Network egress routing through managed proxies	Intent records, idempotency keys, and external transaction receipts

OpenAI’s documentation covers hosted sandbox integrations including E2B, Modal, Daytona, and Cloudflare (OpenAI sandbox agents). Google ADK provides managed code execution within its agent engine (Google Cloud code execution).

However, sandboxes do not replace workflow orchestration. A sandbox can crash while the orchestrator remains healthy; conversely, an orchestrator can restart while its sandbox environment has expired. Production systems treat sandboxes as leased, ephemeral compute resources whose lifecycles are explicitly bound to the parent workflow contract.

Critical sandbox security controls

• Strict egress filtering: Restricting outbound network connections to explicit, whitelisted domains to prevent data exfiltration.
• Short-lived credentials: Injecting temporary, scoped API tokens directly into process memory rather than saving permanent credentials into workspace files.
• Resource quotas: Enforcing hard limits on CPU cores, RAM, disk space, and process counts to prevent runaway resource consumption.
• Snapshot sanitization: Ensuring restored filesystem snapshots do not carry poisoned context or compromised binaries into subsequent runs.
• Automated reclamation: Verifying that containers, scratch volumes, and background processes are purged immediately upon task termination.
• Tenant isolation: Enforcing dedicated compute isolation to prevent data leakage between different enterprise clients.

The Agent Run Contract

Production deployment requires a formal operational contract governing execution persistence, external effect idempotency, and failure recovery:

contract_version: 1

job:
  job_id: <stable business job identifier>
  objective: <expected outcome in plain language>
  acceptance_rule: <who accepts what evidence>
  owner: <accountable person or team>
  authority_reference: <policy and delegated authority record>

run:
  run_id: <unique run identifier>
  parent_run_id: <optional parent or handoff run>
  workflow_version: <immutable orchestration version>
  model_and_prompt_versions: <version references>
  state: admitted | running | waiting_for_input | awaiting_approval |
         retry_wait | compensating | cancelling | needs_reconciliation |
         reconciling | completed | compensated | failed | cancelled
  transition_log: <append-only state transition references>

checkpoint:
  store: <durable store and region>
  last_committed_transition: <transition identifier>
  resume_rule: <what may replay and what must not replay>
  retention_and_deletion: <owner, duration and deletion evidence>

retry_and_timeout:
  retryable_failures: <declared classes only>
  max_attempts: <bounded number>
  backoff_and_jitter: <rule>
  model_tool_run_timeouts: <separate limits>
  cancellation_grace: <termination rule and evidence>

effects:
  - effect_id: <immutable identifier for one intended external consequence>
    attempt_ids: <ordered attempt identifiers for this effect>
    intent_record: <what was requested and when>
    idempotency_key_and_namespace: <stable across retries and unique at the destination>
    receipt_or_status_query: <destination evidence for this effect>
    reconciliation_status: <known, unknown, reconciling or resolved>
    ambiguity_rule: <when to enter needs_reconciliation>
    compensation_rule: <reversal or human route where supported>

waits_and_approvals:
  - wait_id: <immutable identifier for one wait>
    input_required: <field, source and expiry>
  - approval_id: <immutable identifier for one approval decision>
    approval_scope_and_digest: <canonical target, operation, material fields and timing>
    approver: <identity and separation requirement>
    approval_expiry: <timestamp and denial or expiry outcome>

execution_isolation:
  sandbox_provider_and_id: <provider, sandbox and lease>
  image_or_template_digest: <immutable execution version>
  network_and_secret_policy: <allowlist and credential scope>
  resource_limits: <cpu, memory, disk, process and wall clock>
  snapshot_or_workspace_reference: <state and restore evidence>
  cleanup_owner: <termination and orphan cleanup responsibility>

recovery_and_completion:
  process_failure_action: <resume, inspect, reconcile or stop>
  dependency_failure_action: <retry, wait, alternate or fail>
  operator_stop_path: <who can stop and how>
  delivery_record: <result identifier, version and recipient>
  effect_receipt_references: <one per material effect>
  reconciliation_record_references: <one per unknown or disputed effect>
  business_acceptance_reference: <separate job-level decision; may remain open after run completion>
  unresolved_items: <none, or named reconciliation owner and deadline>

Core operational invariants

1. An execution run cannot transition to completed while approval requests or external write effects remain unresolved.
2. Every retry attempt must reference a single logical effect ID and demonstrate idempotency.
3. Unexpected worker crashes must leave a durable checkpoint and route in-flight effects to a designated reconciliation owner.
4. Resuming a paused run requires verifying that model, prompt, policy, and sandbox versions remain valid.
5. Task cancellations must explicitly record whether external side-effects occurred prior to termination.
6. Compensation routines represent defined business rollback procedures, not an assumption that all real-world actions are reversible.
7. Sandbox timeouts, crash events, or cleanup failures must be explicitly logged in the run state.
8. Process completion is distinct from business acceptance; a process can terminate successfully while the business deliverable requires human revision.

Operational evaluation of runtime platforms

Enterprises evaluate runtime infrastructure by testing failure recovery directly:

Operational test	Testing protocol	Failure condition
Worker crash recovery	Terminate worker processes mid-execution and verify state resumption	Run drops from the queue, loses committed state, or duplicates an external action
Idempotent retry safety	Simulate a network timeout on an external call and trigger automated retry	Receiving system generates duplicate transactions or cannot confirm status
Human approval persistence	Pause execution for human sign-off, restart the host, and verify state	Approval scope widens, expires silently, or executes without human authorization
Cancellation integrity	Issue cancellation commands during model generation and sandbox execution	Execution continues in the background, or processes close without recording effect status
Auditability	Reconstruct a failed execution run using only state transition logs and receipts	IT operators cannot determine why a failure occurred without vendor support
Migration portability	Export workflow state, checkpoints, and intent ledgers to an alternate runtime	Export loses execution lineage, receipts, or state history required to reconcile

The trajectory of execution infrastructure

Execution infrastructure is maturing from experimental developer libraries into hardened enterprise control planes. Frameworks, workflow engines, and sandboxes will continue to coexist because they solve distinct operational boundaries: frameworks structure reasoning, workflow engines ensure state persistence, and sandboxes provide execution containment.

The decisive competitive advantage will belong to platforms that seamlessly reconcile internal workflow checkpoints with external real-world actions. Establishing what an external system committed following a network partition is the foundational requirement for autonomous commerce.

External actions inevitably cross into third-party software environments. Chapter 7 examines this external boundary: the tools, web browsers, and API protocols through which an agent interacts with external services, and the security boundaries that govern that reach.

Sources and method

• Primary sources: Technical documentation, API reference guides, and architectural specifications from OpenAI, Google Cloud, Microsoft Azure, Temporal, Prefect, Dagster, LangChain, E2B, Modal, Daytona, and Cloudflare checked as of 13 September 2026.
• Analytical frameworks: The Agent Run-State Machine, the double-durability seam, and the Agent Run Contract are original analytical structures developed to evaluate autonomous execution reliability.
• Evidence boundaries: Provider documentation establishes declared feature sets, state APIs, and commercial terms. Production fault tolerance, duplicate action frequency, and recovery latency require empirical testing under simulated network and node failures.
• Institutional references: Cyber-security guidance from the Australian Signals Directorate (ASD) establishes operational baselines for securing agentic AI harnesses.
• Editorial record: Published by Agentic Economy, Perth, Western Australia. Updates and corrections are logged via the editorial record.

Back to the report hub · Next: tools, browsers and service access.