Agent Engineering
Agent systems are commonly assembled framework first, which settles the architecture for a year or more before memory, retrieval, guardrails and the loop have been designed.
A production agent has six layers: the surface other systems call, the orchestration and runtime that carries a task, memory, retrieval, the tool surface, and the model underneath. Two concerns cut across all six, namely observability with evaluation, and governance with security. Agent failures are predominantly state-management failures rather than prompt failures, so a stack chosen framework first commits to an architecture before the layers that decide its behaviour exist. This engagement designs those layers against your data and your constraints, then builds the system on a specification that is not bound to any single framework.
- Your product: the thing users actually asked for
- Context and memory: context, memory, search, graph
- Multi-agent systems: loop, agentic, code
- Evals and guardrails: eval, safety, red teaming
- Model and runtime: swappable, and rarely the reason a system fails
Portable
One specification compiles to eight frameworks, so the choice stays reversible.
Evidence
Every recommendation traces to a measurement you can rerun.
Handover
The architecture, the instrumentation and the reasoning are yours to keep.
Where it applies
A system being built from the ground up, or one where the framework was chosen before the layers underneath it were designed.
What we design
The six layers and the two rails that cut across them, against your data, retention and access limits.
What you keep
A specification that compiles to eight frameworks, the pipelines it generates, and the instrumentation behind them.
Fit
Where this fits
Architecture decisions that outlast the model you picked, often under security or audit review.
Platform teams
Coding agents rolled out, architecture still inconsistent
Engineering leaders
Agent spend that needs a measurement behind it
Product teams
Systems that demo well and drift under traffic
Large codebases
Repositories that exceed any single context window
Scope
Where agent engineering goes wrong, and how we address each
The layers below decide how an agent behaves under traffic. Each is designed against your data and constraints rather than assembled from defaults, and every component is open source we maintain and you keep.
The six layers
Observability with evaluation, and governance with security, cut across all six rather than sitting among them.
- 01Surfacewhat other systems call
- 02Orchestrationthe loop that carries a task
- 03Memorywhat survives a call
- 04Knowledgeretrieval scored on use
- 05Toolsthe callable surface
- 06Modelassignment per role
| The problem | How we address it |
|---|---|
| Framework chosen before the architecture | The agent is declared once in SuperSpec and compiled to framework-native code across DSPy, OpenAI Agents SDK, Claude Agent SDK, CrewAI, Google ADK, Pydantic AI, DeepAgents and Microsoft. The specification is the asset and the framework is a build target, so the decision stays reversible. |
| State failures treated as prompt failures | Context assembly, memory and retrieval are designed as one layer with explicit ranking and summarization, so what reaches each call is a decision rather than an accumulation. |
| Memory assembled ad hoc on a vector store | A memory layer designed against your retention, access and compliance limits, with ranking and summarization tuned and the window budget derived from measurement. |
| Retrieval added late, scored in isolation | A knowledge layer scored on what the agent actually reads, across ChromaDB, Qdrant, Milvus, Weaviate, LanceDB or SurrealDB depending on what you already run. |
| Guardrails wrapped on at runtime | Guardrails and assertions are declared in the specification and compiled into the generated pipeline, so they exist in the code that runs rather than in a layer someone can bypass. |
| Loop chosen by habit, not by task shape | The runtime and control flow are selected from your task profile and data, covering single-pass work, long-running tasks that must survive interruption, and event-driven triggers. |
| One agent grown until nothing is separable | Responsibilities are split across agents with declared interfaces and an orchestration layer between them, so each part can be measured, replaced and scaled on its own. |
| Model chosen once, never revisited | Model assignment is set per role with local and hosted options evaluated on the same task set, so cost and capability are matched to the step rather than to the project. |
| Not instrumented, so not measurable later | Traces, tool calls and decisions are recorded in a form an evaluation can compute over, through MLflow, LangFuse, Logfire, Weights and Biases or the built-in tracer. |
| Improvement depends on who built it | GEPA optimization is built in from the start, so the agent improves against a scored task set rather than by hand, and the routine survives any individual. |
| No record of what was checked | An Agent Quality Record states what was measured, what the agent was permitted to do and who accepted the result, in an open format any conformant tool can read. |
Process
How we work
Design against your data
We map the task shapes, the data available, and the retention, access and compliance limits that narrow the options. The loop, the memory model and the orchestration follow from that rather than from a framework preference.
Declare the system
The agent, its tools, its memory and its guardrails are written as one SuperSpec specification, which compiles to framework-native code on any of the eight supported runtimes.
Build and instrument
Pipelines generated in the framework you run, with traces, tool calls and decisions recorded from the first commit so evaluation never needs a retrofit.
Optimize and hand over
GEPA and the DSPy optimizers tune the system against a scored task set, the release decision is recorded, and the specification, the pipelines and the runner stay with your team.
Deliverables
What you receive
Scope is set from your requirements rather than from a fixed package. Whatever is agreed at the outset is designed against your data, built on a specification you own, instrumented so it can be measured, and handed over on open source your team keeps.
Stack scorecard
Every layer scored, each finding tied to the change that fixes it
Instrumentation plan
Traces, tool calls and decisions captured in a form evaluation can read
Orchestration design
Roles, handoffs and recovery behaviour, with the reasoning attached
Memory decision
Persistence, retention and access, with the reasoning attached
Scope
Engagement Options
Twelve disciplines, grouped into three columns you can scope and buy independently. Protocol surfaces sit under Agent Experience, improvement against an established baseline under Agent Optimization, and evaluation under Agent Quality. Scope and cost are quoted against your systems once we have seen them.
Typical starting point
Context & Memory
What the agent knows
Typically 4 to 8 weeks
Architecture and build
Context assembly, persistent memory, and the retrieval and search underneath.
- Context and memory engineering
- Retrieval, search and graph design
- Persistence and retention policy
- Implementation into your stack
- Written decision record
Multi-Agent Systems
How the work is divided
Typically 6 to 8 weeks
Design and orchestration
Roles, handoffs, loops and recovery when several agents share the work.
- Loop and agentic engineering
- Harness engineering and control flow
- Recursive routing for large codebases
- Failure and recovery behaviour
- Observability across the system
Observability & Instrumentation
How the system becomes measurable
Typically 4 to 6 weeks
Instrumentation and control surfaces
Making the system evaluatable by design: traces, tool-call records and permission surfaces a later evaluation can compute over.
- Trace and event instrumentation, OpenTelemetry compatible
- Tool-call and decision capture per role
- Permission and policy surfaces made explicit
- Cost and token attribution per task
- Replayable sessions for later evaluation
Open source
Underlying systems
Everything below is ours and open. Engagements run on the stack you already have.
SpecMem
Cognitive memory any coding agent can read.
AgentVectorDB
Vector database for agent memory and retrieval.
PyFlue
Python-native agent harness framework.
SuperClaw
Scenario-driven security testing.
SuperOptiX
Agent framework across the main runtimes.
Agent Engineering
The twelve disciplines behind this practice.
Questions
Questions
Discuss Agent Engineering
Send us what you are building and the constraints around it. We come back with scope.
London and San Francisco. Remote or on site.