4. Agent Orchestration & Workflows
From linear pipelines to manager-led teams: how to design predictable, efficient and recoverable workflows for multi-agent systems.
Defining several agents is only the beginning. You must also decide when each agent runs, what information it receives, which tools it may use, how its output is validated and what happens when it fails.
Without an explicit workflow, agents can duplicate work, act on incomplete information or repeatedly delegate tasks to one another. The solution is not merely better prompting. The application needs a clear orchestration architecture: a controlled set of states, transitions, responsibilities and stopping conditions.
Sequential and hierarchical workflows are two important patterns, but they are not the only choices. Modern agent systems also use routing, parallel execution, evaluator loops, hand-offs and human approval gates. A reliable application often combines several of these patterns.
➡️ Sequential Workflow
A sequential workflow is a linear pipeline. One task produces an output that becomes context for a later task. For example, a researcher creates an evidence brief, a writer drafts an article from it, and an editor reviews the draft.
Best for: Predictable processes with clear dependencies, such as Researcher → Writer → Editor or Extractor → Analyst → Report Generator.
Advantages: Easy to understand, test, trace and estimate. Each stage can have a defined input and expected output.
Drawback: A purely linear pipeline cannot adapt unless correction routes are deliberately added. If an output is incomplete, later stages need a validation failure, retry rule or route back to the responsible task.
“`
👑 Hierarchical Workflow
A hierarchical workflow introduces a manager or orchestrator. It interprets the overall objective, selects workers, delegates subtasks and evaluates their results before deciding what should happen next.
Best for: Complex or partly undefined work where the required subtasks cannot be fully predicted in advance—for example, investigating a fault across several systems.
Advantages: Flexible delegation, dynamic planning and the ability to request additional work when initial results are inadequate.
Drawback: More model calls, greater latency and less predictable execution. A weak manager can create unnecessary tasks, overlook errors or repeatedly send work back without making progress.
“`
Workflows Versus Agents
A workflow follows paths defined by application code. A model may generate content or make a limited classification at individual steps, but the permitted routes are known in advance. An agent has more freedom to decide which tool to call or which step to perform next.
This is a spectrum rather than a strict divide. A workflow can contain an agentic research step, while an agent can operate inside a graph that limits its available actions. In production systems, this hybrid approach is often the most dependable: ordinary code controls permissions and business rules, while models handle tasks that genuinely require language understanding or flexible reasoning.
Sequential Workflows in Practice
A sequential pipeline should pass well-defined outputs rather than entire unfiltered transcripts. The researcher might return a structured object containing claims, source links, publication dates and confidence notes. The writer receives this object rather than every search result and internal message generated during research.
In CrewAI, tasks can declare other tasks as context. This creates explicit dependencies: a downstream task waits for the required upstream output and receives it as input. Framework support makes the hand-off convenient, but the developer must still decide which outputs should be passed and how they should be formatted.
✅ A Safer Sequential Pipeline
- Research: Gather evidence and return a structured, cited brief.
- Validate: Check that required fields and sources are present.
- Write: Produce a draft using only the approved evidence.
- Review: Test the draft against factual and editorial criteria.
- Revise: Return specific failures to the appropriate earlier stage.
- Approve: Require human confirmation before publication if the content is consequential.
A sequential workflow therefore does not have to be incapable of moving backwards. The limitation applies only to a strictly linear implementation. A graph, Flow or ordinary control loop can route a failed draft back to the writer, or send a missing-evidence request back to the researcher.
The important safeguard is to define a retry limit. Without one, an evaluator and writer may exchange revisions indefinitely while consuming tokens without improving the output.
Hierarchical Workflows in Practice
In a hierarchical system, the manager first interprets the objective and decides how work should be divided. It may delegate research to one agent, implementation to another and testing to a third. It then reviews the results and determines whether the task is complete.
The manager does not have to be tool-free. In some designs, it only plans and delegates; in others, it may inspect project state, query progress or use validation tools. Its permissions should be determined by necessity rather than by the title “manager.” A coordinator rarely needs the same write and execution permissions as every worker.
CrewAI’s hierarchical process requires a manager model or a custom manager agent. The manager coordinates task assignment and validates results. Because this process is model-driven, its decisions should be traced and bounded with limits on delegation depth, iteration count, execution time and cost.
👑 Responsibilities of a Good Manager Agent
- Translate the overall objective into distinct, testable deliverables.
- Select workers according to their tools, context and permissions.
- Avoid creating duplicate or unnecessary assignments.
- Define an expected output and acceptance criteria for every delegated task.
- Combine results without discarding disagreements or uncertainty.
- Escalate unresolved decisions to a human instead of looping indefinitely.
Other Useful Orchestration Patterns
🔀 Routing
A router examines an incoming request and sends it to the most appropriate agent or workflow. Routing can be implemented with rules, a classifier or an LLM. It is useful when requests belong to distinct domains, such as billing, technical support and account access.
“`
⚡ Parallel Execution
Independent tasks run at the same time and their results are combined later. For example, several research agents could examine different sources or markets concurrently. Parallelism reduces elapsed time but requires conflict handling and suitable rate limits.
🧪 Evaluator–Optimizer
One component generates an output while another evaluates it against explicit criteria. Failed outputs return for revision until they pass or reach a retry limit. Deterministic validators should be used for requirements that code can check reliably.
🤝 Hand-off
One agent transfers control to another rather than simply returning a result. This is useful in conversational systems where responsibility changes during the interaction, but it requires clear rules about context, authority and when control returns.
🌐 Orchestrator–Worker
An orchestrator creates subtasks dynamically, workers execute them independently, and a final stage synthesises their results. This resembles a hierarchy but is often designed as an explicit graph rather than an open-ended management conversation.
✋ Human Approval
The workflow pauses at a defined checkpoint and requests a decision from a person. Approval gates are appropriate before publishing, sending external communications, modifying important data or performing irreversible actions.
“`
State, Context and Shared Memory
State is the structured record of the workflow’s current position and relevant data. It may include completed tasks, validated outputs, outstanding errors, retry counts, approvals and references to external artifacts.
Context is the portion of that information supplied to a model for a particular call. An agent does not need—and usually should not receive—the entire workflow state. Restricting context reduces token use, limits distraction and helps prevent sensitive information from spreading between roles.
Memory refers to information retained for later retrieval, potentially across separate runs or conversations. Memory may contain user preferences, project decisions or previously learned facts, but it should not replace authoritative application state.
🧠 State Is Not the Same as Memory
- State: “The research task is complete and the draft is awaiting review.”
- Context: “Here is the approved research brief needed for this writing task.”
- Memory: “This client prefers concise reports written in British English.”
- Artifact: “The full dataset is stored at this controlled location.”
Frameworks can help transport task outputs and persist state, but they do not automatically determine what every agent should know. In CrewAI, dependencies can be declared through task context; in graph-based frameworks, state fields and edges control the hand-offs. The developer remains responsible for data scope, validation and access policy.
Using a Shared Scratchpad
A shared scratchpad can allow agents to exchange intermediate findings without passing everything through a manager. It might be implemented as structured workflow state, a database table, a document store or a message queue.
A single JSON file can be adequate for a small local prototype, but it becomes fragile when several agents write simultaneously. Concurrent updates can overwrite one another, partially written data can corrupt the file and there may be no reliable history of who changed what.
For a more robust system, use a transactional database or state store with identifiers, timestamps, ownership and status fields. Prefer append-only events or versioned records when agents may update related information concurrently.
📝 A Useful Shared Record Might Include
- A unique task or finding identifier.
- The agent responsible for creating the entry.
- A typed status such as
proposed,verified,rejectedorcomplete. - The evidence or artifact supporting the entry.
- A creation time and version number.
- Any dependencies, reviewer comments or unresolved questions.
Do not treat scratchpad contents as automatically true. Agents can store incorrect conclusions just as easily as correct ones. Important entries should pass validation before other agents use them as trusted input.
Choosing the Right Architecture
- Choose sequential execution when the stages and dependencies are known in advance.
- Choose routing when different request types require different specialists.
- Choose parallel execution when subtasks are independent and can be combined safely.
- Choose evaluator loops when outputs can be tested against clear acceptance criteria.
- Choose hierarchical delegation when the task decomposition must be determined dynamically.
- Choose human approval when the next action is consequential, ambiguous or difficult to reverse.
Many effective systems combine these patterns. A router might select a workflow, which launches several researchers in parallel, passes their results through deterministic validation, sends the combined evidence to a writer and pauses for human approval before publication.
Designing for Failure
Every workflow needs defined failure behaviour. Set maximum iterations, task timeouts and spending limits. Record which actions have completed, make external operations idempotent where possible and avoid blindly retrying messages, payments, uploads or destructive commands.
Use deterministic code for schema validation, calculations, permissions and fixed business rules. Use model-based evaluation for qualities that are genuinely subjective, such as tone or clarity, and remember that an LLM reviewer can make mistakes.
Finally, trace each transition: which agent ran, what context it received, which tools it called, what it returned and why the workflow selected the next step. Good orchestration is not about simulating an office hierarchy. It is about turning uncertain model behaviour into a bounded, observable and testable software process.