1. Harnessing an Agent Framework
Why build every part of an agent loop yourself? Agent frameworks provide reusable components for connecting models to tools, managing state, controlling execution and handling failures.
At its simplest, an AI agent is a program that repeatedly asks a language model what to do next. You can build one using standard Python and an HTTP client such as requests or httpx: send the model a task, inspect its response, execute any requested function and return the result for another round of reasoning.
For a small experiment, this approach is entirely reasonable. Modern models and local model servers can also support structured tool calling, so you should not normally need to search free-form text for JSON with regular expressions. Where structured tool calls are unavailable, you can request a defined JSON response and validate it with a schema library such as Pydantic.
The difficulty appears as the application grows. Production agents need more than a basic loop: they must validate tool arguments, recover from malformed responses, enforce iteration limits, preserve state, control context size, retry transient failures, record execution traces and prevent unsafe actions. Parallel tools, human approval steps and long-running tasks add still more orchestration code.
This is the problem agent frameworks are designed to solve.
What Does an Agent Framework Do?
An agent framework is a library or runtime that supplies reusable building blocks for model-driven applications. Popular options include LangChain and LangGraph, LlamaIndex and its Workflows system, and CrewAI. They overlap, but they are not interchangeable: LangChain offers broad model and tool integrations, LangGraph focuses on explicit stateful workflows, LlamaIndex is particularly strong in data and retrieval applications, and CrewAI emphasises role-based teams of agents and event-driven flows.
🛠️ The Core Abstractions
- Model Integration: Frameworks provide a common interface for hosted APIs and, where supported by the relevant connector, local model servers such as Ollama. This makes it easier to change models without rewriting the entire application. Compatibility still depends on the selected model: not every local model follows tool schemas reliably or supports every framework feature.
- Tool Definition and Binding: A Python function can be exposed as a tool using type annotations, a docstring and, in many frameworks, a decorator. The framework converts this information into a structured schema that tells the model the tool’s name, purpose and accepted arguments. Explicit descriptions and strict input models are still preferable to relying on an automatically generated schema.
- Agent or Tool-Calling Loop: The runtime sends the current state to the model, detects any tool calls, validates their arguments, executes the appropriate functions and returns the results to the model. It continues until the model produces a final response or the application reaches a configured stopping condition. This resembles the classic ReAct pattern, but modern agents often use native tool-calling messages rather than writing visible “thought” and “action” text.
- State and Persistence: Frameworks can track messages, tool results, intermediate values and application-specific data. Checkpoints allow a workflow to pause and resume, while persistent storage can preserve selected information between sessions. State is broader than chat history and should be designed deliberately.
- Context Management: Long conversations cannot simply grow forever. Frameworks can help trim messages, summarise older exchanges, retrieve only relevant information or move durable facts into external storage. These strategies reduce token use and help keep the prompt within the model’s context window; they do not automatically guarantee that important information will be preserved.
- Workflow Control: Graphs and flows let developers define branches, retries, timeouts, parallel steps, approval gates and deterministic code paths. This is useful when some parts of a process should be controlled by ordinary software rather than left to the model’s judgement.
- Validation, Guardrails and Observability: Mature frameworks can validate structured outputs, restrict tool access and record prompts, responses, timings, errors and state transitions. These traces are essential for understanding why an agent selected a tool or failed to complete a task.
Memory Is Not Automatic Understanding
The word memory covers several different mechanisms. Short-term memory usually means the messages and state associated with one conversation or workflow. Long-term memory means information stored outside the immediate prompt—perhaps in a database, document store or vector index—and retrieved in a later session.
A framework can provide the storage and retrieval machinery, but it cannot decide perfectly what should be remembered. Summaries may omit details, retrieval may return irrelevant material and stored information may become outdated. Developers therefore need retention rules, user controls and appropriate safeguards for personal or sensitive data.
The Trade-off: Abstraction Versus Control
Frameworks accelerate development, but every abstraction introduces another layer to understand. A high-level agent may be quick to assemble yet difficult to debug if prompts, tool schemas and state transitions are hidden behind library internals. Framework APIs also evolve rapidly, which can complicate upgrades and make older tutorials inaccurate.
The most useful distinction is not simply between “heavy” and “light” frameworks. It is between open-ended agents, in which the model decides the next action, and explicit workflows, in which code defines the permitted routes. For predictable business processes, a graph or flow with a few model-powered steps is often easier to test and operate than a fully autonomous agent. Multi-agent designs should be introduced only when separate roles or independent contexts provide a clear benefit; adding more agents also increases cost, latency and failure points.
Choosing the Right Level of Abstraction
A hand-written loop can be the best choice for a prototype with one model and a small number of tools. A general agent framework becomes valuable when you need integrations, structured state, retries, tracing or rapid experimentation. A graph-based or event-driven runtime is better suited to long-running jobs, branching processes and human approval. Multi-agent orchestration is most appropriate when a task genuinely benefits from specialised roles, delegation or parallel work.
Whichever approach you choose, keep tools small, typed and independently testable. Set limits on iterations, execution time and resource use; log every tool call; require confirmation before consequential actions; and test with malformed inputs and tool failures. A framework removes boilerplate, but reliability still comes from careful application design.