AI Agents: Cutting Through the Noise to What Actually Works

So What Actually Makes Something an Agent?

Let's be honest — "AI agent" has become one of the most abused terms in tech. Everyone slaps it on everything. But at its core, an agent is pretty straightforward: it's a system that looks at its environment, figures out what to do, takes action, checks the results, and keeps going until the job is done. That's fundamentally different from a chatbot that just fires off a single response and calls it a day.

I've found that three things separate a real agent from a glorified API wrapper. First, autonomy — it decides what to do without you holding its hand through every step. Second, tool use — it can reach out to search engines, databases, code interpreters, APIs, whatever it needs. Third, planning — it breaks a big goal into smaller pieces and adjusts the plan as it learns more. If any of these are missing, you've got a chain, a pipeline, or maybe an assistant with tool access. Not an agent.

This distinction isn't just pedantic. The way you build, test, and debug these systems changes dramatically depending on which category you're actually in.

An agent isn't defined by the model powering it — it's defined by the loop it runs. A GPT-4-class model answering one question? Not an agent. A smaller model iterating through observations and actions to get something done? That's an agent.

Architecture Patterns That Actually Matter

Several patterns have emerged for building agents, and each makes different trade-offs between reliability, speed, and complexity. Picking the right one can make or break your project, so let's walk through the big three.

ReAct (Reason + Act)

ReAct is the bread and butter of agent architectures. It weaves reasoning and action into a single loop: think, pick an action, run it, look at the result, repeat. It keeps going until the model decides it has enough to give you a final answer. The beauty of ReAct is its simplicity — it's easy to build, easy to debug (you can literally read the model's reasoning), and it handles most tasks that need fewer than ten steps.

Here's the thing, though: ReAct only plans one step ahead. For complex tasks that need coordination across many subtasks, the agent can drift off course and start doing irrelevant things. Error recovery is also pretty fragile. If a tool call blows up, the model has to think its way out without any structured way to backtrack.

Plan-and-Execute

Plan-and-Execute takes a different approach by splitting planning and execution into two separate phases. A planner creates a structured step-by-step plan upfront, and an executor works through each step one at a time. After each step, the planner can revise the remaining steps based on what it's learned so far. You get more coherent multi-step behavior, and you can see the full plan before anything actually runs — which is great for debugging and monitoring.

The downside? More latency and more tokens. That planning phase adds an extra LLM call at the start, and potentially after every execution step too. It also assumes you can decompose the task upfront, which doesn't always work for exploratory tasks where you don't fully know the goal yet.

LATS (Language Agent Tree Search)

LATS brings Monte Carlo Tree Search ideas into the agent world. Instead of following one path, it explores multiple branches in parallel, scores intermediate states (usually with another LLM call), and backtracks when a branch looks like a dead end. For tasks where the first move is uncertain — think complex coding problems or multi-hop research — this approach can dramatically improve results.

But it comes at a serious cost. We're talking 5x to 20x more LLM calls compared to ReAct or Plan-and-Execute, plus you need careful engineering of the scoring function and branching logic. Use LATS when getting the right answer matters way more than speed or cost.

Tool Use and Function Calling

Tool use is what turns a language model from a fancy text generator into something that can actually do things in the real world. Modern LLMs handle function calling natively: you give the model a schema of available tools, it decides when to call one, and it generates structured arguments that match the schema. Your runtime executes the function and feeds the result back into the conversation.

In practice, tool design matters just as much as model selection — maybe more. Each tool should do one thing well, with a clear name and description. Keep your input schemas tight: use enums and constrained types instead of free-form strings to cut down on hallucinated arguments. And keep outputs concise — returning a 10,000-token document when a 200-token summary would do just wastes context space and muddies downstream reasoning.

from typing import Any
import json
import httpx

# Define tools as structured schemas for the LLM
tools = [
    {
        "type": "function",
        "function": {
            "name": "search_knowledge_base",
            "description": "Search the internal knowledge base for relevant documents. Use this when the user asks about company policies, procedures, or technical documentation.",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {
                        "type": "string",
                        "description": "The search query in natural language"
                    },
                    "max_results": {
                        "type": "integer",
                        "default": 5,
                        "description": "Maximum number of results to return"
                    },
                    "filters": {
                        "type": "object",
                        "properties": {
                            "category": {
                                "type": "string",
                                "enum": ["policy", "technical", "hr", "finance"]
                            },
                            "date_after": {
                                "type": "string",
                                "format": "date"
                            }
                        }
                    }
                },
                "required": ["query"]
            }
        }
    }
]

# Agent loop with tool execution
async def agent_loop(client, messages: list[dict], max_iterations: int = 10):
    """Run the agent loop with tool calling until completion."""
    for _ in range(max_iterations):
        response = await client.chat.completions.create(
            model="gpt-4o",
            messages=messages,
            tools=tools,
            tool_choice="auto"
        )

        message = response.choices[0].message
        messages.append(message.model_dump())

        # If no tool calls, the agent is done
        if not message.tool_calls:
            return message.content

        # Execute each tool call and append results
        for tool_call in message.tool_calls:
            result = await execute_tool(
                tool_call.function.name,
                json.loads(tool_call.function.arguments)
            )
            messages.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": json.dumps(result)
            })

    raise RuntimeError("Agent exceeded maximum iterations")

One thing that catches people off guard: even with strict schemas, models sometimes generate arguments that are technically valid JSON but semantically wrong. Think a future date for a historical query, or a filter value that matches zero results. Adding a lightweight validation layer between the model's output and actual tool execution saves you a lot of headaches — and gives the model useful error messages to course-correct.

Memory Systems

Without memory, your agent is stuck with whatever fits in a single context window. That's fine for quick tasks, but if you're building something that runs for a while, spans multiple sessions, or needs to accumulate knowledge over time, you need structured memory. Three types have proven their worth in practice.

Short-Term (Working) Memory

This is basically the conversation history and scratchpad within a single agent run — the message list you're passing to the LLM. The main challenge is managing the context window: as the conversation grows, you need to summarize or drop older messages to stay within token limits. A sliding window with periodic summarization of older turns is the most common approach, and it works well enough for most use cases.

Long-Term (Semantic) Memory

Long-term memory sticks around between sessions, usually backed by a vector database. After each task, the agent pulls out key facts, decisions, and outcomes and stores them as embeddings. Next time a relevant task comes up, those memories get retrieved and injected into the prompt. This lets the agent learn from experience without retraining. The tricky part is curation — without proper deduplication and relevance filtering, your memory store fills up with noise that actually makes retrieval worse over time.

Episodic Memory

Episodic memory captures full trajectories of past runs — every thought, action, and observation from start to finish. When the agent hits a similar task, it pulls up relevant episodes and uses them as few-shot examples. This works especially well for tasks with clear success and failure patterns, since the agent can learn both what worked and what didn't.

from dataclasses import dataclass, field
from datetime import datetime

@dataclass
class MemoryEntry:
    content: str
    embedding: list[float]
    timestamp: datetime
    source: str  # "conversation", "tool_result", "reflection"
    importance: float = 0.5  # 0.0 to 1.0, set by the agent
    access_count: int = 0

class AgentMemory:
    """A hybrid memory system combining short-term and long-term storage."""

    def __init__(self, vector_store, max_short_term: int = 20):
        self.short_term: list[dict] = []
        self.vector_store = vector_store
        self.max_short_term = max_short_term

    def add_to_short_term(self, message: dict):
        self.short_term.append(message)
        if len(self.short_term) > self.max_short_term:
            self._summarize_and_archive()

    async def retrieve_relevant(self, query: str, k: int = 5) -> list[MemoryEntry]:
        """Retrieve the most relevant long-term memories for a query."""
        results = await self.vector_store.similarity_search(query, k=k)
        # Boost recently accessed and high-importance memories
        scored = []
        for entry in results:
            recency = 1.0 / (1.0 + (datetime.now() - entry.timestamp).days)
            score = 0.5 * entry.importance + 0.3 * recency + 0.2 * entry.access_count / 100
            entry.access_count += 1
            scored.append((score, entry))
        scored.sort(key=lambda x: x[0], reverse=True)
        return [entry for _, entry in scored[:k]]

    def _summarize_and_archive(self):
        """Summarize oldest short-term messages and move to long-term."""
        to_archive = self.short_term[:5]
        self.short_term = self.short_term[5:]
        # In production, call the LLM to summarize before archiving
        for msg in to_archive:
            self.vector_store.add(msg["content"], source="conversation")

Multi-Agent Orchestration

Once tasks get complex enough, a single agent starts losing the thread. Multi-agent orchestration tackles this by spreading the work across specialized agents, each with its own tools, system prompt, and expertise. You'll typically see two patterns: supervisor architectures where a coordinator delegates to workers, and peer-to-peer setups where agents talk directly through shared channels.

For production systems, the supervisor pattern is your best bet. A top-level agent takes the user request, breaks it into subtasks, hands each one to a specialist, collects results, and puts together the final response. Each specialist can be tested, versioned, and optimized independently. And if one fails, the supervisor can retry, route to a backup, or return a partial result — your whole system doesn't go down.

Peer-to-peer architectures sound great in theory — more flexible, more dynamic. In practice? They're a coordination nightmare. Without a central coordinator, agents can get stuck in infinite delegation loops, produce contradictory outputs, or deadlock waiting for each other. You end up needing so many guardrails — message limits, timeouts, conflict resolution — that you've basically reinvented the supervisor pattern with extra steps.

The Limitations Nobody Wants to Talk About

The demos are impressive. The reality is... messier. The biggest issue is reliability. Even top-tier models make reasoning mistakes, hallucinate tool arguments, and sometimes just ignore your instructions. Here's a sobering thought: for a ten-step task where each step succeeds 95% of the time, your odds of completing the whole thing without an error are only 60%. At twenty steps? Below 36%. That math is brutal.

Then there's latency. Every agent step means at least one LLM call, often two or three when you factor in processing tool results. A ten-step workflow with typical 1-3 second API response times easily takes 20-30 seconds end-to-end. Users who expect sub-second responses won't tolerate that for interactive use cases, which pushes agents toward background task patterns rather than real-time conversations.

Cost adds up fast too. Long-running agents that accumulate tool results can chew through 100,000+ tokens per task. At current API prices, one complex task can cost several dollars. That's fine for high-value enterprise workflows, but it's a non-starter for consumer apps at scale.

Evaluation is still an open problem. Traditional testing assumes deterministic behavior, but agents are inherently stochastic — the same input can produce different action sequences across runs. That makes reliable regression testing really hard. The best approach we've found is evaluating on outcomes (did the agent get the right answer?) combined with trajectory-level metrics like step count, tool usage patterns, and reasoning quality.

And then there's security — the elephant in the room. An agent with tool access can, by definition, take real actions in the real world. Prompt injection attacks, where malicious content in tool results tricks the agent into doing something unintended, are a genuine threat. Defense in depth — input sanitization, output validation, least-privilege tool access, human-in-the-loop for high-stakes actions — isn't optional. It's table stakes for any production deployment.

Key Takeaways

1. An agent is defined by its loop — autonomy, tool use, and planning — not by whatever model is under the hood.
2. ReAct is great for straightforward tasks. Plan-and-Execute shines with multi-step workflows. LATS is your go-to when correctness trumps speed and cost.
3. Tool design matters as much as model selection. Clear schemas, strict validation, and concise outputs make your agents significantly more reliable.
4. Memory systems — short-term, long-term, and episodic — unlock long-running and multi-session capabilities, but they need careful curation or they'll fill up with noise.
5. Multi-agent orchestration is powerful but adds real complexity. The supervisor pattern is the most battle-tested approach for production.
6. Reliability, latency, cost, and security are hard constraints. Building around them — instead of pretending they don't exist — is what separates production systems from flashy demos.
7. Evaluate agents on outcomes, not trajectories. Stochastic behavior is a feature of the paradigm, not a bug you need to squash.