Barry Zhang of Anthropic took the stage at the AI Engineer Summit in New York and opened with a line that rarely comes from someone building AI agents. “Don’t build agents for everything.” His presentation, “How We Build Effective Agents”, quickly became one of the most widely shared practical frameworks for debating when agents should be built at all. The point goes to a question that teams usually do not ask. When is an agent actually the right answer?
Yes means ambiguity plus value
Zhang says that agents prove themselves where a decision tree cannot reasonably be drawn in advance. When a task has ambiguous input, an open space of variants and value that lies in the number of paths that can be taken, an agent makes sense. For this to hold, the output must be worth the cost. “That exploration I just mentioned is going to cost you a lot of tokens.” For a 10-cent customer support task, this does not pay off. For coding, it does. Coding comes out in this framework as a textbook yes category: high ambiguity, high output value and a result that can be verified with unit tests.
No is a workflow that can be described in advance
Where the decision tree can be mapped in advance, an agent loses its value. “It’s a lot more cost-effective, and it’s going to give you a lot more control.” When you know the inputs, outputs, branching and edge cases, a programmable workflow costs a fraction of the tokens. The same applies to tasks where an error is costly and hard to find. A read-only approach or human oversight helps here, but “this will also limit how well you’re able to scale your agent in your use case”. At that point you have built an agent that cannot scale, which undermines its purpose.
Maybe is about what the agent actually sees
The hardest part comes at the end of the presentation. Zhang says that most teams trip up here. “think like your agents.” The agent works only with the limited context you give it. Anything you do not put into it, or do not make accessible to a tool, practically does not exist in the agent’s world. Meanwhile, a developer looks at a task with dozens of open tabs, documentation, Slack history and product knowledge in their head. The agent gets a prompt, the available tools and limited context.
A simple analogy from the video “exact instructions challenge” helps here. Children write instructions for making a peanut butter and jam sandwich, and an adult follows them literally. If the instructions do not say “open the bag of bread”, “take one slice” or “open the jar”, the adult does not take those steps. They understand the goal, but the instructions do not contain it.
It is the same with agents. The problem often lies in the brief, the context and observability. Zhang therefore recommends simulating the task from the agent’s perspective. Open Claude, send it exactly the prompt the agent receives in production, and watch what it does with it. Most gaps appear within five minutes.
The five-minute test that most people skip
Most non-tech CEOs read “build an agent for this” as an unambiguous yes and send the brief to the team. Zhang warns against exactly this move. An agent can only become genuinely usable after the team has gone through the maybe phase. In other words, it has accepted the agent’s constraints as part of the product reality. This is not a technical detail.
It helps to think of agents as a new job role. A short video in which a manager stands in front of a “team”, congratulates it on revenue growth and starts the next month within two minutes captures this change of perspective exactly. If an agent is to work as a team member, you have to hire it for a specific job. Simply “switching it on” is not enough. Ethan Mollick of Wharton works with the analogy of an intern whom you have to look after until they understand your context.
With a person you would ask what exactly they are supposed to do, what they need to know, which tools they will get, when they should escalate a problem, how we will recognise a good result, and how much their time costs. The same applies to an agent. Instead of an employment contract you write a prompt, instead of onboarding you assemble context, instead of access you set up tools, and instead of a performance review you monitor traces, costs, errors and success rate.
Zhang’s “think like your agents” is in fact a hiring test. First you check whether the work is suitable for an agent at all. Then you test whether it has enough information and tools. Only then does it make sense to decide whether it should run in production.
Yes, no, maybe is an economic model. It has nothing to do with whether someone likes AI. Yes means complexity plus value. No applies to tasks that can be converted into a workflow in advance. Maybe lies in those five minutes when a developer opens Claude and sends it exactly what their agent receives in production. Zhang concludes the presentation with a combination of these questions. A number of apparent yes answers turn into no after the five-minute test.
The video with the peanut butter and jam sandwich shows the same principle in its childhood version. When instructions contain no context, the system carries them out literally. An agent does the same. Zhang’s presentation can be read as a recruitment manual for agents. The question stops being whether an agent can be built. It becomes whether you would hire a person for that work, how you would onboard them, how you would measure their performance and when you would not grant them autonomy. An agent without context works as a tool that guesses.
We are used to people working a lot out for themselves. They grew up in the same city, went to a similar school and know the company for a few months, so something feels “obvious” and “normal” to them. An agent has no such implicit shared context. It does not have the same city, the same school or the same assumptions about what is usual. The greatest skill when working with agents is therefore not to make assumptions, and to give the agent the same things you would give a person. Role definition, context, tools, boundaries, evaluation. With an agent you see this immediately, because if you do not do it, it will not do it either. With a person you often overlook it until something fails.
Comments