Do you actually need an AI agent? Four questions before you build one

Often, you don't. An AI agent earns its cost when a task needs judgment on messy input, such as emails or documents, when the steps can't be written down in advance, and when a wrong move can be caught before it does harm. If you can list every step, plain automation is cheaper and more predictable. If the job is to find or summarize information for a person, an AI assistant is enough.

That is close to the advice of Gartner, the research and advisory firm. In June 2025, Gartner analyst Anushree Verma said organizations can start by using AI agents when decisions are needed, "automation for routine workflows and assistants for simple retrieval." The same press release carried a warning about agentic AI, meaning AI agents and the systems built around them. Gartner predicted that over 40% of agentic AI projects will be canceled by the end of 2027, due to escalating costs, unclear business value or inadequate risk controls.

Below: what an agent is, what Gartner's forecast does and doesn't show, a four-question test with one worked example, how to start safely if you do need an agent, and six questions to ask whoever builds it.

What counts as an AI agent?

An AI agent is software in which an AI model decides its own next step and picks its own tools to finish a task. Tools here are the actions the model is allowed to take in other systems, such as looking up an order or sending an email. Anthropic, the maker of the Claude models, draws this line in its guide Building effective agents, published in December 2024. In a workflow, the model and its tools follow "predefined code paths" that a developer wrote. An agent "dynamically" directs its own process and tool use. Anthropic also notes that "agent" "can be defined in several ways", so anything called an agent may be either one.

Four other kinds of software often get called agents:

• Automation: software that follows steps someone wrote down. Without AI, the same input gives the same result every time. When one step uses an AI model, for example to sort emails by topic, the path is still fixed. Anthropic calls that a workflow, a narrower meaning than the everyday word for a business process in Verma's quote above.
• Robotic process automation (RPA): scripts that repeat a person's keystrokes in other applications to move data or run transactions, as Gartner's glossary describes it. It is automation that works through the screens people use.
• Chatbot: a program that answers messages in a chat window. The window says nothing about what works behind it, which may be fixed answers, an assistant or an agent.
• AI assistant: an AI tool that finds information, summarizes it or drafts text when a person asks, and leaves the action to that person.

OpenAI, the maker of ChatGPT, draws the same line in its 2025 practical guide to building agents. Applications that "integrate LLMs but don't use them to control workflow execution", such as simple chatbots, "are not agents." An LLM, or large language model, is the kind of AI model this article is about.

The difference matters for cost and risk. Anthropic writes that agentic systems "often trade latency and cost for better task performance". In plain words, they are slower and cost more, in return for better results. An agent's autonomy also means "higher costs, and the potential for compounding errors", where a mistake in one step feeds every step after it.

Why does Gartner expect so many agent projects to be canceled?

Gartner names three causes for its forecast that over 40% of agentic AI projects will be canceled by the end of 2027: escalating costs, unclear business value or inadequate risk controls. Verma said most agentic AI projects at the time were early experiments or proofs of concept, meaning small trial builds, "that are mostly driven by hype and are often misapplied." Her conclusion is the one this article builds on: "Many use cases positioned as agentic today don't require agentic implementations."

Read the figure for what it is:

• It is a forecast. Gartner published it on June 25, 2025, about projects that were still running. It is not a count of failed projects. As of October 2026, we found no updated figure from Gartner.
• The release doesn't explain how Gartner reached 40%. It also quotes a January 2025 poll of 3,412 webinar attendees, but that poll asked about investment: 19% said their organization had made significant investments in agentic AI, and 31% were waiting or unsure. If you see the poll cited as the basis of the forecast, the release doesn't say so.
• Gartner isn't against agents. The same release predicts that at least 15% of day-to-day work decisions will be made autonomously through agentic AI by 2028, up from 0% in 2024. Its advice is narrower: pursue agentic AI only where it delivers clear value or ROI, meaning a clear return on the investment.

What is agent washing?

Agent washing is Gartner's name for the rebranding of existing products, "such as AI assistants, robotic process automation (RPA) and chatbots, without substantial agentic capabilities." In the same release, Gartner estimates that only about 130 of the thousands of agentic AI vendors are real. That is an estimate from June 2025. The release names no vendors and gives no method.

For your own project, the label tells you little. What matters is what the product decides on its own, and whether your task needs that. A relabeled chatbot isn't a bad product, and if your task only needs a chatbot, it may be the right one. The trouble starts when you budget and plan for an agent but end up with a script. It also starts when the agent label talks you out of the simpler option your task needed.

Does your task need an agent, automation or an assistant?

It depends on two things: whether you can write the steps down, and whether the software must act on its own. First split the process into tasks, as the example below splits one support inbox into three kinds of email. Then take each task through the questions below. Stop at the first "yes" among questions 1 to 3. Question 4 then applies to whatever you chose.

1. Can you write down every step and every exception? Then use automation. RPA is one kind, for work done through screens. If the input is messy, such as emails, one AI step can read and sort it while the path stays fixed. Automation costs less to run than an agent and gives the same result every time.
2. If not, is the job only to find, summarize or draft information for a person, who then acts? Then an AI assistant is enough. People may reach it through a chatbot.
3. If neither, does the task need judgment you can't write down, and actions in other systems? Then it is a candidate for an agent. Messy input, such as emails, documents or conversations, often comes with it.
4. Can a wrong action be caught and undone? If yes, the software can act on its own within set limits, once a test shows that it handles the task. If not, no such action happens without a person's decision, at least until results show that the software can be trusted with it.

A candidate is not yet a yes. Gartner advises pursuing agentic AI only where it delivers clear value. That takes enough cases, each costing your staff real time, for a tested agent to save more than it costs to build and run. For a few cases a week, the cheapest option may be a person doing the work, perhaps with an assistant's help.

The first three questions condense three sources. Verma's advice was agents "when decisions are needed, automation for routine workflows and assistants for simple retrieval." OpenAI's guide says agents fit work with complex decisions, rules that are hard to maintain, or heavy reliance on unstructured data, such as documents and conversations. It adds: "Before committing to building an agent, validate that your use case can meet these criteria clearly. Otherwise, a deterministic solution may suffice." A deterministic solution gives the same output for the same input every time. Anthropic recommends "finding the simplest solution possible", which "might mean not building agentic systems at all." The fourth question comes from OpenAI's advice on high-risk actions, covered in the next section.

An illustrative example. Picture a small online shop whose support inbox gets three kinds of email:

• "Where is my order?" The steps never change: find the order, read the tracking status, reply. That is automation. An AI model can sort incoming emails by type, but that is one AI step in a path you wrote. Anthropic gives this as its own example of a fixed workflow it calls routing: "Directing different types of customer service queries (general questions, refund requests, technical support) into different downstream processes, prompts, and tools." A support form with an order number field may need no AI at all.
• "How do I return this?" The answer is in the returns policy. An assistant can draft a reply from the policy, and a person sends it.
• "The item arrived broken, and I want a refund." Each case differs: the photos, the order history, the courier, the exceptions in the policy. OpenAI's guide uses this exact kind of judgment as its example: "refund approval in customer service workflows." It is the only one of the three emails that is a candidate for an agent. Question 4 has a clear answer here: a refund is money paid out, and it is hard to get back once sent. So no refund goes out without a person's decision.

If you do need an agent, how do you start safely?

Start with less autonomy than you think you need, and let measured results earn the next step. A Gartner press release from May 2026 describes four levels of autonomy:

1. Observe: read-only access, for tasks such as summarizing documents or finding information.
2. Advise: the agent drafts or recommends, and people review all outputs and act themselves.
3. Act with approval: the agent can write data or send messages, but only after a person approves every action.
4. Act autonomously: the agent acts on its own within set limits, and people review exceptions, logs and overall results.

Levels 1 and 2 can look like an assistant's job. The difference is who chooses the steps. An assistant answers what a person asks. An agent at level 2 decides for itself what to read and check, then reports to a person, who acts.

The same release makes a second, separate 40% forecast: by 2027, 40% of enterprises will demote or decommission autonomous AI agents. It counts companies that move such agents down a level or switch them off, not canceled projects. Gartner's cause is governance gaps, meaning missing rules and controls on what an agent may do. Companies find them only after production incidents, which are failures in real use by customers or staff. The release also warns that approvals at level 3 can degrade "under time pressure or approval fatigue", when people approve without reading. Like the 2025 figure, this is a forecast, and the release gives no method.

In the shop example, the refund agent would start at level 2, with no access to the payment system. It decides which records to open, such as the order, the courier's tracking and the customer's photos, checks them against the policy, and proposes a decision with its reasons. A person decides and issues the refund.

OpenAI's guide supports that order. It says actions that are sensitive, irreversible or high-stakes, such as "authorizing large refunds, or making payments", "should trigger human oversight until confidence in the agent's reliability grows." The same page tells teams to limit an agent's retries or actions and to hand the task to a person when it goes past those limits, for example when it still can't understand what the customer wants after several attempts. Whether the agent could later issue refunds after a person's approval (level 3), or small refunds on its own (level 4), is your decision, based on its results.

Two habits keep this honest:

• Measure against the simpler option. Anthropic's advice is to add complexity "only when it demonstrably improves outcomes". Run the agent beside your staff on the same cases for a set period. Count how often its proposal matches the staff decision, and read every case where it doesn't. Count the two kinds of mismatch apart: a refund the agent would wrongly approve costs money, while one it wrongly sends to a person costs only time. Set the pass mark before the test starts, based on what each kind of mistake costs you. None of the sources we read gives one pass mark for all tasks. If you have no technical staff, ask whoever builds the agent to run this test and show you every mismatch.
• Grow in small steps. OpenAI's guide ends with "Start small, validate with real users, and grow capabilities over time." Anthropic recommends "extensive testing in sandboxed environments", which are separate copies of your systems where a mistake can't reach customers.

What should you ask the team that builds your agent?

Ask these six questions, whether your own developers build the agent or an outside team does, and get the answers in writing:

1. What does it decide on its own, and what follows a fixed script? If it decides nothing on its own, you are looking at automation, an assistant or a chatbot. That can be fine, but judge it as one.
2. Which actions can it take in our systems, and which need a person's approval? OpenAI's guide rates each tool an agent can use by read or write access, reversibility, permissions and financial impact. A good answer lists each action, and the risky ones wait for a person.
3. What happens when it fails or isn't sure? A good answer describes a handover to a person after a set number of failed attempts, as OpenAI's guide advises.
4. How will we know it does better than the simpler option? A good answer is a test on your own past cases, with the simpler option's results beside it and a pass mark agreed in advance.
5. What does one completed task cost to run? An agent may take many steps for one task, and each step adds cost. A good answer is a range per task from a test on your cases, plus what makes a task cost more, such as more steps or longer documents. Compare it with the cost of the simpler option for the same task. This article gives no cost figures, because they depend on the model, the number of steps and the provider's prices. A test on your own cases gives a number you can use. Escalating costs are the first cause Gartner names in its forecast.
6. Can someone else change it if the people who built it leave? This matters most when an outside team builds it. In September 2026, Gartner predicted that by 2028, 70% of enterprises will abandon agentic AI built by vendor forward-deployed engineering, "trapped by soaring costs and unable to evolve it on their own." That is a forecast too. Forward-deployed engineering means the vendor's engineers work directly with your team to build the solution and put it into use. Gartner analyst Mukul Saha says the best-scoped engagements set out ownership, knowledge transfer and "an exit strategy from day one." A good answer names who owns what gets built and how your team takes it over. If no one in your company changes software, the code and data should still be yours, so another developer could take over.

How does Arc AI answer this question?

"Do you actually need AI?" is one of the questions our process at Arc AI asks before we recommend anything. We do build agents, such as a customer support agent that checks who the customer is and resolves their ticket. Since we build agents, hold any advice we give you to the same four questions. Our page on AI chatbot development for support teams walks through one real conversation with such an agent, with the customer's details removed.

The AI step in our own blog import isn't an agent. There, a language model fills a fixed list of import fields around an article's text, our code checks every field, and a person publishes the draft. In Anthropic's terms, that is a workflow: the path is fixed in code, and the model only fills in the fields.

For developers and tech leads: use Anthropic's distinction in design reviews. Start with "single LLM calls with retrieval and in-context examples", meaning one model call that receives the relevant documents and a few worked examples. Anthropic calls that "usually enough" for many applications. Move to a workflow next, and to an agent only when an evaluation, a repeatable test on real cases, shows the gain. If you build one, OpenAI's guide recommends that you maximize a single agent's capabilities first before you split the work across several agents. It also says every run needs an exit condition, such as a maximum number of turns, so an agent can't loop without end. A turn is one round in which the model acts and reads the result.

If you're weighing an AI agent and want a second opinion on whether your task needs one, tell us what you have. You will hear from a person within one business day.