Agent washing: telling a real agent from a chatbot

By asking five questions, only two of which can be checked during a demo. A real agent writes into a system, chooses the order of its steps, notices its own failure, triggers without a human and remembers from one time to the next; anything ticking none of them is an assistant, and anything ticking two or three is an automation with a language model inside.

The question is worth asking coldly because the market has stopped being legible. Gartner estimates that around 130 vendors, out of the thousands claiming the agentic label, genuinely sell an agent, the others having rebranded assistants, automations and chatbots without changing much underneath. The same firm expects more than 40% of agentic AI projects to be cancelled before the end of 2027, and the reasons for those cancellations are not unrelated to the gap between what was bought and what was delivered.

What exactly is agent washing?

It is selling under the “agent” label a product belonging to another category, without lying about its functions and without describing them either. The technique is rarely crude. Nobody claims a chatbot takes decisions; the copy says it “orchestrates”, that it “reasons”, that it is “agentic”, and the reader fills in the rest.

The vocabulary lends itself to this, which often makes the accusation of bad faith unfair. There is no normative definition of an agent, every analyst firm has its own, and a sincere vendor can use the word for a product its neighbour would refuse to call by that name. Which is precisely why a buyer needs a test bearing on observable behaviour rather than on words.

The five questions that settle the category

None requires technical skill, and each is answered yes or no.

Does it write into a system? This is the question that eliminates fastest, and it costs nothing to ask before a demo has even started. If the output of the conversation has to be copied by hand into the ATS, the CRM or the document in order to exist, the product has saved you some drafting and nothing else. An agent leaves a trace somewhere other than the thread it was spoken to in.

Is the order of steps decided at runtime? Ask the vendor to draw the path on a whiteboard. If they manage it completely, with its branches and conditions, you are looking at an automation, and a good automation is an excellent purchase that simply needs paying for at the price of an automation. An agent is recognisable by the fact that the diagram depends on what it will have found along the way.

Does it know when it has failed? A product that executes and then declares it has executed has no feedback loop. An agent reads the result of its action, notices that a send was rejected or that a record is empty, and does something else rather than carrying on as though nothing had happened. It is the most discriminating criterion and the least advertised, because it requires admitting that failure is a normal case.

Can it start without being asked? A product that acts only after a human message remains a tool, however clever it is in between. The ability to trigger on an event, a deadline or an anomaly changes the nature of the relationship, and it is also what makes it essential to define what it must not be able to do.

Does it know what happened last time? Many products keep the context of a thread and lose it between sessions, forcing you to explain everything again the following Monday. Persistent memory is what separates a colleague from a consultant you re-brief, and it has its own cost, since what an agent forgets matters as much as what it keeps.

Why only two of these questions survive a demo

Writing into a system and choosing the steps can be seen on screen, which explains why demos are built around them. The other three are structurally invisible in that exercise, and it is worth understanding why rather than suspecting a manoeuvre.

A demo is a happy path: the file is clean, the API answers, the profile exists. Reaction to failure therefore cannot show itself, for want of a failure. It is triggered by a human typing a sentence in front of you, which makes the autonomy criterion unobservable by construction. And it runs on a prepared session, where memory of a previous week has no occasion to serve.

The practical conclusion is that a demo serves to rule out, never to choose. It tells you with certainty that a product is not an agent, and it cannot tell you that it is one, which brings us back to what we write about testing an agent on your own files rather than on the vendor’s.

Getting the category wrong costs more than being lied to

Here is where most articles on agent washing stop too early, leaving the impression that the agent is the premium option and everything else a counterfeit.

The most frequent mistake we see among our clients is not having bought a chatbot at the price of an agent. It is having wanted an agent where a deterministic automation would have done the job better: cheaper to run, faster, predictable, auditable line by line, and incapable of going off the rails on a Friday evening. Extracting skills from a CV into a fixed format, chasing everyone who has not replied after seven days, posting the same advert on three channels, all of that is repetitive work whose steps are known in advance, and handing that kind of task to an agent amounts to paying a premium for uncertainty.

The mirror mistake exists and costs more over time. Working out why a consultant on the bench was never put forward, deciding which profile to present to which client, preparing a dossier by fetching what is missing from wherever it sits: these tasks have no drawable path, and an automation answers them with a cascade of rules nobody maintains after a year. The symptom is recognisable, it is the conditions file nobody dares touch any more.

Which phrasings should prompt one more question?

None of the following is dishonest in itself, and that is the problem: they are compatible with a real agent as much as with its imitation, which makes them good places to dig rather than good grounds for suspicion.

“It orchestrates your tools.” Ask which ones it writes to and which it only reads. A read-only integration takes a day, a write integration commits the vendor on error handling, duplicates and permissions, and the cost difference between the two explains why the list of connectors is often longer than the list of actions.

“It reasons over your context.” Ask where the context comes from and how long it survives. A model receiving three paragraphs pasted into the prompt does reason over your context, strictly speaking, and it will have forgotten it within the hour.

“It works autonomously.” Ask what triggers it. Many products call autonomy the fact of chaining several steps after a human click, which is uninterrupted execution rather than initiative, and the nuance starts to matter on the day you rely on it to spot what nobody noticed.

“Up to 40% time saved.” Ask on which population, measured how, and above all who reviewed the output. A gain observed by whoever produces the work gets declared readily, whereas the time spent reworking it disperses among reviewers and appears in no table, which is enough to turn a transfer of load into a headline saving.

Conversely, the word “copilot” used plainly is rather a good sign. A vendor owning up to a product that assists without acting tells you exactly which box it belongs in, and leaves you free to compare that with its price.

What the grid will never tell you

It gives the product’s category, not its quality, and confusing the two would mean swapping one empty word for another.

A mediocre real agent exists, ticking all five criteria and getting it wrong one time in three, which makes it a considerably worse purchase than an honest assistant. The category determines what the product can do, and says nothing about what it does well, rather as knowing that a candidate holds the required degree tells you little.

What remains to be measured is measured on your material, with files whose right answer you already know, and over several days so that memory and autonomous triggering have the chance to show themselves. The next question is therefore one of protocol, and it has an answer of its own: thirty cases drawn from your history, replayed three times, whose variance says more than the success rate.