"Let every eye negotiate for itself and trust no agent" - William Shakespeare
This post is about critiquing the idea of AI agents and trying to settle my thoughts on the topic.
Currently at the time of writing AI agents are very in vogue in the business world. My current opinion on them is they seem very good at short range tasks and poor on longer. I have seen posts where a whole company of them was setup with no human user and it failed spectacularly.
Though I realised I had not really had much use myself nor had I found enough material of real world use cases to settle my thoughts.
What is a agent
There seems some debate on the subject of what a agent is. Tech crunch says no one knows what a agent is.
Source
https://techcrunch.com/2025/03/14/no-one-knows-what-the-hell-an-ai-agent-is/
They also say even the top brass says no one knows what a agent is.
Source
https://techcrunch.com/2025/05/12/even-a16z-vcs-say-no-one-really-knows-what-an-ai-agent-is/
Though somehow a thing we cannot define is posed to create a business revolution.
Academia definition
The source of the term seems to have been traced back to the 1990s. Specifically I could find the below paper Intelligent Agents: theory and practice.
Source
https://www.cs.ox.ac.uk/people/michael.wooldridge/pubs/ker95.pdf
It is important to note this definition has been defined before the development of various memory units, and the attention mechanism culminating in the 2017 attention is all you need.
Being earlier than current technological developments it should be seen as theory rather than a technichal underpinning.
Nonetheless the paper predicts by the year 2000 agents could be a 3.5 billion dollar industry. I cannot help but feel the concept of AI agents have a history of hype that does not deliver.
Despite this even at its initial inception they admit that the concept of AI agents is a loose concept. But comprises the following traits.
Aurtonomy: agents operate without direct intervention by humans.
Social Abillity: Agents interact with other agents using some for of agent language.
Reactivity: the language is a agent perceives and reacts to their user.
Pro-Activness: exhibits goal directed behaviour.
This is the weak definition of a AI agent. The strong argument sounds like AGI or even ASI.
Mobility: it can move
Veracity: it will not lie
Benevolence: it's a agent of humans so it has to at least not be hostile to us.
Rationality: if it has goals it would make sense that it pursued them at least semi rationally otherwise how could it even pursue them?
The paper continues but discusses the theory that underpins agents at the time. I would say it is not especially relevant in the modern context.
Traits modern AI do not have
We can immediately accept we do not have strong AI. No one would claim Chat GPT is mobile. Many transformer hallucinate and therefore lie or fail to act rationally.
Also the standard prompt based transformer does not have the traits of the weak agent hypothesis. Though attempts to build agents have taken a few forms and let's compare against them.
I think we can safely argue that a purely prompt based AI cannot be autonomous. If it has to receive a prompt and then acts on that prompt it has to be rejected it is autonomous yet I have seen many make this insistance. I really do not see a basis for this if the AI is designed to literally do as I say it cannot simultaneously be autonomous. I do not think that is even good marketing I cannot see how AI can simultaneously be prompt based and autonomous.
Ot could be said that reasoning models by being self promoting can take a initial prompt, plan and execute more than one frame of reasoning or planning. Though I think it's still just executing a prompt, I run programs in a while loop routinely you could express that as autonomy but it's a very funny sense of autonomy in the sense will autonomously do the exact next action given to it. I don't necessarily accept that it takes a prompt and generates a program of works from that prompt implies autonomy or goal directed behaviour.
It is not even clear that the argument is settled that LLMs "think" in any sense to have autonomy or goals. What a large language model does is predict the next token from a Bayesian model of what that token should be. Often it has a vector space called a thought vector but while we call it that the debate is not settled if this is just Bayesian prediction of the most likely response to a prompt or if some magic spark of AGI exists. In fact arguments can be forwarded in both directions that the fact LLMs have such high utility suggests there is some secret sauce while also that they seem to rapidly diverge from the initial task if recycled seems to indicate it's not really.
For example if it was "thought" then surely an AI would recognise its own output. Upon receiving text they produced as a new prompt surely it should spark sympathetic neuronal avalanches thereby being easier to identify. This does not seem to be the case.
Source
Though that seems to make it highly likely they do not have goals. How can an AI that does not recognise its own thoughts have complex goals? This to me is why I'm not taking the emergence of agents and agentic AI from LLMs seriously is evidence like this. I suspect AI will get better but I do not take seriously the notion of widespread agentic and AGI emergence in next 10 years as a much simpler explanation of it's just scaled up Naive bayes theorem applies to words seems sufficient to explain current capabilities.
I do grant that developments in memory like GRUs, LTSTM have been rapid. But these are often computationally expensive. The thing is I know at its core a lot of the processing is not memory based and much is forgotten between prompts. I do recognise that there might be a critical point we pass and that issue would be trivialised but if we have passed that threshold why are all these agents still prompt based?
A brief description of current AI agents
The current AI agents are often a wrapper on a LLM like chat GPT. Crudely what you do is you create a team of AI give each of them a role and create some places where they can share team messages.
This seems to work to reduce hallucinations. If one character in the role play makes a mistake others will correct them.
They still have to have a initial prompt but will split up the work each taking different tasks and thereby "thinking" about it will be improved and can send each other messages to correct each other like a team would.
I do feel that this on the surface looks like the social and agentic aspect that describes AI agents. They are in some sense "working together". My issue is simultaneously that seems obviously a form of roleplay of a single AI instance and not autonomous agents working together. It feels if you took off the label of it being AI agents you'd insist it was not a agent but was a form of extended long chain of thought self promoting reasoning model (which strangely after developing these agents have started to go into development).
There is a element here that feels disappointing comparing the theory from the 1990s with what we have.
While this reduces the mistakes dramatically you have the following downsides. You have no idea how many seperate tasks would be generated so do not necessarily know the bill for the task.
This can be bad if you assume some staff members might not know what tasks are easy for other human specialists and might turn to agentic AI running up a bill.
While it reduces hallucinations because the underlying LLM AI is the same model they have a tendency where hallucinating to kind of all join in with the hallucination which can make it worse in that when it does it can be more subtle and be weaved through the whole outputs work of which because took multiple steps can be quite large.
This in no way undermines that productive work could be done this way but at a search I cannot find any businesses who have implemented it and give a case study on before and after. I think some criticism should be made that organisational thinking really has not cemented on where to use it.
Over the horizon
l think my above criticism is mainly focused on as is current technology. Long chain of thought models are emerging and have capability to go back and correct there thoughts.
source
https://arxiv.org/pdf/2502.03373
An issue is getting stuck in reasoning loops where they continue to reassess their own directives though.
It should be clear that extending the time a AI reasons and thinks about actions might imply "more autonomy" some people. Though I think for myself I believe that concept ought to be seen as a binary. Even if an AI takes a prompt analyses it and enacts a extended set of actions it is still using me as the executive function. It is still my goals it is enacting and it is my autonomy that is in play for myself.
There is still reason to criticise even this development though. Even if I was to accept that AI has developed greater autonomy and goal oriented behaviour than before to do so it has to develop robust causal models.
source:
https://arxiv.org/pdf/2402.10877
The logic here is that in any work place to be effective their are always distribution shifts. An AI being a worker would need to be robust against this and would need to be able identify those shifts and respond.
The paper says that a causal model is required to understand those distribution changes I.e. the AI needs to have a world model on cause and affect to even start guessing at this.
One reason that I'm doubtful of agentic AI is how can a large language model trained only on language be said to have a causal model for the world outside the books and the internet?
Surely if you wanted 10 AI workers in a AI team to get those causal models you need to train each model as a specialist on that field? That's not what's happening they are taking one AI trained not to navigate the world and getting it to roleplay all the roles in the team.
That seems questionable that it would work at all in the longterm.
I kind of have no issue that AI can have robust models for the outside world I am just skeptical that to be such a useful agent it needs to have close to a causal model of everything and isnt that just AGI? Are we are having the concept that LLMs can be AGI repackaged and fed back to us repeatedly. I am doubtful that current technology can be AGI.
I prefer my own definition
I prefer to forward a alternative definition. Agents today are nothing about how we originally defined agents.
What is possible now is to instruct a AI and for it to interface with a tool and perform a task. This tool can be another AI creating chains of self promoting systems.
This is remarkable but not the agentic AI we where promised. I think there will be incredible use moving forward for chaining AI together in this way as we can in the future instruct a AI and it can go out and interact with the things that we want it to do and then do them.
Thats better at autonomy in the 1990s we thought we needed autonomy to get AI to do things we wanted. I think we are missing this is better if AI can be useful and useful while supervised you don't need it to be "benevolent" as we already discussed.
If you take this definition that AI is now being given tools to now use in the world and not in some sense agentic in the soft or hard hypothesis formed from the 1990s. Then yes we have agents and agentic AI. Though by that definition we have had useful AI since we first developed machine learning models.
I have 1980s books on using AI in a variety of situations. We have often found AI useful and it has often interacted with other tools and processes. Under that definition you'd just say AI not agent AI not agentic AI. It's useful it is a tool and uses tools with intelligence that is artificial. It's a AI.
That is a agent in the sense going back past the 1990s to Shakespeare quote at the start. You give it authority to go do something on your behalf.
Conclusion
My take on Agents is this. One is there is a genuine advancement towards useful AI and that includes emergence of "agentic" qualities that does not exist previously. We clearly can ask AI questions and it perform tasks.
This is astounding.
There is a social movement that takes these emergent agentic qualities and has consistently over promised based on the hard agentic hypothesis from the 1990s.
I do not believe autonomous or goal oriented AI exist on any commercial scale and I await data to tell me otherwise. To have agentic AI like that implies big robust causal model of much of the world.
We neither train AI nor appear to have the compute to make these types of models. So it is perplexing that people say this; given the vexation of this for the remainder of this conclusion I will adopt the tone I think appropriate to addressing this type of attitude.
No one knows what a agent is!
When you finally find a definition out current technology does not match it.
I feel that agentic AI swarms, self prompting chain of thought AI and long chain of thought AI are in some sense the same technology just with different names.
That the same criticism of that technology underlying them is that they are all something that seems unlikely to possess agency, and goals being very likely to be naive bayes approximation of next token.
This is on top of claiming it to have agency and goals after you and I watching them type in a prompt. You told it what to do; if you tell it what to do ie in some sense program it why is it a agent and not a program?
How does things we type into be said to have agency? Your basically saying you prompted something into existence if you insist agents have agency. I'm not judging but I think it's necessary for the people selling agents to clarify and express clearly what that means.
If it does not have agency why call it a agent? Can we find a better name? That is just bad English.
Because no one knows what a agent is. We appear to now have agent and agentic AI being two different things according to some people.
I find the insistence that agent AI and agentic AI being different annoying when one is the plural of the other.
people are actually writing articles on this...
source:
They have very precise definition of these AI are agents and these are agentic. My understanding was agents form teams of agentic AI. The agentic bit as we defined above means they work together. I've found im the minority in that view and it's really odd because I've never found anyone write a article to explain that geese and goose are specifically different types of bird.
If one agent is defined as having social capability why is that different to agent AI and agentic AI? This article defines agentic AI as having social capabilities and continuous self learning then gives the example of self driving cars. I didn't know self driving cars now operated in swarms; did you? I really did not think they learned how to drive while driving. That appears unsafe!
I've seen the same sentiment on LinkedIn and all over the place but if they are different why did someone use the plural term of one thing to describe something your then arguing is completely unrelated?
That's just grammatically bad! I find it maddening.
Im labelling this as more evidence no one knows what a AI agent is or really does. It's predictable if the AI community has long predicted since the 1990s these AI called agents and it will be a multi billion industry and your trying to redefine that term now being related to what you actually built because of monetary incentives. I still don't need to agree.
It feels as if it's just the search for AGI with a bit more thought put into it that maybe teams of AI would be better.
I do think teams of AI will be better but why are you then using one AI model to play every role on that team?
That if you look at the theory that underpins agents you would need more specialised AI than what we are throwing into this.
I am happy with the idea that teams of AI will correct each other and hence make less mistakes. They will hallucinate less.
But if true why do you then put every AI on that team to be exactly the same model of chat GPT?
it's all a bit odd...
I feel anxious that this is just marketing and really need further convincing that agentic AI is going to happen.
I accept that AI will always progress and our technology will improve. Though I feel it's counter productive to over promise and under perform. In some sense they are saying they will automate all the jobs to one degree or another and yet it's unclear in what sense they mean the technology to meet the softer agent hypothesis.
There is little data out there of successful use cases while it makes sense I'd want a team of AI to do stuff for me; that sounds great. To want to use them I'd need to have really precise information on rates they go wrong and how much that would cost because as much as a team of AI sounds great if they start hallucinating and need a infinite steps to do a planning session and I'm being charged the electricity bill for that; then that seems really bad...
Though I am completely willing to change my mind on all this I'm just raising I feel there is a lot to criticise here.
I really want to see someone else implement at scale successfully first! Show me the data that implies you have the multiple causal models of the entire work force your replacing and have then build them into teams and then show me your electricity bill for the same period.
Then I will believe you....
Add comment
Comments