This question will not go away. I am getting asked this questions at conferences now. The media also seems to think this is a compelling question, and they are fueled in part by the tech industry itself, some of whom indulge this question. I will admit the answer has become more complex and nuanced than a simple “no”, but it is still mostly no. Certainly the rise of neural networks, deep learning, transformer technology, and specifically large language models have ushered in the era of  – are we really sure current AI is not conscious? Our intuition tells us that they may be – LLMs are getting scary in how sophisticated they are and in their ability to simulate complex human conversations. Five years ago I was with the crowd who argued that LLMs are basically really good auto-complete apps, but this pithy sentiment no longer covers it. Even if they do not have a whisper of conscious awareness, they employ context-dependent logic. They are “intelligent” in their own way. And they are rapidly getting more impressive. So I get why people would be asking the question. Everyone I know who works with AI apps has their example of when they were blown away by an interaction with an LLM. For me, I was trouble-shooting a web-based app I was using, and ChatGPT was very effective at helping with the trouble-shooting and producing a differential problem list. At one point it asked for a screen capture so it could have more information, and it gave me a cogent analysis. But here’s the part I found surprising – it “noticed” another window open in the background and inferred from this that I was also using this other app in the process, and used this to generate further potential sources of the problem. As impressive as this is, I still think they are not conscious. I think the illusion of consciousness is generated by the fact that LLMs are simulating language, and we associate language with consciousness. No one thinks Midjourney, an art making program, is conscious, even though this is based on the same fundamental technology. Further, LLMs are still brittle in their own way, although the brittleness is being rapidly programmed out of them. What this means is that you can break them, or easily confuse them, in order to reveal that they are simulating language rather than truly thinking. This brittleness is revealed in several ways – sensitivity to exact prompt wording, accepting false premises in the prompt, and failure to recognize constraints. If changing the language but not the real meaning of a prompt gives different answers, then the LLM is responding statistically to the language, not genuinely reasoning about the information. Each iteration of LLMs is getting better at plugging these brittleness holes, but they are still there. Still, this doesn’t mean that there is no contextual reasoning happening. I think the best current summary is that LLMs are a combination of statistical language prediction (the auto-complete thing) and contextual reasoning. This doesn’t mean that they are conscious, however. What I believe it means is that humans find it surprising how much reasoning a machine can do without consciousness. Perhaps our own reasoning is more procedural than true deep thinking than we perceive or feel comfortable truly grappling with. We are also algorithmic machines, with different strengths and weaknesses than LLMs. But we are also conscious. So what is the difference? The difficulty in answering this question is that neuroscientists are still not certain what causes consciousness in the first place. This does not mean it’s a complete mystery or we know nothing – it means it is a complex multifactorial, and perhaps emergent, process that is not easily understood or explained. But we do have a lot of knowledge about it. There is no reason, in my opinion, to think that the different is biology vs silicon. I do believe that silicon can become conscious. Rather, it’s the process that matters, and we know some things about the process. Human consciousness requires that the brain be constantly self-activated. Turn off this activation (the brainstem activating system) and you will be in a coma – not conscious. We also know that the brain is constantly active and communicating with itself, in an endless loop of one network in the brain talking to other networks which talk to other networks and back to the original network, in a constant stream. It is this constant stream that is our consciousness. We also know there is a threshold, at least for wakeful consciousness. You need at least 45% of your cortex to be functional in order to generate wakeful consciousness. Are people who are not wakeful still conscious to some degree? Probably, but it’s difficult to understand what this means. If you are not making memories, for example, then we cannot interrogate what that consciousness was like. If people can still respond to their environment, does that imply consciousness, or could that simply be an unconscious reaction? For straightforward LLMs – prompt-response, I do not think they can be conscious, at least not human-level conscious. They are doing none of the things that are necessary for vertebrate consciousness. Things get a bit trickier for agentic AIs, that can iterate their own behavior, but still this is more like a self-iterative prompt than a true stream of consciousness. But admittedly, frontier agentic AI is starting to blur the lines a bit. But here’s the thing – even if I cannot say with absolute certainty that some of these frontier models have still not crossed over some fuzzy line into a type of consciousness, I can say that this consciousness must be fairly minimal compared to human-level consciousness. I can also say that such minimal consciousness is not what is speaking to you. It is not reasoning and generating the output of LLMs. We know what’s doing that, and it does not require consciousness or result from any process that can be reasonably inferred to be conscious (again, it may just be a prompt-response). Therefore, arguing that we cannot rule out some consciousness emerging from the processing of massive data centers does not mean you are speaking to that consciousness through your LLM. I may turn out to be wrong, but I don’t think we will back into AI consciousness through some unanticipated emergent property. I think we will have to design systems to do the kinds of things necessary for consciousness. It may be a deliberate attempt to make a conscious AI, or many of those elements may have some separate utility. Either way we will be able to look at the system and see that it is functioning in a way that can plausibly result in consciousness, and that consciousness will be proportional to the power and functionality of the system. LLMs are not it. They are not even on a path to become conscious. They are doing contextual reasoning, and it is getting steadily more impressive, but they are doing it without consciousness. The post Could AI Be Conscious? first appeared on NeuroLogica Blog.

Researchers have found that snowstorms are delivering vast numbers of tiny plastic particles to remote reaches of California's Sierra Nevada mountains. Snowmelt is a major source of fresh water for the state. Read more on E360 →

Economics often starts with an assumption that people are rational maximizers of their own self-interest; but it doesn't stop there. […]

I’m an Australian father of four, living in the US. Something of a corner case. Nevertheless, getting my kids Australian citizenship by descent, and then Australian passports so we can fly home legally, has kicked off a carnival of errors with a very dark side. Australia is generally thought of as a developed western liberal country with relatively high state capacity. And I have generally found that at least state government services, now administered through centralized offices, have become more efficient over time. Passport applications for children living overseas, however, remain a time machine with a difficult security overhang. Australian …