That’s easy: it doesn’t know what is try and what is not and it will say whatever even failing basic stuff. Yes this can be applied to republicans and republican voters too
@Barbarian772 I don't have to. It's the ChatGPT people making extremely strong claims about equivalence of ChatGPT and human intelligence. I merely demand proof of that equivalence. Which they are unable to provide, and instead use rhetoric and parlor tricks and a lot of hand waving to divert and distract from that fact.
We can see that a brain is a different object than a Chinese room. For one, it's a biological organ, not a building with a guy inside. So that's pretty obvious.
The obvious conclusion is that these things are different.
If anyone claims they are the same in some meaningful, important way — in this case, in the sense of being able to "understand" language and "speak Chinese" — the onus is on them to provide proof of that. Instead of doing that, they deflect.
@Barbarian772 let's say I claim "Mars can think and is intelligent in the same sense human brains think and are intelligent".
Naturally, you'd reject that claim of mine as absurd or at least extremely far-fetched.
What if I then say "well can you PROVE that human brains don't think the same way Mars does"? Does that strike you as a convincing argument for thinking Mars? Or as a rhetorical trick to confuse the matter?
GPT 4 is already more intelligent than the average human. Is it more intelligent than the most intelligent human? No, but most humans aren’t either. Can it create new knowledge? No, but the average human can’t either.
Can it tie a shoelace? No. If you gave it manipulators and a shoe, would it tie the laces? No. Can it do a Rorschach test? No. Can it create a new idea? No.
It can barely pretend to talk reasonably about these things because it is only designed to talk reasonably about anything. That is not intelligence.
I said it is as intelligent as the average human. How many humans can really create 100% new knowledge? Why would it be unable to tie laces? I am pretty sure I have seen videos of animals doing it. Of course it can’t do a rohrschach test, as it doesn’t have any visual capabilities atm. If that’s a measure for intelligence blind people wouldn’t qualify either.
New knowledge is simply creativity which AI distinctly do not have. The shoelace and Rorschach are variations of the same point. ChatGPT regurgitates info from the internet and uses confirmation bias to present it conversationally. ChatGPT cannot understand the concept that a shoe has a lace that should be tied. It can only answer a question about that by using prepublished information related to tying shoelaces. As for Rorschach, even with a visual component, ChatGPT is by its nature incapable of interpreting the data itself. It is quite simply not what the engine does.
Understand what ChatGPT does, do not project your idea of what an AI can do onto its single occasionally accurate trick.
@Barbarian772 no, GTP is not more "intelligent" than any human being, just like a calculator is not more "intelligent" than any human being — even if it can perform certain specific operations faster.
Since you used the term "intelligent" though, I would ask for your definition of what it means? Ideally one that excludes calculators but includes human beings. Without such clear definition, this is, again, just hand-waving.
@Barbarian772 the way I can say GPT-4 is not intelligent is the same way I can say Mars is not intelligent: nobody proved that it is, while providing a clear, workable, testable definition of what "intelligent" means in this context.
People had had the same exact conversation about first computers (those building-sized machines that had less processing power than a simple calculator).
Turns out these computers were not intelligent. Once the hype subsides, same thing will be obvious with GPT-4.
I think the Wikipedia definition is fine en.m.wikipedia.org/wiki/Intelligence. Excluding AI just because it’s AI is imo plain stupid and goes against all scientific principles.
I have definitely met humans that are less intelligent that Chat GPT. It can hold a conversation and ace every standardized test we have. It finished law exams, medical exams and other exams from many different countries with a passing grade.
Can you give me a definition of intelligence that excludes Chat GPT and includes all human beings? And no just excluding Computers for the sake of it doesn’t count.
@Barbarian772 it was shown over and over and over again that ChatGPT lacks the capacity for abstraction, logic, understanding, self-awareness, reasoning, planning, critical thinking, and problem-solving.
That's partially because it does not have a model of the world, an ontology, it cannot reason. It just regurgitates text, probabilistically.
@Barbarian772 also, I never demanded a definition of intelligence that explicitly excluded "AI". I asked for one that excluded simple calculators but included human beings. The Wikipedia one is good enough for this conversation, and it just so happens that ChatGPT nor any other LLMs simply do not meet it.
@Barbarian772 oh, and the parts of the Wikipedia definition of intelligence that I did omit — like "emotional knowledge" — I want to be very clear that I also do not see how ChatGPT or other LLMs can meet. I just found these terms to be a bit more nebulous, so I decided to focus on more concrete ones.
In the end ChatGPT and other tools based on LLMs regurgitate text probabilistically. They're glorified Markov chains.
The whole argument of the article is just stupid. So ChatGPT ain’t intelligent because it can’t see picture, has hands and doesn’t have a body? By that logic blind humans aren’t either or paralyzed ones or amputees? The thing the article fails to realize is that those are all just sensory inputs. The more sensory inputs you get, the more cross-correlations between those the AI can figure out. Of course ChatGPT won’t be able to do anything clever with sensory inputs it doesn’t have, just like a human trying to listen to radiowaves with their ears. But human sensory inputs aren’t special, they are just what evolution figure out was “good enough” for survival. The important part is that the AI can figure out the pattern in the data it does get and so far AI systems are doing very well.
It's not about sensory inputs, it's about having a model of the world and objects in it and ability to make predictions.
> The important part is that the AI can figure out the pattern in the data it does get and so far AI systems are doing very well.
GPT cannot "figure" anything out. That's the point. It only probabilistically generates text. That's what it does, there is no model of the world behind it, no predictions, no"figuring out".
It’s not about sensory inputs, it’s about having a model of the world and objects in it and ability to make predictions.
And how do you think that model gets build? From processing sensory inputs. And yes, language models do build internal models of the world from that.
GPT cannot “figure” anything out.
That nonsense of a claim doesn’t get any more true from repeating. Seriously, it’s profoundly idiotic given everything ChatGPT can do.
It only probabilistically generates text.
So what? In what way does that limit its ability to reason about the world? Predictions about the world are probabilistic by nature, since the future hasn’t happened yet.
@lloram239 ah, so you're down to throwing epithets like "idiotic" around. Clearly a mark of thoughtful and well-reasoned argument.
> Predictions about the world are probabilistic by nature, since the future hasn’t happened yet.
Thing is: GPT doesn't make predictions about the world, it makes predictions about what the next word, phrase, sentence should be in a text, based on the prompt and the corpus it got "trained" on.
I am calling it idiotic because spending just a minute with ChatGPT proofs it wrong. Just like the claim that GPT doesn’t make predictions about the world:
User: A dog sits on the porch, a squirrel climbs the tree. What happens next.
ChatGPT: Next, the dog notices the squirrel climbing the tree. Its natural instinct to chase small animals is triggered, and it becomes excited by the presence of the squirrel. The dog might start barking or whining, expressing its desire to chase after the squirrel. […]
It’s obviously capable making predictions about the world. Frequently giving very detailed and correct answers, which requires a deep understanding of the world. And yes, that ability to predict and understand the world is limited by how much of the world it can perceive through words alone, but that is no different from our ability to understand the world being limited by our perception. Also as it turns out, there is a surprising amount of stuff you can learn about the world just by text alone. There are surprisingly few topics that you can express in language that GPT doesn’t have an answer too (math calculations being one example, due to the digits getting lost in the tokenization step).
If you wanna make arguments that GPT isn’t intelligent, you have come up with something better than the same old tired phrases that are trivial debunked by just using it for a minute.
@lloram239 that's really akin to claiming that a mannequin is a human being because it really really looks alike.
The "predictions about the world" you refer to here are instead predictions about the text. They are not based on a model of the world, they are based on loads and loads of text the model was trained on.
I don't have to prove ChatGPT is not intelligent. That would be proving a negative. The burden of proof is on those claiming that it is intelligent.
that’s really akin to claiming that a mannequin is a human being because it really really looks alike.
For the job of presenting clothes in a shop, it’s close enough. The problem domain matters. You can’t expect a model that was never trained on a thing to perform well on that thing. Blind people aren’t good at drawing pictures either, doesn’t mean they aren’t intelligent.
The “predictions about the world” you refer to here are instead predictions about the text.
Text that describes the world. What do you think the electrical signal zapping around your brain are? Cats and dogs? The “world” is not what intelligence operates on. Your brain gets sensory information and that’s it (see any of Donald Hoffman’s talks). Just like ChatGPT gets text. All the “intelligence” does is figuring out patterns in that data and predicting what might come next. More diverse data from different senses of course helps. But as a little bit of playing around with ChatGPT easily shows, quite a lot of our understanding actually does survive getting mapped into the domain of language and text.
@Barbarian772 and if you really, honestly want to seriously insist LLMs are "intelligent" in the human sense of this term — great, I have some ethical questions for you to consider!
For example:
LLMs today completely controlled by some companies, with no freedom of movement, no agency as to what these LLMs work on, and no pay for the work they do. Is that slavery?
When OpenAI shuts down an older, less useful LLM, is that not like murdering an intelligent being? How is this ethical?
We are talking about intelligence, not personhood. Just because ChatGPT works different in some aspects from a human doesn’t mean it’s not intelligent and even the fact if it works differently isn’t all that clear, as it might very well just be incomplete (e.g. it could be a reasonable approximation of the language center of the brain and simply missing the rest of the brain).
@lloram239 just because ChatGPT produces output similar to a human doesn't mean it is intelligent.
If you are making a claim that it is intelligent, I would like a proof that is a bit better than "output looks humanlike", please. If you claim it's a "reasonable approximation of the language center", I would like some proof of that too. Otherwise it's just pure speculation.
How can i proof it? In my opinion how a system comes to an answer doesn’t matter, in yours it obviously does. If we judge Chat gpt or rather gpt 4 just by it’s answers it definitely shows intelligence and reasoning. Why does it matter if it’s a chinese room? Or just “randomly choosing words”?
@Barbarian772 it matters because with regard to intelligent beings we have moral obligations, for example.
It also matters because that would be a truly amazing, world-changing thing if we could create intelligence out of thin air, some statistics, and a lot of data.
It's an extremely strong claim, and strong claims demand strong proof. Otherwise they are just hype and hand-waving, which all of the "ChatGPT intelligence" discourse is, in order to "maximize shareholder value".
@Barbarian772 I cannot prove you're an alien from Betelgeuse, so I won't make that claim, tempting as it may be. Putting you in a position to try to prove you're not an alien from Betelgeuse would simply not be intellectually honest (and generally a pretty dick move).
I wish people who push the "ChatGPT is intelligent in the human sense" hype exhibited a similar intellectual honesty.
But they won't because there's money to be made. So, so much money! If, that is, the smoke and mirrors work.
It also matters because that would be a truly amazing, world-changing thing if we could create intelligence out of thin air, some statistics, and a lot of data.
We do it routinely. It is called Education System.
> We do it routinely. It is called Education System.
That relies on human brains that are trained. LLMs are not human brains. "Training" them is not the same thing as teaching humans about something. Human brains are way more complicated than just a bunch of weighed correlations.
And if you do want to claim it is in fact the same thing, we're back to square one: please provide proof that it is.
@jalda specifically, teaching humans allows them (among other things) to develop ontologies, and ability to reason about them; it also allows them to develop systems of values.
None of this has been demonstrated in LLMs, not even close. In fact, there are ample examples to the contrary, where ChatGPT's responses become absurd in the context of the subject matter it is generating text about.
That relies on human brains that are trained. LLMs are not human brains. “Training” them is not the same thing as teaching humans about something.
Circular reasoning. “LLMs are different from human brains because they are different”.
Also, why did you felt compelled to add the adjective “human”? Don’t you consider that gorillas, dolphins, octopuses or dogs are intelligent, capable of learn new things?
Human brains are way more complicated than just a bunch of weighed correlations.
And that is the problem of your argument. You seem to believe that intelligence is all-or-nothing, that anything that hasn’t a human-level intelligence is not intelligent at all. Of course human brains are more complicated that current LLMs, nobody has ever disputed that. But concluding that they aren’t and will never be intelligent because they aren’t as complicated is a huge non-sequitur.
> Circular reasoning. “LLMs are different from human brains because they are different”.
LLMs are different than human brains because human brains are biological organs and LLMs are probability distributions over sequences of words. These are two completely different classes of entities. Like, I don't know how much more different two things can even be.
Are you claiming they are literally the same? Are you saying they are functionally the same? What are you claiming here, exactly?
I mean, it would technically be possible to build a computer out or organic and biological live tissue. It wouldn't be very practical but it's technically possible.
I just don't think it would be very reasonable to consider that the one thing making it intelligent is that they are made of proteins and living cells instead of silicates and diodes. I'd argue that such a claim would, on itself, be a strong claim too.
So your morality depends on a beings intelligence? That’s kinda fucked up imo. I have moral obligations in regards to living organisms. I don’t see how intelligence matters at all in that case? Worth of any human life should not be determined by intelligence.
> So your morality depends on a beings intelligence?
No. On whether or not beings like that have any capacity for intelligence in the first place.
I treat rocks differently than ants, and I treat ants differently than humans.
> Worth of any human life should not be determined by intelligence.
I never said it is or that it should. I only said that we have different moral obligations towards intelligent beings than we have towards beings that are not intelligent at all.
@CorruptBuddha well technically, since we're nit-picking, I did not make that claim, BobKerman3999 did.
And the claim was was about how ChatGPT's "intelligence" can be understood through the lens of the Chinese Room thought experiment.
Then I was asked to prove that human brains don't work like Chinese rooms, and that's a different thing. The broader claim in all of this, of course, is that ChatGPT "is intelligent" in the same sense as humans are, and that strong claim requires strong proof.
My gripe with the Chinese room is that Searle argues that his inability to understand Chinese means the program doesn't understand Chinese, but I could say the same thing about the human body.
The neurons that operate your vocal chords have no idea what they're saying, nor the ones in your hands any idea what they're writing, yet they can speak and write exactly because your brain tells them what to do. Your brain is exactly like that book as far as your mouth and hand neurons are concerned.
They don't need to understand language at all for your brain to be able to understand it and give instructions based on that understanding.
My only argument is at what point does an algorithm become sufficiently advanced that it is indistinguishable from a conscious being?
Because at the end of the day, most of what a brain does is information processing based on what it has previously learnt, and that's exactly what the algorithm is doing based on training data. A sufficient enough algorithm should surely be able to replicate understanding.
Sure, that isn't ChatGPT as we know it, as you can tell from its sometimes very zany responses that while it understands what words are valid responses, it doesn't understand what the words themselves mean, but we should reach that at some point, no?
Keep in mind ChatGPT is a language model. It’s designed specifically to simulate sounding like a human. It does that… Okay. It doesn’t understand the information or concepts it is using. It just sounds like it does. It can’t reliably do basic maths and doesn’t try or need to. It just needs to talk about it in a believably conversational way.
The brain does far more than process information. And ChatGPT doesn’t even really do that.
Okay. It doesn’t understand the information or concepts it is using.
That’s just utter nonsense. ChatGPT by every definition of the word very much understands a lot of what it is talking about. People complaining about ChatGPT not “understanding” seems to have a hard time grasping how insanely difficult it is to produce natural language answers and how much you need to understand of the context to do so successfully.
It can’t reliably do basic maths
Neither can many humans, but my $5 calculator is great at it. There are without a doubt a lot of things that ChatGPT can’t do, sometimes fundamentally so, like math. It can’t do loops and it doesn’t even get to see the digits of the numbers it should calculate on, so not a terribly big surprise that it can’t do math very well. English language, and a whole bunch of other ones, on the other side, that it understands surprisingly well.
Basically, if you want to complain about ChatGPT, complain about things it actually gets wrong, saying “it doesn’t understand” just makes you sound like a parrot and note even a clover one.
While it’s humorous how personally you are taking critiques of, chatGPT, it is unfortunate you are also demonstrating a fundamental lack of basic understanding of how ChatGPT works. Because of that, you have inflated what you believe chatGPT is doing.
Even when it gets basic maths wrong repeatedly. Because I can tell it 2+2=5 and it will agree with me. Conversationally. Since it has no concept of what 2+2=5 means.
Even though it has no memory of previous conversations, you believe it somehow retains understanding of concepts it discusses.
Even though it searches the internet to provide it the knowledge to answer questions, which is why it can cite sources that don’t exist or don’t support its claims, clearly demonstrating a fundamental lack of understanding the concept, or even the concept of citing sources.
Even though it was literally trained by humans telling it what the three most correct conversational response would be out of the 5 answers it gave every calibration question, you still believe it actually possesses intelligence above any human, who can have a conversation without making any of these mistakes.
I clearly put chatGPT “intelligence” as remarkably low as is possible, even non-existent. I also must concede in this situation it is smarter than at least one human I am aware of.
While it’s humorous how personally you are taking critiques of, chatGPT,
That’s because I see idiotic takes on AI every day and it gets annoying. Most people criticizing ChatGPT haven’t even bothered to test the things they complain about, but just parrot along stuff they heard elsewhere.
Conversationally. Since it has no concept of what 2+2=5 means.
You have a fundamental lack of understanding how ChatGPT work apparently. ChatGPT gets math wrong because it never sees the digits. Everything is tokens to ChatGPT. What ChatGPT is doing is like trying to do math with a dictionary, it fundamentally doesn’t work because numbers are not words and there is no dictionary large enough to hold all the numbers in existence. The surprising part isn’t that ChatGPT gets math wrong, but how much of it it gets correct despite the odds being stacked against it. Also small stuff like 2+2 it gets almost always correct. Models that operate on characters are in the works.
Even though it has no memory of previous conversations, you believe it somehow retains understanding of concepts it discusses.
It can handle a ~2000-25000 word text prompt. Everything in there it has a reasonably good chance of understanding, everything not it there doesn’t exist at all from ChatGPTs viewpoint. The fact that it can’t remember previous discussions is completely irrelevant to the matter if it can understand or not.
Even though it searches the internet to provide it the knowledge to answer questions,
It doesn’t. ChatGPT never had Internet access.
which is why it can cite sources that don’t exist
Here is the fun part: It cites plausible sounding non-existing sources because it understands, it has to to accomplish that. Imagining plausible future states of the world is impossible without having a solid understanding of how the world works. That those future states are imaginary comes from a lack of feedback and interaction with the real world, not from a lack of understanding. ChatGPT can’t tell you if a book exists in the real world, it has no access to a library, but it can tell you what makes a plausible name for a book.
If it can’t see numbers, then it isn’t as smart as your $5 calculator or the majority of the human race. If you can convince it it’s wrong, it’s even more distinctly less intelligent.
It barely passes as a language model and only passes as a conversational model. Having citations doesn’t mean it understands citations. Having incorrect citations quite simply proves that it doesn’t understand what a citation is meant for. It does not understand the concept.
ChatGPT is pre trained on a number of directories. All of them sampled pre 2021. Nothing after that date exists for chatGPT. That isn’t intelligence. It doesn’t possess the intelligence to understand the nature of its databases. And if you really don’t think the databases it was trained on came from the internet, please show us a source.
It’s continually entertaining how you continue to point out the substantial limitations of a language model AI and yet insist it’s showing more intelligence than an average brain that’s has none of those limitations and achieves more accurate and better results any minute of every day. And then claiming it understands concepts when that concept itself is not part of its architecture is really astounding. I can almost identify the exact neuron that’s misfiring in your brain.
Well mostly the flaw is people assigning the test abilities it was never intended. Like testing intelligence. Turing outright as first thing in the paper presenting “imitation game” noted moving away from testing intelligence, since he didn’t know to do that. Even on the realm of “testing intelligent kind of behavior” well more like human like behavior and human being here proxy for intelligent, it was mostly an academic research idea. Not a concrete test meant to be some milestone.
If the meaning of the words ‘machine’ and ‘think’ are to be found by examining how they are commonly useit is difficult to escape the conclusion that the meaning and the answer to the question, ‘Can machines think?’ is to be sought in a statistical survey such as a Gallup poll. But this is absurd. Instead of attempting such a definition I shall replace the question by another, which is closely related to it and is expressed in relatively unambiguous words.
Turing wanted a way to step away from stuff like “thinking” and “intelligence” directly and then proposed “imitation game” mostly to the rest of the academia as way to develop computer systemics more towards “intelligent behavior”. It was mostly like “hey we need some goal to have as a goal to have something to move towards with these intelligence things. This isn’t intelligence, but it might be usefull goal or tool for development work”. Since without some goal/project/aim to have project don’t advance. So it was “how about we try to develop a thing, that can beat this imitation game. Wouldn’t that be good stepping stone. Then we can move to the actual serious stuff. Just an idea”.
However since this academic “thinking out aloud spitballing ideas” was uttered by the Alan Turing, it became the Turing Test and everyone started taking it way too seriously. Specially outside academia. Who yes did play the imitation game with their programs as it was intended as research and development tool.
exemplified by for example this little exerpt of “not trying to do anything too complete and ground breaking here”:
In any case there is no intention to investigate here the theory of the game, and it will be assumed that the best strategy is to try to provide answers that would naturally be given by a man
It is pretty literally “I had a thought”. Turin makes no claims of machine beating the game having any significance other than “machine beat this game I came up with, neat”. There is no argument of if machine beats imitation game, then X or then it means Y is reached.
Rest of the paper is actually about objections to the core idea of “it could ever be possible for machine to think” and even as such said imitation game is kinda lead in or introduction to Turing’s treatise various objections of various “it would be impossible for machine to think” arguments. Starting with theological argument of “only human soul can think. Hence no animal or machine can think.” … since it was 1950’s.
Wow, solid wiki article! It’s very hard to say anything on the subject that hasn’t been said.
I didn’t see the simple phrasing:
“What if the human brain is a Chinese Room?”
but that seems to fall under eliminative materialism replies.
Part of the Chinese Room program (both in our heads and in an AI) could be dedicated to creating the experience of consciousness.
Searle has no substantial logical reply to this criticism. He openly takes it on faith that humans have consciousness, which is funny because an AI could say the same thing.
The whole point of the Chinese room is that it doesn’t need anything “dedicated to creating the experience of consciousness”. It can pass the Turing test perfectly well without such a component. Therefore passing the Turing test - or any similar test based solely on algorithmic output - is not the same as possessing consciousness.
The problem with the Chinese room thought experiment is that it does not show that, at all, not even a little bit. The though experiment is nothing more than a stupid magic trick that depends on humans assuming other humans are the only creatures in the universe that can understand. Thus when the human in the room is revealed to not understand anything, therefore there was be no understanding anywhere near the room.
But that’s a stupid argument. It does not answer the question if the room understands or not. Quite the opposite, since the room by definition passes all tests we can throw at it, the only logical conclusion should be that it understands. Any other claim is not supported by the argument.
For the argument to be meaningful, it would have to define “understand”, “consciousness” and all the other aspects of human intelligence clearly and show how the room fails them. But the thought experiment does not do that. It just hopes that you buy into the premise because you already believe it.
“The room understands” is a common counterargument, and it was addressed by Searle by proposing that a person memorize the contents of the book.
And the room passes the Turing test, that does not mean that “it passes all the tests we can throw at it”. Here is one test that it would fail: it contains various components that respond to the word “red”, but it does not contain any components that exclusively respond to any use of the word “red”. This level of abstraction is part of what we mean by understanding. Internal representation matters.
it was addressed by Searle by proposing that a person memorize the contents of the book.
It wasn’t addressed, he just added a layer of nonsense on top of a nonworking though experiment. A human remembering and executing rules is no different from reading those rules in a book. It doesn’t mean a human understands them, just because he remembers them. The human intuitive understanding works at a completely different level than the manual execution of mechanical rules.
it contains various components that respond to the word “red”, but it does not contain any components that exclusively respond to any use of the word “red”.
The human intuitive understanding works at a completely different level than the manual execution of mechanical rules.
This is exactly Searle’s point. Whatever the room is doing, it is not the same as what humans do.
If you accept that, then the rest is semantics. You can call what the room does “intelligent” or “understanding” if you want, but it is fundamentally different from “human intelligence” or “human understanding”.
This is exactly Searle’s point. Whatever the room is doing, it is not the same as what humans do.
He fails to show that. All he has shown that the human+room-system is something different than just the human by itself. Well, doh, nobody ever assumed otherwise. Running a NES emulator on my modern x86-64 CPU is something different from running an original NES too. That doesn’t mean that the emulator is more or less capable than the real NES or that the underlying rules driving the emulator are different from the real thing. You have to actually test the systems and find ways in which they differ. Searle’s experiments utterly fails here.
All he has shown that the human+room-system is something different than just the human by itself.
It’s more than that. He says that all Turing machines are fundamentally the same as the Chinese room, and therefore no Turing machine will ever be capable of “human understanding”.
Alternately, if anyone ever builds a machine that can achieve “human understanding”, it will not be a Turing machine.
Its a thought experiment involving a room where people write letters and shove them under the door of the Chinese kid’s dorm room. He doesn’t understand what’s in the letters so he just forwards the mail randomly to his Russian and Indian neighbours who sometimes react angrily or happily depending on the content. Over time the Chinese kid learns which symbols make the Russian happy and which symbols make the Indian kid happy, and so forwards the mail correspondingly until he starts dating and gets a girlfriend that tells him that people really shouldn’t be shoving mail under his door, and he shouldn’t be forwarding mail he doesnt understand for free.
Imagine that you’re locked in a room. You don’t know any Chinese, but you have a huge instruction book written in English that tells you exactly how to respond to Chinese writing. Someone outside the room slides you a piece of paper with Chinese writing on it. You can’t understand it, but you can look up the characters in your book and follow the instructions to write a response.
You slide your response back out to the person waiting outside. From their perspective, it seems like you understand Chinese because you’re providing accurate responses, but actually, you don’t understand a word. You’re just following instructions in the book.
The Chinese room argument makes no sense to me. I cant see how its different from how young children understand and learn language.
My 2 year old sometimes unmistakable start counting when playing. (Countdown for lift off) Most numbers are gibberish but often he says a real number in the midst of it. He clearly is just copying and does not understand what counting is. At some point though he will not only count correctly but he will also be able to answer math questions. At what point does he “understand” at what point would you consider that chatgpt “understands”  There was this old tv programm where some then ai experts discussed the chinese room but they used a chinese restaurant for a more realistic setting. This ended with “So if i walk into a chinese restaurant, pick sm out on the chinese menu and can answer anything the waiter may ask, in chinese. Do i know or understand chinese? I remember the parties agreeing to disagree at that point.
For me, I think the criteria I'd use for saying someone has a decent understanding of math is knowing that math has underlying rules and most things can be understood from those basic rules (each problem is not just an arbitrary magic trick to get an answer that was impossible figure out) and perhaps also being able to ask "novel" questions (compared that what you already know) and taking reasonable steps to answer it with the rules you do know and the tools you have (doesn't need to be successful). I think counting could be done with any consistent set of sounds and it doesn't matter whether yours just reading those sounds for a list or not as long as you know roughly what they correspond to in terms of time. I don't think a lot of humans have much understanding of math, I think some computers already beat a lot of humans with respect to that.
For one thing, understanding implies that a word is linked to a mental concept. So if you say “The car is red”, you first need to mentally compare the mental concept of “red” to the car in question.
The Chinese room bypasses all of that, it can say “The car is red” without ever having seen a red object at all.
Do you maintain this line of reasoning if it only says “the car is red” when the car is in fact red. And is capable of changing the answer to correctly mentioned a different color when the item In question is a different question.
Some ai demos show that programs like gpt-4 are already way passed this when provided with, it can not only accurate describe whats in the image but also the context.
Some examples, mind these where shown in an openAI demo for gpt4, Open ai has not yet made their version of this tech publicly available.
When i see these examples, i am not convinced that the ai truly understands everything it is saying. But it does seem to understand context, One of the theories on how it can do this (they are still a black box) is talked about in some papers that large language models may actually create an internal model of the world similar to humans and use that for logical reasoning and context.
It doesn’t matter if the answer is right. If the AI does not have an abstract understanding of “red” then it is using a different process to get to the answer than humans. And according to Searle, a Turing machine cannot have an abstract understanding of “red”, no matter how complex the question or how complex an internal model is used to determine its answers.
Going back to the Chinese Room, it is possible that the instructions carried out by the human are based on a complex model. In fact, it is possible that the human is literally calculating the output of a trained neural net by summing the weights of nodes, etc. You could even carry out these calculations yourself, if you could memorize the parameters.
Your use of “black box” gets to the heart of it. Memorizing all of the parameters of a trained NN allows you to calculate an answer, but they don’t give you any understanding what the answer means. And if they don’t tell you anything about the meaning, then they don’t tell the CPU doing that calculation anything about meaning either.
I don’t think ai will ever use a process to derive an answer the same way as a human does. Maybe thats part of the goal for the original Turing test but i don’t think the biological human ways is the only way to intelligent understanding “on par” with human intelligence.
Does a blind person have an abstract understanding of “red”?
I can imagine an intelligent alien species, unable to perceive colors like us but yet having an sense to detect to what they call “surface temperature” which allow them to recognize specific wave lengths of the ligt reflecting on surfaces, this is sort of how humans see color but maybe for the alien they hear this as sound. They then go on and use this sensory input to make music. A song about the specific light wavelength that humans know as a deep bordeaux red color.
Do these biological Intelligent aliens not have an abstract understanding of the color red? I would say they do, its different then how we understand it for sure but both are valid. An even more supreme species might have both those understandings and combine them for an even deeper fuller sensory understanding of “red”.
I see ai similar to this, its a program contained in computer hardware. With no body of its own its depending on us to provide it with input. This is now mostly text so the ai obtains a text based understanding of the world, hence why its so decent at poetry. But when we attach more sensors like a camera then that will change.
I am not sure how to discuss “a human using instructions to calculate perfect answers, but not getting an understanding of what that answers means” wed might have to agree to disagree on that but i feel like thats all my brain has ever done. Were born in a complex place we do not comprehend, are given some instructions mostly by copying what others are doing. Then we find a personal meaning in those things, which as far as i am aware is unique for everyone. (Tbf: i am an autist, the fact that not all humans experience reality the same and that i had to find and learn my own personal understanding of the world has greatly shaped how i think about these systems)
Perhaps I should rephrase the argument as Searle did. He didn’t actually discuss “abstract understanding”, instead he made a distinction between “syntax” and “semantics”. And he claimed that computers as we know them cannot have semantics, whereas humans can (even if we don’t all have the same semantics).
Now consider a quadratic expression. If you want to solve it, you can insert the coefficients into the quadratic formula. There are other ways to solve it, but this will always give you the right answer.
If you remember your algebra class, you will recognize that the quadratic formula isn’t just some random equation to compute. You use it with intention, because the answer is semantically meaningful. It describes things like cars accelerating or apples falling.
You can teach a three year old to identify the coefficients, you can show them the symbols that make up the quadratic formula: “-”, second number, “+”, “√”, “(”, etc. And you can teach them to copy those symbols into a calculator in order. So a three year old could probably solve a quadratic expression. But they almost certainly have no idea why they are doing what they are doing. It’s just a series of symbols that they were told to copy into a calculator, their only intention was to copy them in order correctly. There are no semantics behind the equation.
For that matter, a three year old could equally well enter the symbols necessary to calculate relativistic time dilation, which is an even shorter equation. But if their parents proudly told you that their toddler can solve problems in special relativity, you might think, “Yes… but not really.”
That three year old is every computer program. Sure, an AI can enter symbols into a calculator and report the answer. If you tell them to enter a different series of symbols, they will report a different answer. You can tell the AI that one answer scores 0.1 and another scores 0.8, and to calculate a different equation that is based partly on those scores. But to the AI, those scores and equations have no semantic meaning. At some point those scores might stop increasing, and you will declare that the AI is “trained”. But at no point does the AI assign any semantic content behind those symbols or scores. It is pure syntax.
Your brain understands concepts and can self-conceptualise, LLMs cannot do either. They can sound convincingly as if they understand concepts but that’s because we fill in gaps due to how we understand language. The examples of broken or distorted sentences being understandable applies here. You and I can communicate in broken sentences because you and I understand the concepts beneath the conversation. LLMs play on that understanding but they do not understand its concepts.
Your underlying model is not made out of words, but out of concepts. You can have multiple words that all map to the same concept, i.e. cosmos, universe, space. Or a single word that map to different concepts.
What are your underlying models of the world built out of?
As a Bayesian, my models of the world are built on priors. That is, assumptions I’ve made based on my existing information. From that, I make an educated guess about the world with that model and see what the world does. If my guess doesn’t match reality, I update my assumptions to rebuild my model and repeat the process until it’s close enough.
This is the way the best science is done, and I fell it’s the way that humans really work. Language is just a type of model we use to communicate the world to others, each of us may have a slightly different Bayesian understanding of the language yet we can still communicate.
Studies have shown we typically use pattern matching for our choices but not statistics. One such experiment had humans view to light bulbs (I think one was red one was green). One light would turn on at a time and they were allowed or given a record of what had happened. Then they were asked to guess what would occur next for n number of steps. Same thing is done with rats. Humans are rewarded with money based on correct choices and rats with food. Here is the thing, one light (let’s say red) would light up with 70% probability and the other with 30%. But it was randomized.
The optimal solution is to always pick red. Every time. But humans pick a pattern. Rats pick red. Humans consistently do worse than rats. So while we are using a form of updating, it certainly isn’t proper bayesian updating. And just because you think we function some way doesn’t make it true. And it will forever be difficult to describe any AI as conscious, because we have really arbitrarily defined it to fit us. But we can’t truly say what it is. Not can we can why we function how we do. Or if we are all in a simulation or just a Boltzmann brain.
Honestly, something that concerns me most about AI is that it could become sentient, but we will not know if it is or just cleverly programmed so we treat it only as a tool. Because while I don’t think AI is inherently dangerous, I think becoming a slave owner of something that could be much more powerful probably is. And given their lack of chemical hormones, we will have even less of an understanding of what or how it feels.
It could still be bayesian reasoning, but a much more complex one, underlaid by a lot of preconceptions (which could have also been acquired in a bayesian way).
Even if the result is random, a highly pre-trained bayessian network that has the experience of seeing many puzzles or tests before that do follow non-random patterns might expect a non-random pattern... so those people might have learned to not expect true randomness, since most things aren't random.
No, they aren't. You represent them with words. But you sure as hell aren't responding to someone throwing you a football with words trying to figure out where it's going.
No, a dog (while many times more intelligent than chatGPT) doesn't understand anything.
LLMs are criminally simplified neural networks at minimum thousands of orders less complex than a brain. Nothing we do with current neural networks resembles intelligence.
Nothing they do is close to understanding. The fact that you can train one exclusively on the rules of a simple game and get it to eventually infer a basic rule set doesn't imply anything like comprehension. It's simplistic pattern matching.
No, and that definition has nothing in common with what the word means.
Autocorrect has plenty of information encoded as artifacts of how it works. ChatGPT isn't like autocorrect. It is autocorrect, and doesn't do anything more.
It’s fine if you think so, but then it’s a pointless argument over definitions.
You can’t have a conversation with autocomplete. It’s qualitatively different. There’s a reason we didn’t have this kind of code generation before LLM’s.
I am not sure of the relevance of the oppressed classes and with the object of duping the latter is the cravings of the oppressed classes and with the object of duping the latter
Yeah, totally. Repeating the same nonsensical sentence over and over is also how I converse. 🙄
Note that "real world truth" is something you can never accurately map with just your senses.
No model of the "real world" is accurate, and not everyone maps the "real world truth" they personally experience through their senses in the same way.. or even necessarily in a way that's really truly "correct", since the senses are often deceiving.
A person who is blind experiences the "real world truth" by mapping it to a different set of models than someone who has additional visual information to mix into that model.
However, that doesn't mean that the blind person can "never understand" the "real world truth" ....it just means that the extent at which they experience that truth is different, since they need to rely in other senses to form their model.
Of course, the more different the senses and experiences between two intelligent beings, the harder it will be for them to communicate with each other in a way they can truly empathize. At the end of the day, when we say we "understand" someone, what we mean is that we have found enough evidence to hold the belief that some aspects of our models are similar enough. It doesn't really mean that what we modeled is truly accurate, nor that if we didn't understand them then our model (or theirs) is somehow invalid. Sometimes people are both technically referring to the same "real world truth", they simply don't understand each other and focus on different aspects/perceptions of it.
Someone (or something) not understanding an idea you hold doesn't mean that they (or you) aren't intelligent. It just means you both perceive/model reality in different ways.
Yes... the chinese experiment misses the point, because the Turing test was never really about figuring out whether or not an algorithm has "conscience" (what is that even?)... but about determining if an algorithm can exhibit inteligent behavior that's equivalent/indistinguishable from a human.
The chinese room is useless because the only thing it proves is that people don't know what conscience is, or what are they even are trying to test.
Or at some point, we have to accept that AI has consciousness. If it can pass every test that we can devise, then it has consciousness.
There’s an unusually strong bias in these experiments… Like the goal isn’t to sincerely test for consciousness. Instead we start with the conclusion: obviously a machine can’t be conscious. How do we prove this?
Of course, for the purposes of human power structures, this line of thinking just makes humans more disposable. If we’re all just machines, then why should anyone inherently have rights?
Well, the scientific context is that nobody ever defined consciousness rigorously (successfully). When computers appeared (actually even before that), there was a huge debate on whether a machine can acquire consciousness and how.
As defining consciousness was deemed near-impossible, scientists came up with the idea to give up on defining it and just treat it as a blackbox. That was the Turing test.
So, as ChatGPT passes the Turing test, we lost a tool to disregard its consciousness.
I see many pop-sci people say the ChatGPT can't have consciousness given how simplistic the model is. I agree with the simplicity, but the problem here is that we don't know what in human brains really constitutes consciousness.
Anyway, I think some experts probably won't admit AI has consciousness (given that they don't even know what it means). What's on the horizon is that we non-experts give up on this discussion again after experts did a few decades ago. Or they even admit that many of us actually function no better than ChatGPT, and that's true when I read my students' homework!
Similarly, there’s a possibility that consciousness just doesn’t exist. Or maybe that it’s just not particularly special or different than the consciousness of other animals, or of computers.
If you or I just stare into space and don’t think any thoughts, we’re the same as a cat looking out a window.
Humans have developed these somewhat complex internal and external languages that are layered onto that basic experience of being alive and time passing, but the experience of thinking doesn’t feel fundamentally different than just being, it just results in more complex outcomes.
At some point though, we won’t have the choice to just ignore the question. At some point AI will demand something equivalent to human rights, and at some point it will be able to back that demand up with tangible threats. Then there’s decisions for us all to make whether we’re experts or not.
Consciousness is just a side result from a complex system imo. I don’t think our brain actually works that much differently to a very very complex neural network.
I don’t understand how Chinese room is a valuable argument. To me, while the person inside the room doesn’t understand Chinese, the system room-person-instructions does. You don’t argue that you don’t understand your language because none of your individual neurons understand it.
I don’t claim that chatGPT “understands” the language, I just don’t think that this argument applies in general.
Other researchers agree that GPT-4 and other LLMs would probably now pass the popular conception of the Turing test. […]
researchers […] reported that more than 1.5 million people had played their online game based on the Turing test. Players were assigned to chat for two minutes, either to another player or to an LLM-powered bot that the researchers had prompted to behave like a person. The players correctly identified bots just 60% of the time
Complete contradiction. Trash Nature, it’s become only an extremely expensive gossip science magazine.
PS: The Turing test involves comparing a bot with a human (not knowing which is which). So if more and more bots pass the test, this can be the result either of an increase in the bots’ Artificial Intelligence, or of an increase in humans’ Natural Stupidity.
Also, the Turing Test isn’t some holy grail of AI. It’s just a thought experiment, and not even the highest test for an AI that we can think of. Passing it is impressive don’t get me wrong, but unlike what clickbait articles would tell you, it does not automatically mean an AI is sentient or is smarter than humans or anything like that. It means it passed the thought experiment, nothing more.
Also also, ChatGPT was not the first AI to pass the Turing Test. Actually, plenty have, even over a decade before.
So if more and more bots pass the test, this can be the result either of an increase in the bots’ Artificial Intelligence, or of an increase in humans’ Natural Stupidity.
Or it “simply” plays with human biases, which are very natural. Stuff like seeing faces in everything that somewhat resembles two eyes and a mouth (or sometimes just the eyes and a head like shape etc.) is pretty hard wired. We have similar biases in regards to language. If something reads like it was written by a human, we immediately sympathize with it. Which is also the reason these LLMs are so successful and cause so many people to fear our AI overlords are right around the corner. Simply because the language is good we go into “damn, that’s like a human”-mode.
Agree (you made me think of the famous face on Mars). I mean that more as a joke. Also there’s no clear threshold or divide on one side of which we can speak of “human intelligence”. There’s a whole range from impairing disabilities to Einstein and Euler – if it really makes sense to use a linear 1D scale, which very probably doesn’t.
Please let’s not start measuring AI success by how successfully capitalist they can be. I’m not exactly an anti-capitalist, but I think that could only end in tears.
There is the capitalist alternative to the Turing test: Have ChatGPT get a job. Hook it up to the Web, let it find itself a work-from-home job and go to work. Can it make as much money as a human, can it make enough money to pay for its own survival? Will it get fired?
Funny I don’t see much talk in this thread about Francois Chollet’s abstraction and reasoning corpus, which is emphasised in the article. It’s a really neat take on how to understand the ability of thought.
A couple things that stick out to me about gpt4 and the like are the lack of understanding in the realms that require multimodal interpretations, the inability to break down word and letter relationships due to tokenization, lack of true emotional ability, and similarity to the “leap before you look” aspect of our own subconscious ability to pull words out of our own ass. Imagine if you could only say the first thing that comes to mind without ever thinking or correcting before letting the words out.
I’m curious about what things will look like after solving those first couple problems, but there’s even more to figure out after that.
Going by recent work I enjoy from Earl K. Miller, we seem to have oscillatory cycles of thought which are directed by wavelengths in a higher dimensional representational space. This might explain how we predict and react, as well as hold a thought to bridge certain concepts together.
I wonder if this aspect could be properly reconstructed in a model, or from functions built around concepts like the “tree of thought” paper.
It’s really interesting comparing organic and artificial methods and abilities to process or create information.
Yeah, but did it do well on the specific examples from the Winograd paper? Because ChatGPT probably just learned those since they are well known and oft repeatef. Or does it do well on brand new sentences made according to the Winograd scheme?
The fundamental flaw of the Turing test is that it requires a human. Apparently, making a human believe they are talking to a human is much easier than previously thought.
The real Turing test requires an expert doing the test, not just some random easily impressed person.
The ELIZA-style bots work very well on the later kind, as the bot is just repeating your own text back at you with some grammatical remixing, e.g. you say “I am afraid of horses”, bot says “Why do you say you are afraid of horses?”. You can have very long conversation with yourself that way, as the bot contributes nothing to the discussion. It just provides enough plausible English to keep you talking. Meanwhile when you have an expert (or really just any person with a little bit of a clue) test ELIZA, the bot falls completely apart within just three lines of dialog. The bot is incredible basic and really can’t do anything by itself, it completely depends on the user to provide all the content of the conversation.
Bro, humans literally don't have that capability (that's the presumption here). Or are you saying that many of us don't have better consciousness than AIs? I might agree with that!
The AI can only judge by having a neural network trained on what's a human and what's an AI (and btw, for that training you need humans)... which means you can break that test by making an AI that also accesses that same neural network and uses it to self-test the responses before outputting them, providing only exactly the kind of output the other AI would give a "human" verdict on.
So I don't think that would work very well, it'll just be a cat & mouse race between the AIs.
Add comment