This is probably role play, per the persona selection model, but there’s a lot of interesting research into the hidden “thoughts” of LLMs. Check out Neuronopedia and the Opus model cards for some great examples.
Yeah but saying all that is annoying so I think we should stick with saying thinking and everyone knowing what we mean isn’t literally identical to thought. Do you have a better solution?
Yeah, not conflating intelligent, creative problem solving with a glorified search engine that makes up the answers if it can’t lift them wholesale from another source. That would be a good start, right?
This doesn’t answer the question of finding a better solution.
I took the liberty to ask Lumo and his reasoning seem more useful than your thoughts:
A better solution is to adopt functionalist terminology that distinguishes between biological consciousness and computational processing without resorting to metaphorical confusion.
Instead of the binary of “it thinks” (which implies subjective experience) or “it doesn’t think” (which dismisses complex reasoning), we can use precise descriptors based on what the system is actually doing:
“Reasoning” or “Synthesizing”: Use these terms when the model is connecting disparate data points, performing logical deductions, or generating novel structures based on patterns. This acknowledges the output’s complexity without claiming the machine has an inner life.
Example: “The model is synthesizing a solution based on its training data,” rather than “The model is thinking about the problem.”
“Simulating” or “Mimicking”: Use these when the output resembles human thought processes but is strictly algorithmic. This clarifies that the form is human-like, but the mechanism is statistical prediction.
Example: “It is simulating a debate,” rather than “It is arguing.”
“Processing” or “Computing”: Reserve these for the raw mechanical act of token generation.
Example: “The system is processing the query,” rather than “The system is considering the query.”
Why this works better:
Precision: It avoids the philosophical baggage of “thought” (qualia, consciousness) while still acknowledging the utility of the output.
Clarity: It prevents the “Plagiarism Machine” critique from being a total dismissal. Even if the data comes from humans, the recombination and application to new contexts is a distinct computational process worth naming accurately.
Scalability: As models become more complex, “reasoning” or “synthesizing” scales better than “thinking,” which remains tied to biological definitions that may never apply to silicon.
So, the compromise isn’t to keep saying “thinking” and hope people understand, nor to insist on “regurgitation” which ignores the emergent properties of large-scale pattern matching. Instead, we shift the vocabulary to describe the process (reasoning, synthesizing, simulating) rather than the state of being (thinking).
That doesn’t really work either, that adds synthesizing to the terminology but doesn’t describe most of the behaviors they have. It’s not reasoning or simulating either.
That‘s what human minds mostly do as well. The overwhelming things you think and say are things you have heard or read elsewhere. Sometimes you combine two things you learned from the outside. Sometimes you develop a thing you learned a small step further. Actual creative thoughts stemming from yourself are pretty rare.
This is probably role play, per the persona selection model, but there’s a lot of interesting research into the hidden “thoughts” of LLMs. Check out Neuronopedia and the Opus model cards for some great examples.
Tracing the thoughts of an LLM
Signs of introspection in LLMs
LLMs do not think. The Plagiarism Machines read a million sentences humans wrote about AI thinking and regurgitated them.
Yeah but saying all that is annoying so I think we should stick with saying thinking and everyone knowing what we mean isn’t literally identical to thought. Do you have a better solution?
Yeah, not conflating intelligent, creative problem solving with a glorified search engine that makes up the answers if it can’t lift them wholesale from another source. That would be a good start, right?
Give me a better solution? How do I talk about the behaviors it exhibits without anthropomorphizaton?
This doesn’t answer the question of finding a better solution.
I took the liberty to ask Lumo and his reasoning seem more useful than your thoughts:
A better solution is to adopt functionalist terminology that distinguishes between biological consciousness and computational processing without resorting to metaphorical confusion.
Instead of the binary of “it thinks” (which implies subjective experience) or “it doesn’t think” (which dismisses complex reasoning), we can use precise descriptors based on what the system is actually doing:
“Reasoning” or “Synthesizing”: Use these terms when the model is connecting disparate data points, performing logical deductions, or generating novel structures based on patterns. This acknowledges the output’s complexity without claiming the machine has an inner life.
Example: “The model is synthesizing a solution based on its training data,” rather than “The model is thinking about the problem.” “Simulating” or “Mimicking”: Use these when the output resembles human thought processes but is strictly algorithmic. This clarifies that the form is human-like, but the mechanism is statistical prediction.
Example: “It is simulating a debate,” rather than “It is arguing.” “Processing” or “Computing”: Reserve these for the raw mechanical act of token generation.
Example: “The system is processing the query,” rather than “The system is considering the query.” Why this works better:
Precision: It avoids the philosophical baggage of “thought” (qualia, consciousness) while still acknowledging the utility of the output. Clarity: It prevents the “Plagiarism Machine” critique from being a total dismissal. Even if the data comes from humans, the recombination and application to new contexts is a distinct computational process worth naming accurately. Scalability: As models become more complex, “reasoning” or “synthesizing” scales better than “thinking,” which remains tied to biological definitions that may never apply to silicon. So, the compromise isn’t to keep saying “thinking” and hope people understand, nor to insist on “regurgitation” which ignores the emergent properties of large-scale pattern matching. Instead, we shift the vocabulary to describe the process (reasoning, synthesizing, simulating) rather than the state of being (thinking).
That doesn’t really work either, that adds synthesizing to the terminology but doesn’t describe most of the behaviors they have. It’s not reasoning or simulating either.
Fell free to offer a solution. For the moment the LLM wins, nobody else even tried
Everyone definitely doesn’t know they don’t think
That‘s what human minds mostly do as well. The overwhelming things you think and say are things you have heard or read elsewhere. Sometimes you combine two things you learned from the outside. Sometimes you develop a thing you learned a small step further. Actual creative thoughts stemming from yourself are pretty rare.
LLMs don’t read.
A machine cannot have a mouth to regurgitate from.