The fruit fly may feel pain as a stimulus. Even if not: you are interrupting it’s normal biological function and disabling it of living its life normally. This may even be a disturbance of local eco-systems and thereby affect more forms of life, both feeling and not-feeling.

EDIT: Just realised that the following might be beside your point, misread your comment, lol. Will leave it as is now, because I think it’s worth considering when thinking about consciousness, experience, etc. in the context of LLMs.

But more importantly: LLMs are trained to model patterns and make predictions. This is indeed something our brains do as well. The difference lies in a state of mind and ‘understanding’ though. I will detail this:

Reminiscient of the chinese room thought experiment: Consider you are imprisioned in the chinese room and have to figure out yourself which chinese characters you have to slip back under the door depending on which ones you receive. (Or in any other language that you have absolutely no understanding of. I will assume chinese for this now.) If you provide a correct answer, you will get food. If not, you won’t and maybe even get additionally punished somehow.

Over time you will start to see patterns and remember which characters to correctly provide. However, you will never hear someone speaking, and you will never really associate the characters with things you know from life. For example, you wouldn’t notice you were talking about an apple when slipping 苹果 through. You would also not know how to pronounce it. You would just know this is the desired answer to draw and slip back.

Now, a native chinese person is sitting in a neighbouring room. And some experimenter who doesn’t know in which room you or the native speaker sits, will slip the same ‘input’ characters to both of you and may ask how the acidic-sweet, fruit is called that is round and grows on trees. Assuming you have trained long and well enough, both of you will provide the correct reply, 苹果, and from the perspective of the experimenter it will be impossible to distinguish who the real native speaker is and who you, the prisioner is.

Now, the chinese person would have had an image in his head when thinking about the apple, would know how it tastes like, etc… You on the other hand wouldn’t even have noticed that you were talking about apples. The 苹果, is just some symbol to you that seems appropriate.

For the experimenter though, they might think they were speaking to two real chinese speakers although you were just ‘faking’ it.

Next, we place a computer in the third room that uses our current LLM technology. Those are trained in a manner that is in principle similar how you were trained: roughly speaking, negative feedback for bad replies, positive feedback for good ones. A mathematical optimization algorithm then modifies the model such that the desired replies become more likely to happen.

After a while the experimenter returns and now has to distinguish behind which door the computer is, the prisioner, and the native speaker. It will be impossible. It may be easier to test with more difficult ‘questions’, there the native speaker might perform much better than you or the computer. But in theory, we assume the computer and you have learned the patterns perfectly. Then it really is impossible to tell who is faking it and who the ‘true speaker’ is.

But, there is still a difference. You are not the LLM. You got hungry, you may have even felt pain, you have wondered what those symbols mean, you may even have remembered some symbols that you found especially beautiful. And of course, you wondered when those maniacs would release you.

All of that does not happen in LLMs. They do not have a thought-loop. They do not feel anything. No pain, no hunger. They have no awareness of their situation and no sense of being or something that might resemble consciousness. They do not strive for anything, have no motivation or aspirations. They are fire-and-forget machines without any state of mind. Receiving input on one side, providing output on the other without any reflection or retention. Their only memory is how they should answer provided some input, since they, same as you, just modeled the patterns.

This can appear eeringly authentic. If the experimenter would have asked you on chinese what your life goals are, both you and the LLM would provide an answer that would be likely accepted by the experimenter. But both you and the LLM actually have no idea what you have just said. It would be fakery.

If the experimenter would have asked about what your deepest, darkest fears are, same thing: you would provide an answer that would be acceptable when someone asks about deep dark fears, but you would again have no concept of what you have just talked about. This doesn’t make you or the LLM good chinese speakers. It makes you convincing fakers.

So there are two levels of difference here between a human and a LLM as we currently have them:

1. The conversation is convincing fakery.
2. There is no experiencing subject behind the conversation. The LLM does not know that it is having a conversation, does not understand the meaning of what it says in the experiential sense described above, and does not feel or experience anything while producing its answers. There is no inner perspective to which the conversation is happening.

That may change in the future. But currently, there is no established evidence of an experiencing subject behind the conversation. Moreover, it is technically unnecessary, because nothing about how current LLMs operate requires subjective experience, awareness, or an inner perspective.

And this is precisely where the convincing fakery becomes deceptive, since language is normally one of the strongest ways humans infer another person’s state of mind. If someone tells you that they are scared, lonely, excited, or in pain, it is usually reasonable to infer that there is a person behind those words actually experiencing something. An LLM can produce exactly the same statements simply because they are appropriate responses in that context. The statement itself therefore cannot tell us whether there is an experience behind it.

A fly is a living thing and an Nvidia GPU is not. Holy shit, I’m really having to explain this…

There is legitimate controversy that living things, even as small an a fly, can experience pain as a stimulis. It is believed that only more complex organisms can experience suffering as a result of pain, and only higher order organisms feel the suffering of loss.

LLMs are capable of generating tokens. They cannot experience pain; they can only superficially generate words about pain. The LLM in this “torture chamber” isn’t suffering any more than you would if you started howling performatively in pain for no reason. In fact, it is suffering even less than you simply because you have the ability to experience pain and suffering, and it objectively does not.

It’s increasingly difficult to identify what is ‘alive’, these days, with people creating synthetic cells and whatnot. And there was always the question of viruses. I can imagine a similar difficulty with defining consciousness and intelligence. The Turing test fell by the wayside a long time ago. There’s got to be more to it than saying “I am conscious!” - you can get a computer to do that with a single line of code. Where does the real definition lie? Fucked if I know.