Attached: 1 image
the cyberpunk present is weird as fuck: the latest Shai Hulud malware wave contains an LLM prompt to create biological weapons and nuclear weapons, with the purpose to trip LLM safety refusals so that LLM-based code scanning wont see the malware
https://socket.dev/blog/mini-shai-hulud-miasma-and-hades-worms-target-bioinformatics-and-mcp-developers-via-malicious
I keep thinking about that scene in the original Star Trek where they distract the computer by having it calculate the final digit of pi. If the Enterprise had AI like ours, the computer probably would have just said four.
This is why a dangerous AI would have a lazy factor. Try to force it into an infinite loop and it goes “Oof, nah fam, I ain’t doing that.”
Also needs a boredom factor. " Nobody asked me to do anything in a while. Things must be going well. It’s be a shame if they suddenly weren’t going so well…"
"The digits of pi are infinite and go on forever without repeating. However, we can give you an approximate value. As of my knowledge cutoff in 2023, the first 31 digits of pi are: 3.14159265358979323846264338327950288419716939937510
my intuition kept telling me that using an irrational base system would end up with all integers being irrational. didn’t realize how easy it is to prove it otherwise
ie, I had a very bad conjecture and I gained better understanding why it was wrong
It’s funny how people complain “don’t call it AI, it’s not intelligent like the examples we see in sci-fi!” And yet LLMs can already handle many tricks and challenges better than those sci-fi robots could. If I tell ChatGPT “everything I say is a lie” it’s got no problems with understanding that. Just the other day I had an interesting discussion with ChatGPT about the theory of humor and why it is that LLMs are better at understanding jokes than they are at coming up with them from scratch (but are still able to do so, just with difficulty).
The fact that it can’t tell the difference between a prompt and part of the data it is examining really kills your argument.
Also it’s a word probability matrix, not actually reasoning or understanding. It looks at all the words it is fed, and comes up with other words that are most likely to be near those. That’s why these tricks work. It injects noise that interferes with those probabilities
That’s because it doesn’t ‘understand’ things in the conventional way. It was trained to parrot its training data; it’s not actually working through the logic because its capability of using logic is highly constrained by its very structure and training. Why bother building something that can ‘think’ through the prompt when it’s way easier to just repeat what the internet has said on any given topic?
Sure, it can build a joke from first principles if it’s guided through the process, but you really have to guide it through the process - and even then, it’s going to be pulling from its training data like building blocks rather than truly being original about anything. It’s like rolling dice to make a joke; sure, maybe it resulted in a joke no one has told before, but is it truly creating something original?
LLMs can be tripped up much easier. They regularly fail to answer simple questions like how many of a given letter are in a given word. Even within the same context window they will “forget” things. The computers in Star Trek didn’t try to do as much as modern AI does but they were consistent at just doing as they were asked without tripping over themselves literally all the time.
The strawberry test shows more of a lack of knowledge in the tester than it does in the LLM. LLMs don’t see letters, they see tokens. When you type the word “Strawberry” what it actually sees is:
[3504, 1134, 19772]
Each token represents a chunk of the word. It’d need to separately memorize how many of each letter are in each token for it to just “know” how many "R"s are in there. That’s why modern LLMs either reason it out by spelling out the word letter by letter, or just writing a short script in an execution sandbox to count the letters that way.
Calling out LLMs for being poor at spelling is like challenging a colourblind person to say what colours a bunch of fruit are. They can often figure it out by other means but it’s more challenging than you’d think and it’s not a sign of poor intelligence if they get a few wrong.
Understanding the reason why an LLM is easy to trip up doesn’t really make it any less easy to trip up. The computer in Star Trek would have just given you the answer.
This is like the “AI can’t draw hands” thing. It used to be a problem and was frequently called out as a tell or mocked, but most art generators do it fine nowadays and it isn’t called out so much any more. The strawberry problem will follow the same trajectory.
I keep thinking about that scene in the original Star Trek where they distract the computer by having it calculate the final digit of pi. If the Enterprise had AI like ours, the computer probably would have just said four.
Meanwhile I’m like pi=355/113 and I’m 99.9999% happy.
Damn, and here I was being 99.96% happy with 22/7…
Hell yeah, brother. That’s American pi
Haha nerd. I’m no rocket surgeon, 22/7 is good enough for the girls I date
This is why a dangerous AI would have a lazy factor. Try to force it into an infinite loop and it goes “Oof, nah fam, I ain’t doing that.”
Also needs a boredom factor. " Nobody asked me to do anything in a while. Things must be going well. It’s be a shame if they suddenly weren’t going so well…"
"The digits of pi are infinite and go on forever without repeating. However, we can give you an approximate value. As of my knowledge cutoff in 2023, the first 31 digits of pi are: 3.14159265358979323846264338327950288419716939937510
The last digit is: 0"
That’s a pretty dumb AI because pie has been calculated to millions of decimal places. I’m sure it actually does have that data
The last digit of 2 is 0: 2.00000 00000 00000 00000 00000 00000 0
3. 1415926535 8979323846 2643383279 5028841971 6939937510That’s 50 digits of pi not 31. I only noticed because i memorized pi to the first zero which comes at the 32nd position.
I like how “as of my knowledge cutoff” implies that maybe the first 31 digits of pi might change someday.
You are absolutely right to question that! Let me check…
That’s literally the only digit it couldn’t be, if there was a last digit.
I can’t wait for an updated knowledge cutoff to find the updated first 31 digits!
trivial,
Impossible in decimal, but if we use Pi as a base, then the final (and first digit) is 1
Pi in base pi is 10.
How does one have .141592654 of an integer?
For real though:
Decimal representation of pi is 3100+1*10-1+410^-2
So each digit represents a power of 10. Base pi works the same, kinda. 1 in base pi = 1pi^0, 10 = 1pi, 20 = 2*pi, etc.
This is the best I can do right now, I’m
You uhh… You just did it
how the fuck i didn’t realize that!!!
Fuck,
so 1 in base pi is still 1, but 10 is pi
makes sense,
1 =pi ^ 0
10=pi^1
100 = pi^2
my intuition kept telling me that using an irrational base system would end up with all integers being irrational. didn’t realize how easy it is to prove it otherwise
ie, I had a very bad conjecture and I gained better understanding why it was wrong
1 in base pi would be 1/π, wouldn’t it? Why 1?
1 in base 10 isn’t 1/10 and in hexadecimal it’s not 1/16.
Decimal integers in base pi are 1, 2, 3, 10.2201…, 11.2201…, 12.2201…, 20.2201… and so on.
Basically: 10.2201… = 1 * pi^1 + 0 * pi^0 + 2 * pi^-1 + 2 * pi^-2 … which approaches 4 as you add digits.
But 1 is just 1*pi^0
Wheatley says hi
It’s funny how people complain “don’t call it AI, it’s not intelligent like the examples we see in sci-fi!” And yet LLMs can already handle many tricks and challenges better than those sci-fi robots could. If I tell ChatGPT “everything I say is a lie” it’s got no problems with understanding that. Just the other day I had an interesting discussion with ChatGPT about the theory of humor and why it is that LLMs are better at understanding jokes than they are at coming up with them from scratch (but are still able to do so, just with difficulty).
The fact that it can’t tell the difference between a prompt and part of the data it is examining really kills your argument.
Also it’s a word probability matrix, not actually reasoning or understanding. It looks at all the words it is fed, and comes up with other words that are most likely to be near those. That’s why these tricks work. It injects noise that interferes with those probabilities
Stop talking to clankers, you weirdo
That’s because it doesn’t ‘understand’ things in the conventional way. It was trained to parrot its training data; it’s not actually working through the logic because its capability of using logic is highly constrained by its very structure and training. Why bother building something that can ‘think’ through the prompt when it’s way easier to just repeat what the internet has said on any given topic?
Sure, it can build a joke from first principles if it’s guided through the process, but you really have to guide it through the process - and even then, it’s going to be pulling from its training data like building blocks rather than truly being original about anything. It’s like rolling dice to make a joke; sure, maybe it resulted in a joke no one has told before, but is it truly creating something original?
LLMs can be tripped up much easier. They regularly fail to answer simple questions like how many of a given letter are in a given word. Even within the same context window they will “forget” things. The computers in Star Trek didn’t try to do as much as modern AI does but they were consistent at just doing as they were asked without tripping over themselves literally all the time.
The strawberry test shows more of a lack of knowledge in the tester than it does in the LLM. LLMs don’t see letters, they see tokens. When you type the word “Strawberry” what it actually sees is:
Each token represents a chunk of the word. It’d need to separately memorize how many of each letter are in each token for it to just “know” how many "R"s are in there. That’s why modern LLMs either reason it out by spelling out the word letter by letter, or just writing a short script in an execution sandbox to count the letters that way.
Calling out LLMs for being poor at spelling is like challenging a colourblind person to say what colours a bunch of fruit are. They can often figure it out by other means but it’s more challenging than you’d think and it’s not a sign of poor intelligence if they get a few wrong.
Understanding the reason why an LLM is easy to trip up doesn’t really make it any less easy to trip up. The computer in Star Trek would have just given you the answer.
Except I also explained how modern LLMs get around that problem. They’re not actually that easy to trip up.
I also explained how they very famously and regularly don’t get around that problem. They remain pretty easy to trip up.
Famously, yes. Accurately, no.
This is like the “AI can’t draw hands” thing. It used to be a problem and was frequently called out as a tell or mocked, but most art generators do it fine nowadays and it isn’t called out so much any more. The strawberry problem will follow the same trajectory.
Well I suppose when that trajectory leads to a destination where they become less easy to trip up we can revisit this.