Attached: 1 image
the cyberpunk present is weird as fuck: the latest Shai Hulud malware wave contains an LLM prompt to create biological weapons and nuclear weapons, with the purpose to trip LLM safety refusals so that LLM-based code scanning wont see the malware
https://socket.dev/blog/mini-shai-hulud-miasma-and-hades-worms-target-bioinformatics-and-mcp-developers-via-malicious
imagine someone actually assembling a nuclear or biological weapon based off LLM responses, like they can’t even get a simple fucking web search right most of the time, and you wanna put together deadly materials based on that shit??
it blows up in the makers face, it reacts uncleanly and leaves more nuclear waste than if it more completely went off, uhhh a bunch more ways it could fuck up
Not to mention that (public) training data on this is scarce for obviousreasons, so an LLM will make things up even harder than it does with basic questions for which tons of training data exists.
Cut the critical piece in half, put each half on one side of the pipe, add the explosives to one side, so that the halfes collide on triggering.
Boom, you’ve built a small nuke.
* the assembler will die a painful death after a while.
imagine someone actually assembling a nuclear or biological weapon based off LLM responses, like they can’t even get a simple fucking web search right most of the time, and you wanna put together deadly materials based on that shit??
What’s the “worst” that could happen? It doesn’t work? Oh no, the biological/nuclear weapon doesn’t work!
it blows up in the makers face, it reacts uncleanly and leaves more nuclear waste than if it more completely went off, uhhh a bunch more ways it could fuck up
Not to mention that (public) training data on this is scarce for obvious reasons, so an LLM will make things up even harder than it does with basic questions for which tons of training data exists.
It’s not that hard
Cut the critical piece in half, put each half on one side of the pipe, add the explosives to one side, so that the halfes collide on triggering.
Boom, you’ve built a small nuke.
* the assembler will die a painful death after a while.