A global workspace in language models
www.greaterwrong.com
[This is the blog post for our new paper Verbalizable Representations Form a Global Workspace in Language ModelsReaders might also be interested in: the Public commentary, Github and Neuronpedia] As you read this sentence, circuits in your brain are adjusting your posture, controlling your breathing, and transforming lines and curves on the screen into recognizable words. Most of this processing is invisible to you. But some of what takes place in your brain you do have access to—an image that pops into your head, or a deliberate plan you make about where to go shopping. Neuroscientists and philosophers sometimes refer to the latter type of brain activity as “consciously accessible,” to distinguish it from all the other processing that goes on unconsciously. This activity has special properties: we can describe it, control it, and use it for deliberate reasoning, in contrast to all the automatic processing that goes on without our awareness.

cross-posted from: https://piefed.world/c/tech/p/1246129/anthropic-uncovered-claude-s-consciousness-like-workbench-the-mysterious-j-space-hides-h

As you read this sentence, circuits in your brain are adjusting your posture, controlling your breathing, and transforming lines and curves on the screen into recognizable words. Most of this processing is invisible to you. But some of what takes place in your brain you do have access to—an image that pops into your head, or a deliberate plan you make about where to go shopping. Neuroscientists and philosophers sometimes refer to the latter type of brain activity as “consciously accessible,” to distinguish it from all the other processing that goes on unconsciously. This activity has special properties: we can describe it, control it, and use it for deliberate reasoning, in contrast to all the automatic processing that goes on without our awareness.

In a new paper, we present evidence that a similar distinction has emerged in modern language models like Claude. We find that Claude has developed a small collection of internal neural patterns that, compared to all its other internal processing, play a special role.

We call the collection of these patterns the J-space—named after the technique we used to find them, involving a mathematical concept called the Jacobian. Each J-space pattern is linked to a particular word. But when one of these patterns lights up, it doesn’t mean the model is saying that word—just that the word is on its mind. If you’ve heard of language models having a “scratchpad” or “chain of thought”—text they write to themselves while reasoning—the J-space is something different. It operates silently, in the model’s internal neural activations, allowing the model to think about a concept without writing it down. Notably, the J-space wasn’t designed or programmed by us, but instead emerged on its own during Claude’s training process.

  • masterspace@lemmy.caEnglish
    13·
    3 days ago

    If you are suggesting that LLMs are sentient, then this isn’t a “mild disagreement”, it’s the San Andreas fault.

    “If you are suggesting X is black then we’re mortal enemies”

    Theres more nuance then black and white, don’t be so internet brained.

    There are so many technological leaps that have to happen before AGI can become real that I am certain that neither of us will see that day.

    Name them if you’re so confident.

    We’re already operating models with simulated neurons on the order of magnitude as brains. Unless you plan on dying this decade or the next, there’s a reasonably likely chance we’ll live to see AGI.

    No one has claimed this is sentient, but that doesn’t mean that it’s not interesting, nor does it mean that it’s not interesting that a structure that seems to function similarly to one in our brain emerged.simply from the way they trained it.