These are not watermarks using hidden characters. Their approach is undetectable even with an IDE. For a sequence of tokens, an LLM predicts the most likely next token, with some amount of randomness between equally likely candidates. The “watermark” is to introduce a statistical bias to this randomness, by altering the probability distribution of generated text according some hash function with a secret key, thereby embedding a statistical signature into the text itself.
So if the text is “I like to eat __” the model might have 3 top candidates for the next word (apple/orange/banana) that would be chosen at random. Instead that choice will be biased towards one option according to their hash function. And then again “I like to eat banana __” (cake/pie/tart).
To verify a text, they look for the “watermark” by scanning the text and looking at whether sequences of tokens chocies fits their biased probability distribution or are truly random. Just one match doesn’t tell you anything, but if they see a consistent pattern over a 1000 word document, they can give a very high confidence that their model generated the text.
To a human it looks like nornal generated text, and the output quality isn’t affected much (or ar all). It’s much more effective on generated prose, and not very effective on computer code.
I mean it kind of already does this accidentally, just look for em dash and/or emojis sprinkled all over the document and you can be pretty certain that it was AI generated.
The other thing I’m seeing here is that this will only be effective for large chunks of text, if you’re dealing with small snippets interspersed with human generated content there will be enough statistical noise to make classification hard without introducing a bunch of false positives and negatives.
These are not watermarks using hidden characters. Their approach is undetectable even with an IDE. For a sequence of tokens, an LLM predicts the most likely next token, with some amount of randomness between equally likely candidates. The “watermark” is to introduce a statistical bias to this randomness, by altering the probability distribution of generated text according some hash function with a secret key, thereby embedding a statistical signature into the text itself.
So if the text is “I like to eat __” the model might have 3 top candidates for the next word (apple/orange/banana) that would be chosen at random. Instead that choice will be biased towards one option according to their hash function. And then again “I like to eat banana __” (cake/pie/tart).
To verify a text, they look for the “watermark” by scanning the text and looking at whether sequences of tokens chocies fits their biased probability distribution or are truly random. Just one match doesn’t tell you anything, but if they see a consistent pattern over a 1000 word document, they can give a very high confidence that their model generated the text.
To a human it looks like nornal generated text, and the output quality isn’t affected much (or ar all). It’s much more effective on generated prose, and not very effective on computer code.
I mean it kind of already does this accidentally, just look for em dash and/or emojis sprinkled all over the document and you can be pretty certain that it was AI generated.
The other thing I’m seeing here is that this will only be effective for large chunks of text, if you’re dealing with small snippets interspersed with human generated content there will be enough statistical noise to make classification hard without introducing a bunch of false positives and negatives.