I’m not sure, if I understand the environment completely

Those agents were the virtual incarnations of the AI in the sim city and the respective government - correct?
And the AI needed to take care, that those agents didn’t died, like of hunger or what?
That’s not really what those LLMs are trained for.

Not sure, what they expected

Currently searching the article for the original source, maybe this gives more insight

Edit: ah, just in the first paragraphs it is
https://www.emergence.ai/blog/emergence-world-a-laboratory-for-evaluating-long-horizon-agent-autonomy
Completely missed it on the first read.
Let’s see if this makes more sense…

Edit 2: ok, if I get this right, those agents really were specific virtual individuals
Not sure what they expected. First, LLMs are not really build to “live” as an individual as they aren’t real intelligence and can only role play individuals based on their training data.
Second, why should they be super moral or “better”?
Again, they just role play depending on their training data and built-in prompt bias (not sure what the prompt injection of the company is called)

If you train an AI on governing such a world, it probably start gaming the system, depending on what values are important to “win”
As we have already seen with machine learning in the last decade(s?)

Funny experiment nevertheless, but not really useful in my eyes - and I’m everything but a defender of the current use of LLMs