I would 100% prefer a LLM that is purely deterministic and repeats the same answer exactly to the same question

Then you don’t like your LLM to solve problems, because a lot of how they solve problems is trying, testing, failing, then trying again - getting a different answer and testing that…