I run my LLM locally, and still have to turn the heating on because it’s not enough power. A high end card is normally rated for about 300W - and it’s only running in short bursts to answer questions. so if you are really pushing it, over time you will probably reach around 150W/h - that’s not enough at all. You would for sure use more power playing a game using Unreal Engine 5.
Power consumption of LLM’s is a lot lower than people think. And running it in a data center will surely be more energy efficient than my aging AM4 platform.
I run my LLM locally, and still have to turn the heating on because it’s not enough power. A high end card is normally rated for about 300W - and it’s only running in short bursts to answer questions. so if you are really pushing it, over time you will probably reach around 150W/h - that’s not enough at all. You would for sure use more power playing a game using Unreal Engine 5.
Power consumption of LLM’s is a lot lower than people think. And running it in a data center will surely be more energy efficient than my aging AM4 platform.
What is W/h? twitch
I run mine on a Steam Deck.
Fairly low power draw on that lol.
Though I’m using it as a coding assistant… not a digital girlfriend.
… though I have modded my Deck a bit, so… I guess I already know what ‘she’ looks like on the inside, hahaha!