• MrQuallzin@pie.eyeofthestorm.place
    link
    fedilink
    English
    arrow-up
    8
    arrow-down
    1
    ·
    3 hours ago

    I’ve got an old 1060ti in my server. Ollama shares it with just a couple other containers. Electricity here is majority hydro with some natural gas, $0.08/kWh.

    It’s a little slow, but I can comfortably run qwen3:14b. Of course that’s not all done on the GPU, a large part is offloaded to server ram (generally 32GB available so more than enough headroom)

    My server and my gaming PC combined last month came out to $13.32

    • ag10n@lemmy.world
      link
      fedilink
      English
      arrow-up
      3
      ·
      edit-2
      3 hours ago

      How does that compare to closed models that Anthropic offers, at the context and scale they offer.

      I run Qwen3.6 27B locally and it’s usable with 16G vram but still not the same as a data centre of Blackwell clusters.