• ☆ Yσɠƚԋσʂ ☆@lemmy.mlOP
      link
      fedilink
      arrow-up
      2
      ·
      3 hours ago

      It depends on what precision you run it at. For 4 bit, you need at least 16 GB VRAM and ideally 24 GB to be comfortable, for the full 16 bit you’d need at least 54gb. A RTX 4090 is minimum. I’ve got 64gb myself, so I can run 8 bit quant comfortably.