• ☆ Yσɠƚԋσʂ ☆@lemmy.mlOP
    link
    fedilink
    arrow-up
    2
    ·
    4 hours ago

    It depends on what precision you run it at. For 4 bit, you need at least 16 GB VRAM and ideally 24 GB to be comfortable, for the full 16 bit you’d need at least 54gb. A RTX 4090 is minimum. I’ve got 64gb myself, so I can run 8 bit quant comfortably.