fubarx@lemmy.world to Technology@lemmy.worldEnglish · 3 days agoSam Altman says OpenAI will not go public this year as AI fears growwww.sfchronicle.comexternal-linkmessage-square77linkfedilinkarrow-up11arrow-down10file-text
arrow-up11arrow-down1external-linkSam Altman says OpenAI will not go public this year as AI fears growwww.sfchronicle.comfubarx@lemmy.world to Technology@lemmy.worldEnglish · 3 days agomessage-square77linkfedilinkfile-text
minus-squaresetVeryLoud(true);@lemmy.calinkfedilinkEnglisharrow-up0·2 days agoNah I got an RX 6800 XT for $300 with 16 GB VRAM, perfectly cromulent for a 27B model.
minus-squareSquizzy@lemmy.worldlinkfedilinkEnglisharrow-up0·2 days agoThats is pretty decent alright, I had heard 24 was what was needed for decent local AI.
minus-squaresetVeryLoud(true);@lemmy.calinkfedilinkEnglisharrow-up0·2 days ago24 is ideal, 16 works fine especially with some MoE offloading. If you use llama-swap, a fast PCIe bus and fast storage is ideal. Takes about 20 seconds to swap between two models on my X570 board.
Nah I got an RX 6800 XT for $300 with 16 GB VRAM, perfectly cromulent for a 27B model.
Thats is pretty decent alright, I had heard 24 was what was needed for decent local AI.
24 is ideal, 16 works fine especially with some MoE offloading. If you use llama-swap, a fast PCIe bus and fast storage is ideal. Takes about 20 seconds to swap between two models on my X570 board.