I bought a datacenter GPU that doesn't fit in a normal motherboard, macgyvered the fan with jumper wires, and now I'm running a model that ties with Claude Sonnet 4.6 on benchmarks, all for £200.
    • melfie@lemmy.zipEnglish
      3·
      1 month ago

      Good point, that should theoretically be double the decode speed, though not sure how the prefill would differ since it’s mainly compute-bound.