“It” is very ill defined. The AI they are selling you was never supposed to cure cancer. It was meant to make money. The AI which cures cancers is of a completely different kind than LLMs and media generation. And it does not have any of the same drawbacks nor any of the same funding despite it’s far greater usefulness.
The AI which cures cancers is architecturally pretty similar to LLMs in that it’s based on transformers and GPT in particular is also based on transformers (that’s the T in GPT). It is also generative AI, you give it tokens that describe proteins and it generates folding structures. It’s like an LLM except the tokens aren’t words or syllables like in LLMs.
It’s used very differently and the energy cost is a few orders of magnitude less (you can run it on a single H200 with 141 GB VRAM for ~9000 tokens or an A100 or H100 for ~5000 according to this university and it even has a slow CPU/IO heavy first stage preparing everything so they can minimize the GPU time in stage 2).
There is yet hope that such AI applications will eventually benefit from all the research that’s been put into LLMs. Obviously not everything will carry over, but there may be architectural improvements to be made in research models too.
Yeah, and any “it will cure cancer” usually boils down to “uhmm, if we feed it enough training material, it will turn into a sentient superintelligence, which will cure cancer”.
“It” is very ill defined. The AI they are selling you was never supposed to cure cancer. It was meant to make money. The AI which cures cancers is of a completely different kind than LLMs and media generation. And it does not have any of the same drawbacks nor any of the same funding despite it’s far greater usefulness.
The AI which cures cancers is architecturally pretty similar to LLMs in that it’s based on transformers and GPT in particular is also based on transformers (that’s the T in GPT). It is also generative AI, you give it tokens that describe proteins and it generates folding structures. It’s like an LLM except the tokens aren’t words or syllables like in LLMs.
It’s used very differently and the energy cost is a few orders of magnitude less (you can run it on a single H200 with 141 GB VRAM for ~9000 tokens or an A100 or H100 for ~5000 according to this university and it even has a slow CPU/IO heavy first stage preparing everything so they can minimize the GPU time in stage 2).
There is yet hope that such AI applications will eventually benefit from all the research that’s been put into LLMs. Obviously not everything will carry over, but there may be architectural improvements to be made in research models too.
Yeah, and any “it will cure cancer” usually boils down to “uhmm, if we feed it enough training material, it will turn into a sentient superintelligence, which will cure cancer”.