"It really seems like anyone with some renders and a white paper written by someone being gassed up by an overly agreeable AI can get VC funding these days."
There is an unsolvable compute problem. The average PC on earth has multiple bit-flips a year from cosmic rays. The space hardened chips we use are 50nm and the chips used from inference are 4 to 6nm. 50nm is far more cosmic ray resistant than 6nm because of the transistor size. Are we supposed to think making H100s with a 65nm process is possible? The speed of light creates a die size limitation as well.
The way I see it is they are doing inference, not transfiring bank account balances. I’d be curious to see some actual experimental data, but I’d expect LLMs to skip past bit flips same way you shrug and move on from spelling errors. At worst you can do your critical calculation in triplicate on your 6nm nodes (with redo upon dissensus) and reduce your bit error from 4/year (or 4000/year or whatever have you in orbit) to (4/year)^3
There is an unsolvable compute problem. The average PC on earth has multiple bit-flips a year from cosmic rays. The space hardened chips we use are 50nm and the chips used from inference are 4 to 6nm. 50nm is far more cosmic ray resistant than 6nm because of the transistor size. Are we supposed to think making H100s with a 65nm process is possible? The speed of light creates a die size limitation as well.
Either that or they’d have to triplicate everything and have a voting system.
this is the current real solution on the ISS - three main computers doing everything.
The way I see it is they are doing inference, not transfiring bank account balances. I’d be curious to see some actual experimental data, but I’d expect LLMs to skip past bit flips same way you shrug and move on from spelling errors. At worst you can do your critical calculation in triplicate on your 6nm nodes (with redo upon dissensus) and reduce your bit error from 4/year (or 4000/year or whatever have you in orbit) to (4/year)^3