Completely agree, I forgot to mention that part. I am testing a few models ranging from 18b to 26b on my 7900xt. It is far from “make this complete system”, but it can handle some smaller tasks. I think that will be the end goal anyway since cloud models fail a lot at maintainability, security, and other higher levels of thought that goes into coding. They can make a convincing prototype but I wouldn’t hook it up to production.
Local models are already functioning well as a force multiplier. It can help explain logic, do minor refactoring, debugging etc. but with a bit of latency. I do think this is where we’re headed since the frontier models required for generating a full prototype can’t make production quality code and it is prohibitively expensive to do so. As far as I’ve heard, they’re generally running spending ten times as much as they earn per token.
Completely agree, I forgot to mention that part. I am testing a few models ranging from 18b to 26b on my 7900xt. It is far from “make this complete system”, but it can handle some smaller tasks. I think that will be the end goal anyway since cloud models fail a lot at maintainability, security, and other higher levels of thought that goes into coding. They can make a convincing prototype but I wouldn’t hook it up to production.
Local models are already functioning well as a force multiplier. It can help explain logic, do minor refactoring, debugging etc. but with a bit of latency. I do think this is where we’re headed since the frontier models required for generating a full prototype can’t make production quality code and it is prohibitively expensive to do so. As far as I’ve heard, they’re generally running spending ten times as much as they earn per token.