I’d much rather have a more powerful generic CPU than a less powerful generic CPU with an added NPU.
There are very few people who would benefit from an added NPU, ok I hear you say what about local AI?
Ok, what about it?
Would you trust a commercial local AI tool to not be sharing data?
Would your grandmother be able to install an open source AI tool?
What about having enough RAM for the AI tool to run?
Look at the average computer user, if you are on lemmy, chances are very high that you are far more advanced than the average computer user.
I am talking about those users who don’t run Adblocker, don’t notice the YT ad skip button and who in the past would have installed a minimum of of five toolbars in IE, yet wouldn’t have noticed the reduced view of the actual page.
These people are closer to the average users than any of us.
Google Coral is probably the most famous and is mainly suited for small IoT devices, e.g. speeding up image recognition for security cameras. They come in all shapes and sizes though.
The fact that i didnt know about those means that consumers have zero need for them and building them into consumer hardware is just an attempt to keep the AI bubble afloat.
There’s also the fact that many NPUs are pretty much useless unless used for a very specific model built for the hardware, so there’s no real point having them
My understanding from a very brief skim of what Microsoft was doing with Copilot is to take screenshots constantly, run image recognition on it, and then make it searchable as text and have the ability to go back and view those screenshots in a timeline. Basically, adding more search without requiring application-level support.
They may also have other things that they want to do, but that was at least one.
EDIT: They specifically called that feature “Recall”, and it was apparently the “flagship” feature of Copilot.
Do you mean Copilot, the local indexer and search tool or do you mean Copilot the web based AI chat bot or do you mean Copilot the rebranded Office suite or do you mean … etc.
Seriously, talk about watering down a brand name. Microsoft marketing team are all massive, massive fuck knuckles.
Probably not even general purpose GPUs, although we sucked it up when RT and Tensor cores were put on a plate whenever we like it or not. These though at least provided something to the consumer unlike NPUs.
I’d much rather have a more powerful generic CPU than a less powerful generic CPU with an added NPU.
There are very few people who would benefit from an added NPU, ok I hear you say what about local AI?
Ok, what about it?
Would you trust a commercial local AI tool to not be sharing data?
Would your grandmother be able to install an open source AI tool?
What about having enough RAM for the AI tool to run?
Look at the average computer user, if you are on lemmy, chances are very high that you are far more advanced than the average computer user.
I am talking about those users who don’t run Adblocker, don’t notice the YT ad skip button and who in the past would have installed a minimum of of five toolbars in IE, yet wouldn’t have noticed the reduced view of the actual page.
These people are closer to the average users than any of us.
Why do they need local AI?
Just offer NPUs as PCIe extension cards. Thats how computers used to be and should be. Modular and versatile.
Already existed for half a decade.
Google Coral is probably the most famous and is mainly suited for small IoT devices, e.g. speeding up image recognition for security cameras. They come in all shapes and sizes though.
M.2 Accelerator A+E key | Coral - https://www.coral.ai/products/m2-accelerator-ae
The fact that i didnt know about those means that consumers have zero need for them and building them into consumer hardware is just an attempt to keep the AI bubble afloat.
Exactly!
I could even see the cards having ram slots, so you can add dedicated ram to the NPU to remove the need for sharing ram with the system
There’s also the fact that many NPUs are pretty much useless unless used for a very specific model built for the hardware, so there’s no real point having them
My understanding from a very brief skim of what Microsoft was doing with Copilot is to take screenshots constantly, run image recognition on it, and then make it searchable as text and have the ability to go back and view those screenshots in a timeline. Basically, adding more search without requiring application-level support.
They may also have other things that they want to do, but that was at least one.
EDIT: They specifically called that feature “Recall”, and it was apparently the “flagship” feature of Copilot.
Do you mean Copilot, the local indexer and search tool or do you mean Copilot the web based AI chat bot or do you mean Copilot the rebranded Office suite or do you mean … etc.
Seriously, talk about watering down a brand name. Microsoft marketing team are all massive, massive fuck knuckles.
Hey, the last one is great.
Now when I get asked “what do you think about Copilot,” I can just say, “I prefer LibreOffice”
cmiiw but I don’t think NPUs are meant to be used on general-purpose personal computers. A GPU makes more sense.
NPUs are meant for specialised equipment e.g. object detection in a camera (not the personal-use kind)
They are in general purpose PCs though. Intel has them taking up die space in a bunch of their recent core ultra processors.
That’s stupid.
Probably not even general purpose GPUs, although we sucked it up when RT and Tensor cores were put on a plate whenever we like it or not. These though at least provided something to the consumer unlike NPUs.