They have the option to use a local llm but default to online inference. Local is not good enough for most people and people would be really unhappy having their browser use 10gb of ram and spin their fans every query.
Minstrel is working on specialist small models. Maybe one day they are small enough to run in the background no worries.
Honestly im not sold yet on the AI in the browser meme but I see a lot of normies at my office say the cant live without google chromes AI integration. Maybe I just hate AI summaries.
My dream is AI navigating websites I hate like facebook marketplace and finding me deals i might like.
They have the option to use a local llm but default to online inference. Local is not good enough for most people and people would be really unhappy having their browser use 10gb of ram and spin their fans every query.
Minstrel is working on specialist small models. Maybe one day they are small enough to run in the background no worries.
i see, thanks for the information.
Honestly im not sold yet on the AI in the browser meme but I see a lot of normies at my office say the cant live without google chromes AI integration. Maybe I just hate AI summaries.
My dream is AI navigating websites I hate like facebook marketplace and finding me deals i might like.