the old Encyclopedia Britannica.

No one uses this.

You’ve got dozens of alternative Wiki competitors with their own rarified communities of editors and contributors.

No one uses them.

You’ve got subject-specific Wikis for a thousand different fandoms, hobbies, and crafts.

Fandom has a grip on this space, unfortunately, and that’s under Wales too. Yes, I know there are community alternatives, but their traffic/SEO is small in comparison.

But the idea that tens of millions of curious people and subject-experts are just going to endure a vacuum? Come on.

They absolutely will.

They already have!

Look at the rest of the internet. Literally every space, every niche, every congregation of experts has rotted away, and these “curious people and subject-experts” are clingimg to big social media toxic spaces because it’s the default.

It’s a very English-language and Euro-History oriented source of truth for tens of millions of online nerds. If it gets subsumed or shut down or otherwise undermined by western-aligned governments, I might suggest that the problem will not be an absence of Wikipedia but the presence of a malignant surveillance-enabled censorship bureaucracy.

You’re speaking of an internet that, for the bulk of humanity, doesn’t exist anymore. It is all but gone. That’s what I’m saying.

Maybe a few million nerds would create new Wikipedias, but the billions of other people on their phones and tablets would almsot never see it.

And that is dangerous for society. Wikipedia’s sheer critical mass is a huge monkey wrench in “soft” censorship, and for all its faults, every bit Wikipeida is delegitimized is a fascist dream-come-true.

And I hear a lot of people talking like that popularity isn’t important. There’s this sort of “old internet” attiude like we can just archive it and keep to ourselves. What does everyone else matter?

Well. Without Wikipedia putting the braKES on loss of freedoms in our IRL lives, we may lose the ability to do that.

Well, unfortunately, there is no project like that with Wikipedia’s critical mass. Most attempts with any attention are rather horrific. See: Grokipedia.

If Wikipedia dies, nothing good will replace it.

I know that’s a bit defeatist, but I really think its “the best we got.” Nothing better is coming with the state the internet is in now.

And I just want to emphasize that maybe a small community could build something better. I know.

But in terms of “keeping a healthy knowledge base for civilization,” that kind of project doesn’t matter if no-one sees it. Wikipedia is valuable because its a reasonably sane default reference for billions of people, that isn’t some megacorp or government.

I think you missed my point.

I’m not worried about them running out of funding. I’m worried about their reputation in the public’s eye.

Duct it!

I have a 400W 3090 with zero case fans.

Its always cool, because I bought like $10 of weather stripping to “seal” its intakes against the edge of an SFF case. So it’s always sucking in ambient air:

You can also severely undervolt a 5090 to like 250-300W, with very little performance impact. Honestly its stock speeds are kind of crazy.

…But again, you aren’t gaining much over a 4090. A 4090 + DDR4 threadripper would be way faster than a 5090 + occulink, for many reasons. And TR CPUs are pretty reasonably priced compared to GPUs these days.

But even if you do go 5090, I’d highly recommend finding some way to shove it in the case and duct some air into its intakes. Its going to be way faster on a PCIe slot.

You could even get a riser and mount it somewhere else in the case, theoretically.

This is bad.

…But I’m very worried about the future of Wikipedia.

Questionable leadership or not, it is a lighthouse in the information dystopia of the 2020s, which literally helps hold society together.

The far right wants it gone, and are not shy about screaming it, because Wikipedia is not compatible with fascism.

If it becomes hated by the left too, and ignored by the masses because of Big Tech blacklists it, I worry it won’t have the political support to keep its prominence.

And to be clear, I’m not worried about it dying. Wikipedia will stay where it is for decades.

I’m worried about it ceasing to be popular.

It doesn’t function as a uncontrolled source of information if no one sees it. Modern fascist censorship works by simply shifting attention.

First of all, I mean zero offense with any purchase decision. A 5090 is very good.

…But if I were paying that kind of money, I’d probably get a 4090 and a new motherboard/CPU instead. Maybe a used DRR4 threadripper system.

Hybrid (CPU + GPU) inference is where it’s at these days. It opens up a whole world of huge MoE models, whereas on an 5090 you are stuck with Qwen 27B.

Having a fast CPU, with lots of RAM channels, with full PCIe bandwidth is much more important for that than having a 5090, where a 3090 or 4090 will get the job done.

Even if pure speed is your primary concern, you can tune a sparse 120B model (like Laguna) to run almost as fast as Qwen 27B on a 5090, and get at-least-good results.

It’s more finicky and involved, though. For sure.

Running an LLM on a 5090 is a task. Hybrid CPU + GPU inference is a hobby.

Be aware that you pay a significant performance penalty for going over occulink.

The hit modest for fully offloaded dense models (like Qwen 27B), but dramatic for hybrid inference of big MoEs.

Even my old 3090 got a noticeable performance gain going from a PCIe 3.0 x16 riser to a PCIe 4.0 one.

• It’s just under 300B, trained at FP4; absolutely the perfect size for servers with 128GB-192GB CPU RAM to spare.

• Its fast. I’m getting 17 tokens/sec on a single RTX 3090 GPU, all experts offloaded to RAM; for a 300B model this smart, that’s crazy fast.

• Its attention mechanism is cutting edge, good for long context without too much processing time.

• The benchmarks for coding/agenic usage are absolutely bonkers, within margin of error of frontier models or Deepseek Pro in some cases: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731

• …Though I don’t put much stock in benchmarks. And I haven’t tested it enough to tell you if it lives up to that hype for specific use cases.

• Deepseek also publishes its base model. That means I can “unfry” the model with a merge if I have to.

I am afraid of the the model being “overfit” to coding and agenic stuff.

For reference, my previous favorite model was Xiaomi MiMo V2.5 310B. It benched well, but it also feels “smart” outside of benchmarks, like in knowledge of trivia without tool usage/internet access or comprehension of weird questions.

[AIT] I know this isn’t everyone’s cup of tea, but I’m excited about Deepseek V4 flash. It’s (for me) the perfect size and architecture to self-host an LLM.

My box (and brain) are chugging through a queue:

• Figure out why my swap is going crazy, and how to ban processes from it [Done].

• Figure out why Code OSS is unhappy [Partially Done].

• Make an ik_llama.cpp iMatrix for Deepseek V4 [Done].

• Figure out why quantization isn’t working [Done].

• Make a test IQ2_KL/MXFP4_R8 quant to see how it does squeezed onto my box [in progress].

• Test. Tune. Inevitably troubleshoot the dozen other things that go wrong. Figure out how much spare RAM that leaves me.

• Make a higher quality IQ3_KT quantization. This will take all night on my CPU.

• KLD test both of them vs the full precision, to quantify quantization loss. Likely an overnight task, too.

• Try merging the new model release with the base model, 50/50, for a less “deep fried” model. imatrix, quant, test.

The goal is to host it on a single RTX 3090, Ryzen 7000 with 128GB RAM, for anyone curious. Though I may try smaller models too, like Laguna S1.

No.

Why would they? “Moderate” Republicans would get zero engagement on anything, and if they’re way down the MAGA rabbit hole they are on different platforms to indulge in that.

People here label “a particular liberal/leftist that’s not in my political tribe” as Republican all the time, though. It’s honestly mind boggling to me. Lemmy is in like the 99th percentile of the leftism scale in the US.

I use an exl3, with 4 bit MLPs but higher bit depth attention layers. And I force some custom sampling so I can lower the temperature a bit while keeping it out of loops.

This won’t work in LM Studio though. You have to run such a thing in TabbyAPI or some other backend that supports exllamav3.

It’s kind of insane that integrated wikis with citations/documentation aren’t the centerpiece of (Reddit) communities.

Instead, we got… Discord?

It’s a bit misleading.

Qwen 27B has way less “world knowledge” than GPT-5. Ask it random trivia without internet search access, and GPT would know waaay more.

This is generally true of small vs large models.

…But honestly, Qwen 27B is better at tool use or agentic stuff. It’s hyper optimized for just that and coding assistance, basically.

This is often true of old vs new. Most newer models have hyper focused on agents/coding, often to the detriment of other use cases.

Quantization for practically running Qwen 27V also has an impact. A off-the-shelf Q4_K_M is not the same as the unquantized weights in real-world use, or even an “optimized” quantization like a custom exl3.

Also, AI or not is really irrelevant for this case.

They said as much:

Saflor uses AI tools and does not believe that AI itself is the problem. Goldman pointed out that his arguments against Memes Apps would be largely the same even without the AI aspect. But Saflor considers Memes Apps’ platforms to be examples of irresponsible AI products, where the operators problematically advertise that you can “fire your ad agency” and replace all creative work with a meme generator.

Which sounds to me like effectively abolishing the copyright system for anyone who can’t afford paying lawyers to hunt down every little infringement, only leaving it up for wealthy corporations.

And yeah, that would be a really terrible precedent…

Well, the current state is that there are a couple of “best” models, but literally hundreds of independent providers serving them. As an example, one can get GLM 5.1 from its trainer, or one can get it from DigitalOcean, or Baidu, or SiliconFlow ASICs, or get it at very high speed from Cerebras ASICs, or AMD providers, or finetune it from a number of services, or rent the self hosting…

The companies aren’t keeping the models to themselves, and that blows the marketplace open to a boatload of competitors.

They don’t have to host it themselves. They could use a number of providers for the same model, and basically keep doing whatever they were doing with OpenAI/Anthropic via the exact same APIs.

Bonsai? Or whatever it’s called? It’s a con, so far; it’s not better than smaller models quantized to 3-4 bits.

I love, love the idea of bitnet, but it only seems to work with models trained from scratch, which no one has done at scale yet.

You want this one:

https://huggingface.co/turboderp/Qwen3.6-27B-exl3_3.30bpw/tree/main

Or maybe the 3.5bpw one if you don’t mind less context, or 3bpw if you need more:

https://huggingface.co/turboderp/Qwen3.6-27B-exl3

For faster inference at the cost of a little more VRAM usage:

https://huggingface.co/turboderp/Qwen3.6-27B-DFlash-exl3

And you run those in:

https://github.com/theroyallab/tabbyAPI

And FYI, if you have 64GB of RAM or more, you might consider hybrid inference instead.

I also forgot to emphasize this, but Xiaomi’s plan, in my opinion, is undiscovered fruit.

GLM had a similar coding plan, but once it got in the news and popular, it got WAY more expensive and limited. I’m grandfathered into 6 more months of a GLM plan you literally cannot buy now.

And I think Xiaomi is in the same situation GLM was 6+ months ago. It’s a fantastic model series, but unlike Kimi/GLM no one knows about it yet, which is how it’s still $60 for a year.

+1 for Hermes.

If you have a newer Nvidia GPU, you can run Qwen 27B via exllamav3 and get good quality/speed in 16GB. And it’s worth the trouble, as 27B is an amazing model.

If it’s AMD, yeah, a3B is a good bet, depending on how much spare CPU RAM you have.