Recent post re: AI as utility
Myself, I’m a fan of local LLM / self hosted ML… but if you ever needed a clarion call that a hard pivot is coming (soon) for online/ cloud based AI…Altman et al are making some concerning mouth noises (to say nothing of broader concerns with OAI, Anthropic etc).
Right now, I’m sketching out a plan where my Raspberry Pi (always on, 2-3w) uses a magic packet to wake up my modest AI server (Lenovo P330 with Tesla P4) if/when needed (Qwen 3.6-35B-A3B); no point in chugging down 80-100w, 24/7 for no good reason.
If the trend continues the direction it appears to be (increasing costs, environmental impacts etc) then I’d feel a lot better hosting my own as port of first call and replacing simpler tasks with more traditional programs. YMMV.
If you’re already using node-red, the Wake On Lan node works well, and with node-red it’s easy to trigger the magic packet based on whatever trigger condition you want.
The only limitation I know is WOL doesn’t work after a power outage, because the switch and RPI doesn’t know where to find the target machine
Thanks for the tips on reusable enterprise cards btw
maybe, but the pi does not need to know that, only the mac address and the interface. the switch doesn’t need to know either because it’s a broadcast frame, it’s forwarded to all cables. the problem sometimes is that if you configure WOL from linux, the network adapter will probably forget on power cycling that it is supposed to react to magic packets. I think not all hardware is susceptible to that, but even then it could help to configure WOL in the BIOS
@[email protected]
Maybe something else going on then, but ive never gotten WOL to work after a blackout when there’s two switches between sender and receiver. After powering up the receiver once, WOL works again
Good tips - thanks!
PS: sad to report the 24GB Tesla p40s are now around $250 USD on eBay, so not quite as cheap as I remembered. P4s are still cheap tho, though frankly if you’re going that end of town, a 1080 is about on par, less fussy and probably cheaper - it just won’t fit in a uSFF.