If you want https it’s gotta be a external DNS record anyway, might as well point to internal ips

Depending on how paranoid your setup is not everything will use the DHCP provided DNS servers

That rack is a thing of beauty.

14c/20t CPU

How does that work? 1.4 threads per core?

Nearly everything is PoE as the gods intended

Do you have that many poe devices?

Adding an OPNsense Firewall soon, just to mess around with some sort of dynamic firewall capabilities.

That sounds cool, but can’t you do most of that with the mikrotik router you have already?

You have a fiber uplink leaving the switch, does it go to the hypervisor?

I’m very happy with it; it works well, has good control of different projects, allows limited api access to different vlans; supports clustering and live migration. I haven’t found anything I can’t do with incus that I can do with libvirt. it is surprisingly mature.

I do have a second machine, so I have juggled VMs around when i played with different hypervisors. You can always take a disk image of your home assistant and restore it in incus.

oh, i love the colours! Are your mini-pcs sagging in the middle? I think you might need a shelf!

The mikrotik gets external internet access from the while cable? You have a patch panel - that’s awesome.

128GiB of ram across your HPs! nice! What do you use all that memory for? 250 pods is impressive.

Why is the NAS sitting on foam? vibration dampening?

My UPS is only used to shutdown the NAS gracefully, if i lose power the whole thing goes off in 30s.

I really want to rebuild the NAS and build a beefy worker with a GPU but component pricea are just absurd.

Yeah, the intel arc gpus are not so bad, and they support sriov!

also running VMs on your NAS?

That is awesome! it’s not a mess - it’s lovingly artisanal home made spaghetti!

that little chuwy is running vms? what kind of cpu do you have in there?

KVM behind the psp - is that a keyboard video mouse switching device behind your handheld gaming console?

it’s all magic, don’t let that magic smoke out!

Not sure, but its about 240W right now (doing lots of nonGPU work)… the full stack all the devices (including the spinning rust and the printer)

I like the whale tail! The chuwi is a low power pc? The clear plastic behind the speaker is another? Your doing wifi and wired right next to each other? KVM behind the PSP? NAS on the right?

Fair enough - how would you describe it?

vlans don’t route to each other, you need a device that routes between vlans (if you want traffic to cross)

Why have one when you can have two for just twice the price?

I always have at least two now, outages on one are a curiosity rather then a touch grass event.

yeah, its just a gateway router with multiple wan inputs (up to 3), it does dhcp, routing, vpn termination, and manages all the vlan segmentation and cross routing.

The hypervisor is just nixos running kvm and incus - it doesn’t have any opinion about pxe booting; So pxe gets dhcp from the san, which supplies the bootstrap, which is a tpm wrapped luks volume, which supplies the iscsi target, which bootstraps nixos over iscsi… Once the nixos kernel loads anything goes, so KVM isn’t aware of the iscsi nature of the diskless system at all.

I did play with nixos -> xen, but nested virtualization wasn’t a thing with xen yet, so i went back to KVM. incus over libvirt so i can have a proper hypervisor API, so i can have projects with different credentials and scopes creating and destroying VMs willy nilly.

A central proxy adds a central point of failure.

However, if you give each service its own DNS entry, your proxy can look at the incoming DNS of the request, and use that to proxy to the service. At least for http services

I think it’d be much cleaner, to give each VM its own DNS entry, run the service on Port 80, or 443 so you don’t have to remember port numbers.

I’m not sure the reverse proxy is buying you much ease of use for a totally internal system. If you like, you can have a central web page, that links to all your different services from one location.

we are kindred spirits! What do you keep in the safe? RAM? is that a room vent fan running off a motherboard fan header? What do you use the DVD drive for?

by keeping the cats at other people’s houses! works a treat.