anotherandrew

Kind of a piggyback on this – is there a favoured “search my saved links from karakeep/linkwarden/hamster.io and show them first before feeding the search terms to google/bing/whatever” extension for your browser?

How did you do this? I have only seen the google postmaster tools and they’re absolutely useless unless you are sending significant email volume. If you’re a little guy they won’t even give you basic reporting on deliverability.

There was a recent thread on reddit about this, where I wrote this comment (copied here):

I’ve been hosting my own email for a long time (almost 25 years).

Today it’s better than it was, but there are some hurdles:

• Microsoft has their own system, but it’s reasonably easy to get listed
• Google does their own thing, and it’s IMPOSSIBLE to get anywhere
• UCEPROTECTL3 is just a fucking extortion scam

When I switched providers, I found out I was in a “bad IP neighbourhood”. Microsoft wanted a letter from my VPS provider saying that I am in control of the IP I wanted listed, and that was not too hard to get. Also, Microsoft’s blacklist management is sane - you can log in, see the status, raise issues and get a hold of people. A little frustrating, but workable.

Google, on the other hand… You can’t participate in their spam system unless you have a minimum volume of email, which means little guys like me who send maybe 50-100 emails a day end up in gmail’s junk folders by default and there’s abso-fucking-lutely nothing you can do about it. There’s no one to report it to, there’s no way to fight it… they simply don’t care. And whether an email gets flagged as junk or not seems completely random. It has nothing to do with the content as far as I can tell. All you can do is contact people from your personal gmail and ask them to check spam/whitelist. It’s been years and I’m still waiting for the “eventually your domain will get whitelisted globally” bullshit to happen.

That leaves UCEPROTECTL3. Fuck these guys sideways. They block entire ASes and no, you can’t get an exception made. You can pay them to get whitelisted which is why I call them an extortion scam. They’re the only blacklist I’m on and I’ll be fucked if I’ll pay them to get off it. Bunch of fucking pretentious scammers.

Everything else is pretty easy: DNS, DMARC, DKIM, SPF… it’s hoops to jump through but not overly difficult. Ensuring you’ve got SMTPS set up and constraining the encryption protocols to get it tight takes some iterative work, but nothing too difficult.

I totally understand why people give up. This is a huge problem with these gigantic monolithic companies – they hold way too much power over the internet and there’s no way to hold them accountable.

I’ve been selfhosting various things for almost 25 years now. Started with email/web, but now I’ve got the following (in no particular order):

• email (postfix/dovecot)
• web (nginx)
• shared notes (obsidian, but also through dovecot)
• calendar (davical)
• telephony (asterisk)
• replicated storage (syncthing)
• media server (plex)
• home automation (homeassistant, mosquitto, grafana, influxdb)
• power monitoring (empora device on the breaker panel + a few smart outlets talking to homeassistant)
• security cameras (securityspy)
• irrigation (a controller of my own design, adding OpenSprinkler support this year)
• offsite backups (duplicity + rclone)
• project management/issue tracking (redmine)
• social media (gnu-social + lemmy, but also testing mbin)
• bookmark management (karakeep)
• local copies of web stuff (yt-dlp, hamsterbase, singlefile)
• VPN (openvpn)

Virtualization is mostly docker containers, but also some ESXi/VMWare Fusion. I also have Obsidian in the mix but that’s not really a self-host but more of a way to organize/access my data. I have also been doing a (very!) little bit of experimentation with local LLMs, but it’s all on ARM, using either the GPU or the NPU available on the RK3588.

This stuff either exists on an OVH VPS for the “internet facing” stuff or on an old Dell C6100 blade server. ESXi uses one blade and another blade runs Debian and talks to an old SATA/SAS disk shelf I got for $50 to see if I could make it work (it was super straightforward). I have a bunch of 2T and 4T “spinning rust” drives in two RAID6 arrays (mdadm) and then carve out storage for various things using LVM. I am experimenting with zfs on the VPS but am not a big fan of it. I used to run OpnSense on another blade since I couldn’t find a router which would properly shape gigabit internet traffic, but now I’m using an ER605 and it seems to be doing quite well. I have a tiny KeepConnect device which will physically cut power to the cable modem if it can’t see the internet which is very helpful since the biggest source of trouble for me has always been the damn internet service doing weird things when I’m not at home.

I’ve even been working toward “self hosting” my own educational electronics stuff for my kids using https://microblocks.fun (the actual project is called smallvm) - think scratch running completely in the browser and executing code on a “vm” which is actually running on a microcontroller over BLE or serial.

This sounds like a shitload of work and sometimes it can be, but one of the best parts of self hosting is that once it’s set up, it hardly ever has to be updated/changed. Security updates are the biggest reason of course, but a LOT of this is not on the open internet so I can be more lenient about keeping things up to date. I also try to keep everything that needs a database to use ONE database (postgres), which also makes it easier to back up or use data from several tools in a new way. Honestly it’s largely fire and forget these days. I add more space or replace drives as needed and try not to touch things otherwise. I keep a set of notes to help me remember not only the how but the WHY I set things up in a particular way, and those notes are accessible 100% offline. (After all, what good are notes on how things are set up if the thing you’ve stored them on isn’t working?)

My infrastructure at home (C6100, SAS shelf, switch, etc.) consumes about 700W 24/7 which is not awesome but I figure the power bill saves a lot of service costs. The VPS runs me about $30/mo.

That sounds pretty great - do you have a link to your site using this?

This is a post near and dear to my heart. I too am looking for something markdown-centered, static, reasonable for code/technical content and that can embed not just photos but browser-side js graphs based on data within the markdown or REST calls. It’s been a few years since I went down this rabbit hole and I got lost very quickly. Thank you for posting, I’ll be revisiting over the next few days to see what the fediverse has to say!

I’m not sure how you would do that if you are asking about something you don’t have expertise in yet, as it takes the exact same authoritative tone no matter whether the information is real.

I agree – That’s why I’m chalking it up to some kind of healthy sense of skepticism when it comes to trusting authoritative-sounding answers by themselves. e.g. “ok that sounds plausible, let’s see if we can find supporting information on this answer elsewhere or, maybe ask the same question a different way to see if the new answer(s) seem to line up.”

So far, research suggests this is not possible (unsurprisingly, given the nature of LLMs). Introspective outputs, such as certainty or justifications for decisions, do not map closely to the LLM’s actual internal state.

Interesting – I still see them largely as black boxes so reading about how people smarter than me describe the processes is fascinating.

I don’t know if it’s just my age/experience or some kind of innate “horse sense” But I tend to do alright with detecting shit responses, whether they be human trolls or an LLM that is lying through its virtual teeth. I don’t see that as bad news, I see it as understanding the limitations of the system. Perhaps with a reasonable prompt an LLM can be more honest about when it’s hallucinating?

I too contributed fairly significantly over a long period of time, particularly on electronics.stackexchange.com. I generally just ignore the weak/low quality questions or vote them down. I might respond and ask them to fix the question if I felt charitable, but I never understood the “question nazis”.

Completely forgot about kiwix; I have that on my ipad and laptop, along with Dash which is like a modern day HELPPC.COM if anyone remembers that thing…

There are poor personality types everywhere, but I have found stackexchange/stackoverflow to be one of the better sources of user curated help. LLMs are a new and interesting avenue and I’ve had some good success with them too, but Stackoverflow was really, really good.

Grab a copy of the stackoverflow database and use it locally, or train your own local LLM on the datastore.

And if you can, donate to the Internet Archive – those people do really important work in today’s age of killing off old information and constant enshittification.

A couple things I’ve been working on

First, I spun up a larger VPS to consolidate two smaller ones. This time I dockerized almost everything. Still a docker newb, but karakeep, redmine, mbin, lemmy (still deciding which I want), davical. Asterisk and postfix/dovecot are probably gonna stay on the vps root. I’m using zfs and compression. Interestingly, the postgres database that everything is using seems to get better compression than the mail spool.

A couple weeks ago I picked up a NetApp 7 bay disk shelf for $30. It uses fibre channel (AT-FCX) controllers and I’ve never used that before. I grabbed a $7 FC HBA (QLE2560), a 2m cable and an m2-to-PCIe adapter meant for an eGPU. The idea is to see if I can’t get the RK3588 board I’m playing with to see it. I did something similar with a $50 Dell 12 drive bay and my old C6100.

You could use KaraKeep (formerly Hoarder) with the Hoarder’s Pipette extension maybe?

A full archive of the Japanese TV show Supreme Skills. I can find bits and pieces but I don’t think I even have one complete season.