Shirley you’ve heard of absurdist humor?

You say “Not even close.” in response to the suggestion that Apple’s research can be used to improve benchmarks for AI performance, but then later say the article talks about how we might need different approaches to achieve reasoning.

Now, mind you - achieving reasoning can only happen if the model is accurate and works well. And to have a good model, you must have good benchmarks.

Not to belabor the point, but here’s what the article and study says:

The article talks at length about the reliance on a standardized set of questions - GSM8K, and how the questions themselves may have made their way into the training data. It notes that modifying the questions dynamically leads to decreases in performance of the tested models, even if the complexity of the problem to be solved has not gone up.

The third sentence of the paper (Abstract section) says this “While the performance of LLMs on GSM8K has significantly improved in recent years, it remains unclear whether their mathematical reasoning capabilities have genuinely advanced, raising questions about the reliability of the reported metrics.” The rest of the abstract goes on to discuss (paraphrased in layman’s terms) that LLM’s are ‘studying for the test’ and not generally achieving real reasoning capabilities.

By presenting their methodology - dynamically changing the evaluation criteria to reduce data pollution and require models be capable of eliminating red herrings - the Apple researchers are offering a possible way benchmarking can be improved.
Which is what the person you replied to stated.

The commenter is fairly close, it seems.

That’s very fair, indeed.

Perhaps awareness of one will spark awareness of the other. I suppose my concern is that plasticisers are sort of a ‘hidden’ risk, for the most part. They’re used in nearly every food packaging (and prep, such as hoses) that isn’t contained in glass, or served up in its own peel.

Microplastics are terrifying and all that, but I’m sort of more worried about plasticisers like BPA, BPF, BPS and the rest of the alphabet of BP-whatever’s that was created and brought into use after the dangers of BPA were realized.

Just a heads up - if something plastic says it’s BPA-free, it probably uses a different bisphenol compound that is less studied than BPA. And is likely as toxic (or even more toxic)!

But nobody ever talks about those, because science words.

I’ve turned off my voicemail with my cell carrier, and send all unknown numbers to voice mail.

It’s a peaceful life.

The problem is that now the first page of results is all AI garbage and wrong, so you’re not 100% sure at what point you’ve reached the sane internet.

Well. That’s it. Get the flamethrowers. Time to burn down the Amazon.

No. Not the one that’s already burning. The other one.

I’m thinking about those poor suckers on the other side of the lakes, catching all the humidity the prevailing winds are going to pick up. Michigan is more or less flat from the last glacial period and it’s only just barely not one big marshland. They’d better start building houses on stilts and investing in improving their drainage systems.
Buffalo catches hell from lake effect snow, but just wait until there’s lake effect monsoons.

I haven’t yet looked at the map (I will!), but I’m struck by the idea that perhaps a map should exist that shows how USDA hardiness zones will shift. (I mean - according to best guesses.)

If I had the ability, it would be interesting to make a map that asks users what their favorite local tree or animal is, and tells them how long it will be able to survive near them. Nearly impossible to account for all use cases, but I digress. Even simpler - Go for a map of state trees, flowers, and animals with extinction times for each to let folks know how long each state can hold onto its signature species. Well, for the ones that aren’t already gone, anyway.

Well, I just realized I completely goofed, because I went with .arpa instead of .home.arpa, due to what was surely not my own failings.

So I guess I’m going to be changing my home’s domain anyway.

For only way more time and money, you can buy a zigbee smart plug and a vendor agnostic zigbee hub flashed with FOSS, or you can buy a esp-based board, wire it up with a relay, and flash it with something like esphome.

Sure, it’s way more money and hours of work (cumulatively), but it won’t lose support!

My wife shared this with me yesterday, but I didn’t see it:

Somebunny is gonna learn those things aren’t windows-based today!

The other commenter on this pointed out that I should have said crisis management rather than disaster recovery, and they’re right - and so were you, but I wasn’t thinking about that this morning.

That’s a really astute observation - I threw out disaster recovery when I probably ought to have used crisis management instead. Imprecise on my part.

Ah, you’re right. A poor turn of phrase.

I meant to say that intel brands their IPMI tools as AMT or vPro. (And completely sidestepped mentioning the numerous issues with AMT, because, well, that’s probably a novel at this point.)

I think we’re defining disaster differently. This is a disaster. It’s just not one that necessitates restoring from backup.

Disaster recovery is about the plan(s), not necessarily specific actions. I would hope that companies recognize rerolling the server from backup isn’t the only option for every possible problem.
I imagine CrowdStrike pulled the update, but that would be a nightmare of epic dumbness if organizations got trapped in a loop.

Honestly kind of excited for the company blogs to start spitting out their disaster recovery crisis management stories.

I mean - this is just a giant test of disaster recovery crisis management plans. And while there are absolutely real-world consequences to this, the fix almost seems scriptable.

If a company uses IPMI (Called Branded AMT and sometimes vPro by Intel), and their network is intact/the devices are on their network, they ought to be able to remotely address this.
But that’s obviously predicated on them having already deployed/configured the tools.

Literally last week my wife noticed one while out and remarked “I can’t believe they’re still around.”

I just sent the article to her with the caption “You did this!”

Anyone else think this was the “taking a selfie using a silly object” meme at first?

I’m cynically viewing this as not a positive. I assume this is so they can make pages 2, 3 and so on as spammy as page 1.

Not at first, obviously. You don’t boil that frog on high heat.
You throw out a second page with a cute little text ad off to the side, then 1 or 2 at the top, then a mid-page ad. Maybe some suggested content.

Instead of having to scroll through a page’s worth of ads to get to semi-relevant results with a gem hidden in them, it’ll be a pages worth of ads for your semi-relevant results per page, and maybe what you were looking for 4 or 5 pages in.

Google used to be good. They ‘know’ what people are looking for. So they’ll probably hire someone familiar with gambling to figure out a minimum dispersion of relevant results on the pages, to keep people using the service and scrolling past ads. … I used to remember this. Variable-ratio reward schedule?