FaceDeer

Whether inference is profitable or not is not a global yes/no question. It depends heavily on the circumstances, what you’re using it for and what you’re charging for it. A lot of the money being invested in research right now is going into making inference cheaper, which would of course make it more profitable to sell at current price points. Or just run it yourself, local models are getting quite capable these days.

I wouldn’t bet on any specific company being the ones to survive this, especially not first-movers like OpenAI. More likely they’ll spend their money blazing the trail and the ones to profit from it will be the ones who followed along behind. When a company goes bankrupt it doesn’t poof out of existence, its assets get sold off at pennies on the dollar and then whoever bought those assets gets a chance at running them without the overhead of the previous company’s debt.

This seems somewhat misleading. Lots of products take a lot of investment in them for many years before they reach profitability. The Boeing 787 Dreamliner, for example, was in development for 7 years and another three years after that before it was profitable. The Falcon 9 rocket took 13 years to develop and now it’s the most profitable satellite launcher around. The Dyson bag-free cyclonic vacuum cleaner took 15 years to develop.

Most of this AI stuff has only been in heavy development since ChatGPT burst upon the scene in 2023. It’s not unreasonable to see the industry still heavily into the investment and development side of things.

I think prohibiting reverse engineering would do far more harm.

I’m told that AI can’t actually replace humans, so presumably those jobs will be back in short order.

Yes, that’s the plan, tens of thousands of satellites. SpaceX has actually proposed a million of them in the long term, this sort of constellation is one of the things that a launcher like Starship is intended to support.

Starlink is already on this scale and that’s just using Falcon 9.

Over and over the “it can’t be cooled” refrain. The math and engineering says otherwise. Here’s a video by Scott Manley that walks you through the calculations. It’s quite straightforward, this is not an obstacle.

Musk is not the only person planning these sorts of satellites.

So don’t pack them as densely as Earth-based data centers are packed.

In another comment in this thread I posted a link to a youtube video by Scott Manley explaining the math and engineering behind cooling computer hardware in space, it’s actually pretty straightforward.

That article was incorrect, then. There are many satellites already in orbit that have computers in them - basically all of them do, nowadays - and cooling them is a well understood engineering problem.

The radiator panels on the ISS are 2,500 square meters in area. The radiator panels are 645 square meters.

Most of the proposals for space-based data centers have ended up focusing on plans to place thousands of individual satellites into orbit, not just one big space station with everything packed inside it. Scott Manley recently did an analysis of the cooling requirements, he worked through all the numbers and explained how it works, and there really doesn’t seem to be a problem here.

Yeah, this is fundamental; if you use a thousand joules of energy to do work (of any kind) you will ultimately end up producing a thousand joules of waste heat. The only choice one has in the matter is where that heat goes.

This is a major reason why I get annoyed at the people pooh-poohing space-based data centers. It literally puts the waste heat outside the environment. It should be everything that data center opponents say they want.

While the current splashy “state of the art” models in terms of cognitive ability are American, IMO the real foundation for future AI is coming out of China these days. It’s not quite as smart but they’re focusing heavily on making AI training and inference cheaper in terms of compute (and therefore more efficient in terms of energy usage). It’s a mother-of-invention situation, sure - they’ve been cut off from the latest and greatest NVIDIA cards so they’re having to find ways to make do with less powerful hardware. But that’s going to be super important once AI is “good enough” for various real world tasks and the most powerful models aren’t needed for most activities.

Indeed, it’s basic thermodynamics. The energy coming in to Earth gets turned into heat one way or another, the only question is where that heat goes. In this case it goes into the ocean either way.

Yeah, I wouldn’t use a framework that didn’t let you select the basic model. I’m just thinking about having it automatically switch to a different one during the review “phase”. It’s not as popular a coding agent these days but I like using Google’s Antigravity and it’s capable of being told to go through the sequence of steps “plan - > write documentation -> implement the plan -> run unit tests -> do a code review” automatically without needing to be prompted at each step. That’s where it would be nice to have it automatically switch for the review.

“Wear the reviewer hat now” does seem to work quite well with the same model, but if more models from different lineages are available it just seems like the right thing to do to switch to another one.

I’ve become rather disillusioned with Gemini’s use of search tools lately. It’s odd given that it’s a Google model, you’d think Google would be at the top of the search engine game. But honestly, Deepseek’s been my go-to lately when I want an answer that’s likely to be synthesized from a lot of web searches. I’ve had it search over a hundred different pages for a generic “how does this work?” Sort of query. It didn’t read them all, but it’s casting a wide net and it’s letting me actually see the details. Gemini seems more willing to just tell me what it “thinks” the answer to a question is based off of its training data, which is not a particularly reliable thing for an LLM to do.

A thing I found quite amusing about the AI agents I’ve toyed with is that they have a step where they do a code review of their changelist, usually switching to a different “persona” when they write it so that they’re not seeing it as “their own” code. It’s funny reading at the critiques and compliments it gives the “other agent” it’s checking the changes for.

I haven’t seen this feature yet, but it might be a good future enhancement to ensure that the harness literally uses a different model for the code review from the one that wrote the code in the first place. If Claude wrote the code have GPT do the review, and vice versa, for example. Wouldn’t be surprised if the feature exists and I just haven’t spotted it yet though, things change fast.

Indeed. My usual analogy is “it’s a team of junior devs at your beck and call, who will do a huge amount of work quickly when you tell them to so make sure you told them to do the right work.”

As for the documentation, commenting, unit tests, and so forth - AI is very handy for getting that stuff written too, just make sure to check up on it. For large pieces of work I will often tell the AI to write up an architecture document first, as a separate step, both to make sure it understands what I asked for and to store as future reference for the AI to make sure it doesn’t “forget” how the code is supposed to work or why it exists.

There was a fun Python application I worked with an AI on a few months back where most of the code ran as a conventional Python program in a conventional Python environment, but one particular file was being “injected” into an entirely separate and very locked-down Python sandbox to use as a bridge between that sandbox and the rest of the application out in the outside world. That particular file would be the only one that was able to import key modules and access key data, and it couldn’t import any other modules that the “outside” application might have access to. Two very different execution environments bundled together in the same repository. But I made sure the file had comments explaining this strange setup and that there were system architecture documents explaining how it worked, and I only recall having to remind the AI once or twice that the change it wanted to make would run afoul of that. It otherwise managed the separation of functionality just fine. And that was with the previous generation of models, the current ones are even better.

It’s a phenomenon called NIMBY and it’s not specific to data centers.

The noise is annoying, for sure. But ideally they can go have their AI-free projects and refuse to use the ones that do use AI, and the rest of us can use whichever of the two types of projects works better. It would be win/win.

In this case the evidence is literally first-hand experience. There is nothing that will change my mind on this because it’s my direct personal experience from actual use.

I honestly don’t care what marketing says, and if other people have different experiences then that’s just them. In my personal actual real-world experience I found that they let me get tons more done and their quality of work is perfectly fine as long as you’re using the right tools and giving them the right instructions.

The article says that developers are disagreeing with that in situations where they are “forced” to use AI, and that’s fair, it doesn’t make sense to force a tool to be used for something it’s not good at. They might be using it wrong. I use it whenever it’s better than not using it, and that ends up being quite often in my workflow.