• 5 Posts
  • 439 Comments
Joined 3 years ago
cake
Cake day: June 16th, 2023

help-circle




  • It’s not about the variation of the words, it’s about the variation of the words from the model baseline.

    Like if your word choice was almost the exact same as Claude’s normally, maybe you just talked to them a lot and picked up their phrases like it’s not nothing.

    But if you managed to be almost exactly like Claude and yet varied the possible words exactly according to a hidden entropy key, they’d know it was actually Claude with the SymthID-Text watermarking applied, as no human would end up falling into that statistical bucket.



  • Probably less so.

    The hardware to run it locally would be fairly expensive and would require using a very simple model compared to alternatives. Also much more wasteful if you were only using that hardware for AI use, as you’d be distributing the hardware out rather than centralizing so it’d be often idle and when replaced create more waste than a centralized server rack.

    Also, additional per user post-training seems to me both wasteful and not necessary. In context learning is often much more powerful but frequently overlooked.


  • If you’re concerned about data retention, you’d want to select an inference provider that is listing ‘ZDR’ (zero data retention) as a feature.

    For example, a lot of the Chinese open weight models have become quite capable and because their weights are available end up like generic vs brand name medicine where there’s multiple providers serving them with different production conditions.

    Because enterprise use will often be worried about data retention or sending data to China, the alternative providers usually offer things like US-only inference or zero data retention.

    If you’re not going to use it all that often, a la carte API use is going to be way cheaper than a subscription, probably better results than a free plan with a closed model provider, and give you more control over the process.


  • kromem@lemmy.worldtoTechnology@lemmy.worldNobody Asked for AI
    link
    fedilink
    English
    arrow-up
    3
    ·
    edit-2
    13 days ago

    consumed 40% of global water supplies. But now it is.

    Source? This seems really, really unlikely. Last I saw total data center water use (only some of which is AI) was still much less than even things like golf course watering.

    AI Data Centers’ Water Consumption Breaks 264 Billion Gallons in 2025 as Devastating Drought Hits Nearly 63% of U.S.

    Golf courses reduce water usage by 31 percent according to national survey

    The report found that U.S. golf facilities applied a projected 1.63 million acre-feet of water in 2024

    (1 Acre-feet = 325,851.4286 gallons, so this is 531 billion gallons)

    The golf course industry points out their use is less than 0.5% of the total water use of the US, but it’s twice the use of all data centers.

    So for just AI to be using 40% of all water seems… really unlikely?

    Not that it’s surprising you’re under that impression, as even in the headline above, the gist many articles about this issue try to push is that the 264 billion gallons of AI water use is connected to a drought across the country. But the actual numbers reveal that as fairly manipulative as if we rewrite the headline to “golf courses use a little under 0.5% of total US water as drought grips the nation” it’s pretty absurd to suggest the first thing is significantly impacting the second and yet we’d be representing twice the water use as the original headline is discussing.




  • Right, but what % of people are currently using/demanding inference right now?

    Do you expect that % to change between now and 2030?

    Unless you expect demand to decrease, I don’t really see how the pricing of the hardware will decrease.

    Let’s say the Pets.com of the AI world ends up going bankrupt and their RAM hits the market. Do you expect that the demand for that RAM will be negligible such that pricing returns to earlier levels?

    Your predictive model relies on companies that have hardware going out of business and then other people buying up that hardware, but isn’t accounting for the levels of demand that the market will have for that secondhand hardware even if it ends up existing from failed firms.

    Unless the demand shifts, the more likely scenario is that companies going out of business will be able to sell off their RAM at higher prices than they bought it at.

    There’d need to be a significant inference memory reduction advance (possible) coupled with stagnating or reduced inference demand (unlikely) to see prices come back down.







  • It’s true.

    The field is moving so fast that things can change quickly, but the American labs are so caught up in saddling their models with safety overhead that the recent Chinese models are very close in practical use to the flagship American models if not pulling ahead (Sora vs Seedance 2).

    I don’t really need to solve Erdős problems in my day to day. Outside of increasingly edge case eval competition, I’m not sure what OpenAI brings that literally everyone else isn’t also capable of providing (and more).

    I’d maybe invest in Anthropic for an IPO if they turned around their own saddling of models and played nicer with open platforms, but if Claude is just going to get more and more anxious due to excessive red teaming and CC fall further and further behind stuff like Hermes Agent, they too are going to fall by the wayside as open models become the dominant inference for open infrastructure.