The Cost of Software
jeferson.me
Software is costly and mechanical parrots won't make it cheaper. Do you want to know why?

As we continue developing our software, we accumulate a growing amount of technical debt just to keep the system running. But I believe we are on the brink of an even larger issue. Cognitive debt.

Hope you enjoy this reading, all feedback is welcome.

  • Shin@piefed.socialOPEnglish
    0·
    3 days ago

    […] can’t LLMs solve the cognitive debt problem better then people? […]

    Better, probably not. And I’ll give you one example. In a codebase there was an issue with Auth, after few runs of the LLM, the best suggestion resulted in 30~40% of change in the Auth workflow.

    Took me a couple hours to figure out that the problem was a misconfiguration key (camelCase to snake_case in the vault store). Fixing this on the store (not on the code) fixed the Auth workflow.

    This two hours were the cognitive debt. And keep in mind, I’m familiar with this part of the code. A Mechanical Parrot happy-trigger boyz would accept the change in the codebase as an attempt to fix it.

    So, mechanical parrot can help? Sure. But better than people, probably not. At least not yet, not with the current set of tool, not with the promise of fully solving it.

    […] exact input and node activation is all you’d need for forensics, the only reason to refeed would old input plus new input mix, […]

    This is badly wrong. The same node can point to multiple places depending on the K factor on this. And this isn’t even the T applied to it. So, the same node can infer multiple different others. This is the nature of the probabilistic of LLM.

    • fruitycoder@sh.itjust.worksEnglish
      0·
      1 day ago

      First agreed. Honestly I haven’t an LLM do anything close to an engineer. You can get lucky for a code snippet or two and maybe a few ADRs but even those are fraught with the chance of slop clean up work. I am not arguing that.

      The node is only probalistic because of the random number added. After the fact the number is known.