The Cost of Software
jeferson.me
Software is costly and mechanical parrots won't make it cheaper. Do you want to know why?

As we continue developing our software, we accumulate a growing amount of technical debt just to keep the system running. But I believe we are on the brink of an even larger issue. Cognitive debt.

Hope you enjoy this reading, all feedback is welcome.

  • wholookshere@lemmy.blahaj.zoneEnglish
    0·
    3 days ago

    For training maybe. But execution as far as i know are just a bunch of probabilities chucked into matrices. Back in my physcos days I could u derstand the math, but not today.

    Also what needs citing, that nodes translate to debugable, reproducible outputs.

    Because again, it’s all probablilties under the hood.

    • fruitycoder@sh.itjust.worksEnglish
      0·
      3 days ago

      I mean probabilities in matrices are nodes in a neural net, right?

      The only thing that makes it non-deterministic is the tempature value which is known after the fact from my understanding, so that should be able to deterministic.

      • wholookshere@lemmy.blahaj.zoneEnglish
        0·
        3 days ago

        No… No they’re not. Either that or nerual nets are even dumber than I thought. My understanding was emulating nodes of information like brains do. Thats not probabilities. But willing to be wrong if you can bring sources. The connection between points in probability doesn’t make sense. Thats a nonsense statement.

        Thats also no how simulated annealing works. The temperature value is roughly the probability it will pick a different, less optimal step, in order to try and find better alternative paths. The temperature value is roughly the probability of trying something ‘random’. Not deterministic at all. The temperature value is lowered with time and progression. Its known the entire time, but its still a weighted coin flip that determines which path to take.

        https://en.wikipedia.org/wiki/Simulated_annealing

          • wholookshere@lemmy.blahaj.zoneEnglish
            0·
            2 days ago

            I’m not convinced of that you do. Nor is knowing that flip helpful. Because it’s non deterministic. An internal coin flip is not an input either.

            • fruitycoder@sh.itjust.worksEnglish
              0·
              2 days ago

              What I mean is for post hoc analysis since you know the coin flips and the inputs you can step through the layers and nodes calculation deterministically.

                • fruitycoder@sh.itjust.worksEnglish
                  0·
                  1 day ago

                  You can try to reason about it which is an arguable improvement from the former engineer you can’t get ahold of anymore. So you can could see that it triggered a node associated with c++ when it generated this string of tokens or compare it other generations to map it to a type of call.

                  Where it doesn’t compare to a human engineer at the moment is quality or novelty.

                  • wholookshere@lemmy.blahaj.zoneEnglish
                    0·
                    1 day ago

                    But you can’t?

                    It didn’t reason its way down a branch, it made coinflips all along the way. So its not actually thinking, just weighted coin flips. There’s no ‘why’ it chose a branch. Just the result of a coin flip.

                    You can’t ask a coin why it chose heads. You can unroll the physics of it all day long til the cow comes home. But it still doesn’t help you on the next one.