I am a programmer with 25 years of professional experience, mostly embedded C/C++, with a decent amount of Python and the last year has been mostly Rust with no prior experience.

Current LLMs write both worse and better code than me, often at the same time. Let me explain.

LLMs are very good at syntax, with an encyclopedic knowledge of libraries, languages, and algorithms that is a few years out of date. I particularly notice this in Rust, which has a lot of syntax and features, like the 482ish ways to chain Options and Results that I still haven’t memorized all of.

So for a relatively simple function in a language that I’m not very famialiar with they will often produce a bit tighter code that makes better use of the language features. They will also sometimes come up with an obscure algorithm that I haven’t though about in decades or even heard of that can solve the problem better than my first attempt.

On the other hand, they are terrible at larger scale work, generally building fragile code while protecting against things that can’t actually happen, using overly complex architecture, organizing code in dumb ways, missing obvious opportunities for simplicity, and relying too much on existing bad patterns instead of fixing them along the way, etc. They also tend to be a terrible judge of when to use a library vs roll their own, leaning heavily towards rolling their own instead of just adding a single include, which means more new untested code.

On the third hand, I do find that they can be pretty good at reviewing code, and it is nice to have them write tests, since we all hate writing tests and tend to write as little as we can get away with.

Not a programmer either but I’ll throw in my anecdotes:

• I was writing a “cog” for pycord (Python Discord bot framework). I wanted a slash command to hit an API for some reason or another. Whatever AI I was using hallucinated a non-existent endpoint to achieve that so, of course, it didn’t work. I wasted a half hour reading past and upcoming specs of the API to find that endpoint never existed and wasn’t going to exist in the next release. The code was structurally sound, pretty much identical to other functions I’d written by hand, but it reached out to something that does not exist. It did this with a popular, well-documented piece of software.
• Speaking of structurally sound, there’s no real time save if I have to double back to make sure it filled in all the blanks correctly. If the AI can fuck up with an API that it should “know,” it’s a guarantee that will be necessary to correct everything when an AI has no knowledge of it whatsoever. So now time is spent making sure it isn’t referencing variables it failed to define, pointing to API endpoints that aren’t there, etc. I can define boilerplates and snippets in any decent text editor. An autocomplete plugin will fill out the entire structure of a function being defined the second I type def and leave me to fill in the blanks - all without deferring to a water and energy sucking, data mining, douchebag of a company. What’s being gained that isn’t costing me privacy, depriving myself the use of my own cognitive abilities, and ultimately making the world a little worse off?
• Last one, I wrote a bash script years ago using exiv2 to pull the EXIF data and sort unsorted photos into ../YYYY/MM/DD/. directories. When the AI hype started gaining traction I tasked one with doing the same to see what it would do differently. Aside from some capitalization and variable names, it was identical. Now it’s not exactly a complicated task but my takeaway was either I’m a better programmer/scripter than I thought (not likely) or AI is just as bad (or worse) of a programmer as me. I’m leaning towards worse, considering I manage to actually know if an endpoint or variable exists before referencing it.

Anyway, the only conclusion I can draw is those “programmers” are developing unmaintainable blackboxes out of twigs and playing cards. If some big bad wolf by the name of “Intellectual Debt” doesn’t inevitably come along to huff and puff, they’ll just end up collapsing under their own weight.

Makes me wonder how good they really are at their craft to begin with.

That is exactly the point.

LLM’s are great at what they already do, they simply statistically predict the output text from an input text. This means that if an expert needs LLM’s in their line of work they were never an expert. Unless of course they are trying to brute-force solutions to extremely complex theorems were humans would take a lifetime (literally) to answer.

It’s shades of gray honestly. Sometimes they are very good (especially if you need some small tool that scratches a particular itch or want something that is 50% there right now), other times they’re extremely dumb and write horrible code. Especially of you want them to use some modern feature of your language that they don’t know anything about, or when the language in question isn’t very popular, or when the library you are using at the same time isn’t very popular, changed a lot recently and is big. They can go in circles for hours with 'wait actually …, actually …, but wait …, but actually …, ad nauseam. But when you limit them in scope, give realistic goals and constraints they more often than not work good.

for small programs that are simple to confirm that they work AI is great, for anything particularly complicated it starts forgetting everything.

deleted by creator