Indeed. My usual analogy is “it’s a team of junior devs at your beck and call, who will do a huge amount of work quickly when you tell them to so make sure you told them to do the right work.”
As for the documentation, commenting, unit tests, and so forth - AI is very handy for getting that stuff written too, just make sure to check up on it. For large pieces of work I will often tell the AI to write up an architecture document first, as a separate step, both to make sure it understands what I asked for and to store as future reference for the AI to make sure it doesn’t “forget” how the code is supposed to work or why it exists.
There was a fun Python application I worked with an AI on a few months back where most of the code ran as a conventional Python program in a conventional Python environment, but one particular file was being “injected” into an entirely separate and very locked-down Python sandbox to use as a bridge between that sandbox and the rest of the application out in the outside world. That particular file would be the only one that was able to import key modules and access key data, and it couldn’t import any other modules that the “outside” application might have access to. Two very different execution environments bundled together in the same repository. But I made sure the file had comments explaining this strange setup and that there were system architecture documents explaining how it worked, and I only recall having to remind the AI once or twice that the change it wanted to make would run afoul of that. It otherwise managed the separation of functionality just fine. And that was with the previous generation of models, the current ones are even better.
having to remind the AI once or twice that the change it wanted to make would run afoul of that.
I find that doing periodic “directed reviews” with the AI agent looking for violations is helpful. Not a 100% guarantee of compliance on the first pass, but pretty close after 3 or 4 passes from varying perspectives.
Also, when something is really important, having the agent break up the changes as small as practical, then reviewing each change by hand as it goes through, definitely is more confidence inspring than attempting to review 3000 lines of change in a 20,000 line module of code all at once against a 20 page description of changes.
Indeed. My usual analogy is “it’s a team of junior devs at your beck and call, who will do a huge amount of work quickly when you tell them to so make sure you told them to do the right work.”
As for the documentation, commenting, unit tests, and so forth - AI is very handy for getting that stuff written too, just make sure to check up on it. For large pieces of work I will often tell the AI to write up an architecture document first, as a separate step, both to make sure it understands what I asked for and to store as future reference for the AI to make sure it doesn’t “forget” how the code is supposed to work or why it exists.
There was a fun Python application I worked with an AI on a few months back where most of the code ran as a conventional Python program in a conventional Python environment, but one particular file was being “injected” into an entirely separate and very locked-down Python sandbox to use as a bridge between that sandbox and the rest of the application out in the outside world. That particular file would be the only one that was able to import key modules and access key data, and it couldn’t import any other modules that the “outside” application might have access to. Two very different execution environments bundled together in the same repository. But I made sure the file had comments explaining this strange setup and that there were system architecture documents explaining how it worked, and I only recall having to remind the AI once or twice that the change it wanted to make would run afoul of that. It otherwise managed the separation of functionality just fine. And that was with the previous generation of models, the current ones are even better.
I find that doing periodic “directed reviews” with the AI agent looking for violations is helpful. Not a 100% guarantee of compliance on the first pass, but pretty close after 3 or 4 passes from varying perspectives.
Also, when something is really important, having the agent break up the changes as small as practical, then reviewing each change by hand as it goes through, definitely is more confidence inspring than attempting to review 3000 lines of change in a 20,000 line module of code all at once against a 20 page description of changes.