Coding with AI

Yesterday, I used "generative ai" for something useful, for the first time.

I have done a few very small experiments using ChatGPT or Google Gemini to write or edit small chunks of code. Results were pretty mixed; debatably it was a net benefit, i.e. its output was actually worth the time it took to talk to the bot and review its output. But the benefit was very marginal.

Yesterday, I wanted to write a java program to help me find identical files on my computer. Basically read through every file in a directory, recursively, get some metadata (full path, size, last modified, and a hash of the contents), and write all of that to a single file.

This was a challenge, because my total experience with writing java was about 5 hours, 15 years ago. I could easily have written the same functionality in C#, at least for Windows specifically; but I'm now using linux and wanted to learn java. So I decided, instead of large amounts of time reading reference docs and Q&A sites, I would give this whole "new paradigm" I've been hearing about a chance.

I opted to use Google's Gemini, since I like the interface, and just because historically I've found Google to be much better than other companies at protecting data from hackers. I was overall very happy with the process and results. 

I started just asking for a program from nothing - which merely wrote an output file from a list of files. Then I modified the code, tested it, and asked the bot to add more functionality. This went on for hours, taking turns asking the bot for a modification, which I would often tweak myself even before trying to run it. In the end, I now have a simple utility which does exactly what I wanted.

For anyone who wants to try this, I strongly recommend reviewing all of the changes that the bot makes. using source control to help you review every change that the bot suggests. Every time I made a code change, I would:

1. commit my code to git
2. copy the full text into the chat bot interface
3. have the bot make changes, possibly multiple
4. copy the modified version back into my code
5. manually review the git compare of all changes

Aside from ensuring no bugs are added to my own code, this also gives me a chance to learn some things that I might not have known before.

I'm sure I could have gotten the same basic functionality on my own, although it would have taken much longer -- probably double or triple the time. If I were experienced at java programming, I expect the time taken would be pretty close to break-even.

But the nice benefit of the ai assist was really in helping me identify a few things I hadn't thought of, or reconsider things I had just decided without any serious thought. Examples: 

• It changed my main data-holding class to a "record", which I didn't know was a thing -- so therefore I could not have considered until I learned the java language better. (a "record" is closer to a "struct" which I know from .NET, although not really the same)
• It changed the file reading from one large value, to instead read the data in chunks, which means using less RAM for large files. I thing this had crossed my mind already, but I would not have bothered myself, since I would have had to spend 10 minutes looking up how to do it, then another 10-20 minutes trying it out and making sure it worked as expected.
• It suggested using SHA-256 instead of MD5. I had briefly considered which hash to use, or even just a checksum. Since a malicious user is not a real consideration in this utility, I had opted for just MD5 since it's very fast and pretty universally supported. But when it suggested SHA-256, I figured why not try it, so I did. The overall performance seems effectively identical, presumably the processing is all io-bound so adding some extra CPU cycles has no effect on runtime.

There were probably other things it added, but that's all I can recall at the moment. Now for what it *didn't* do, since it is not "actually intelligent".

• It did not, at any time, seem to understand the purpose of the utility. That is, it never anticipated what my next instruction was, despite a large number of times that I asked it to suggest improvements. For example the biggest optimization by far was to skip recalculating the hash if it has already been done previously, i.e. if the saved (old) filename, size, and last modified date were identical then just keep the already calculated hash value.
• It added lots of clutter, mostly in the form of code comments. Adding a 1 line comment to explain 1 line of basic code is not a benefit - aside from simply bloating the file and creating visual noise, it also adds a maintenance hazard of either 1) spending time keeping the comments updated or 2) having comments that do not match what the code actually does, which is just inviting future bugs.
• It never adopted my coding style. I repeatedly deleted comments, changed variable declarations to be implicit (var x = new myClass();), and remove extra needless code-block sections. But every time the bot generated new code, it did not take the hint to make its additional code conform to the local patterns. Even worse it occasionally added comments to code it wasn't modifying, which I had just deleted. Eventually, I explicitly told it to stop adding comments which were obvious from the actual code, which did greatly reduce that noise although not entirely.

All in all, this was a very good experience, where the chat bot saved me time and improved my end result. But I believe I was at the perfect sweet spot for using ai assistance, namely an experienced developer trying to do something outside their area of expertise. I believe other use cases would also benefit, but not as much.

If I were a very inexperienced developer, yes arguably the help factor would have been much greater compared to doing the task without any assistance. But also, the final product would almost certainly not have been as good, missing significant code improvements for clarity (i.e. maintainability) and performance.

If I were very experienced in java, I would have been able to write a similarly functioning application in nearly the same amount of time. But I would have skimped on a few things. I suspect in the future I will end up fully writing the application, then feeding it to a bot as essentially a code-review to make suggestions.