Has AI killed open source

… at least for me?

This will be a personal opinion piece, as well as the thoughts behind an AI contribution policy I am working on for my personal projects.

To start off, I have been contributing to open source projects, and have released basically every side project of mine under an open source (primarily copyleft) license. This goes back over 20 years, when SourceForge was still a thing.

A dream in my early days was contributing to Firefox, which I made a reality, and it went as far as contributing minor JS features into SpiderMonkey, the Firefox JS engine. Various contributions to the JavaScript and TypeScript ecosystem followed. Then 2020 I started working for Sentry, and shifted to the Rust ecosystem, where I also started contributing to the Rust compiler as well. The highlight there was my speaker role at EuroRust where I explained how async functions are being treated internally in the compiler, and what I have done to improve those in some aspects.

Those were contributions to existing projects, and even though I stuck around for a bit on some projects, I rather consider myself as an allrounder. I was able to quickly dive into a foreign projects, scratch my own itch as the saying goes, and get things merged.

Then there is my personal projects, of which two gained widespread popularity.

rollup-plugin-dts is the go-to solution to bundle TypeScript type definitions before publishing a package to npm. It is specialized, covering a small niche, and I suspect a lot of people are using it without knowing about it, as it is often used as an internal dependency of more all in one bundler solutions.

rust-cache is the go-to solution for Rust projects hosted on GitHub. It is a one-liner that you copy-paste into your GitHub Actions workflow to add caching and thus speed up CI workflows. It is really a no-brainer and the defaults cover most projects out of the box. But it does have more options to fine-tune. Fun fact, at one of the Rust conferences, someone recognized my name and told me they were copy-pasting it around all day :-D

I consider both projects done, they serve their purpose well, and require very minimal maintenance. Well except now with a new TypeScript version on the horizon that will completely overhaul its compiler API. And possibly some Rust/cargo changes related to how they handle their compilation artifacts.

That shouldn’t discourage you from sponsoring me though :-) If you get value out of using one of my projects, or appreciate my contributions to external projects, consider sponsoring me.

As a side-note, I have also quit my job at Sentry late last year for mostly personal reasons I do not want to get into. Since then, I haven’t actively contributed to open source projects, apart from the absolute minimum of maintenance for my own projects.

Which brings us to today, and the question why? And the answer to that is AI, and I will explain.

Well its not a big surprise that a lot of folks drawn to programming and open source are well, socially awkward, me included. I do not feel particularly comfortable in all social settings, and the internet and open source community feels like a safe space for social interaction with like minded individuals.

Especially the open source communities are a place to learn from other maintainers when contributing, and to mentor other contributors when doing reviews. I think this is very important, and is a main driver of how engineers grow, and I put particular emphasis into the reviews I do, as I do see myself as a mentor who is helping others become better engineers, to help them understand what is important and why.

This whole relationship and the motivation behind it is just null and void when you are dealing with AI generated contributions. There is no social interaction happening. No one has put an effort into the contribution. And it is pointless to put efforts into a review, as no one benefits from being mentored though it. There is no learning effect.

There is an unwritten agreement between contributor and maintainer. When a contributor puts in genuine effort, curiosity, and a willingness to learn. And the maintainer appreciates that effort in kind by providing mentorship. This does not work for AI, as there is zero genuine effort involved.

But things are actually worse. AI contributions are often times excessively verbose, and create more work for me as the maintainer.

One the one hand, there is overly long and verbose issues and PR descriptions with sub headings and prose as long as a short story. Then there is overly extensive test cases that cover every minute edge case.

To be quite honest, this completely discourages me from even taking a look at those obviously AI generated descriptions.

A human would respect my time as maintainer and spare me from having to read all this. Not to mention that it is in everyones self interest to keep things short and concise, as it takes a human quite some effort to write things. There is zero effort involved for an AI to generate all that slop. I just wish they would consider the effort it takes a human to read all of that. Unless getting the human out of the loop completely is part of the plan all along. But that might be a bit too much conspiracy theory thinking.

Similarly, a human would also have better judgement of how exhaustive tests really need to be, and can make a better tradeoff considering also the complexity of the test suite, the time it takes a human to read and understand it, as well as the time it needs to run. An AI makes it effortless to generate a ton of exhaustive test cases, without any regard for how long it takes to run those test, or to read them.

Fuzzing, property testing and mutation testing exists. And they are worth the effort to review their respective harness. But you wouldn’t create every single sample as its own separate test case, would you? It almost feels like AI has no problem doing that though.

# An AI policy

Well long story short, what does this all mean?

First, every issue and PR needs to be a social interaction and communication between two humans. You as human contributor need to put in real effort and a willingness to engage in a social interaction with me as the human maintainer.

Second, please respect my time as a maintainer, and try to keep things short and concise. Be mindful and try to reduce the effort that I have to put into reviews. Less is more.

I am not strictly against usage of AI, as long as a human understands and can explain in their own words what a change does. And as long as the interaction and communication between contributor and maintainer has to be human to human.

In more descriptive words:

• Issues, comments, PR descriptions and commit messages MUST be human made.
• Code MAY have been created with AI assistance, as long as a human understands what is being done and why.
• Respect each others time, and prefer conciseness over exhaustiveness.

# Would I use AI for anything?

With that out of the way, would I actually use AI myself for anything. Or are there things I think AI would be a good fit for?

So far I have only tried AI code generation twice while working on the codecov codebase. Both times I tried to use it for large scale refactorings, and both times I was very disappointed with the results. Something like a code-mod, but more sophisticated. I have changed a particular function signature, and asked the AI to basically update all call sites, including tests.

The AI first dumped a lot of text into the prompt I didn’t really read, and my reaction to it was: Don’t tell, just do. It then started hallucinating completely new test cases, started renaming random variables, etc. Not what I asked it to do. I was completely disappointed and it wasted more of my time than if I had just done the trivial changes myself.

But apart from my bad experience so far, I could think of two examples where I would consider trying AI for in the future.

One of those would be generating readable changelogs. I really hate all the automated changelog generation methods that have existed before AI. I hate having to prefix every PR title with fix:, ref:, chore: or whatever other policy there might be. I have hated to write a changelog entry for every single trivial PR, or having to explicitly opt out of that using a #skip-changelog comment, as was the policy at Sentry. The resulting generated lists full of those fix: lines are completely unreadable, and more often than not are far too verbose.

Even though changelogs are meant for human consumption, I think an AI might be well suited to summarize things into a few important items, maybe even with generated code examples, while skipping unimportant details. Like who the hell cares about 30 lines of chore: updated dependency X from version Y to Z?

Another example where I think AI could be beneficial is generating documentation and examples. Like a lot of other engineers, I am notoriously bad at writing documentation. And lets be honest, what else would you want me to write for a set_foo function other than, well Sets the foo.

I think I might have even used the term documentation driven development at some point on this blog, owing to how great rustdoc as a tool is. But actually coming up with reasonable examples and documentation for every single function is a bit too much for me.

Well both of these two examples are interesting in the sense that thus far both artifacts have been for human consumption, though this might as well change at some point?

Both of these also require a lot of effort to get right. And both are already being automated by tools that result in really low quality output. I think that using AI here might improve the output quality by keeping the effort minimal.

On the other hand though, hand-crafted, organic, human-made changelogs and documentation are really super good and a joy to read and use. Again, maybe because I am notoriously bad at it, I can still wish for a shortcut here, can I?