Generative AI has been a transformative technology to software engineers. It can help us reason about previously unknown codebases, generate code and tests asynchronously from other work, review pull requests, propose features with context of a codebase, and many more tasks that improve the efficiency of our job. To reap these benefits, much of the Strands team day-to-day work has transitioned into Human-to-Agent conversations. Along with this increased usage, a popular idea has come up: “the more we let Agents do our work, the more our efficiency increases”. In many ways this has been true, as seen by multiple projects that have voiced the efficiency gains by utilizing AI (ref, ref, ref). However, these efficiency gains are pushing us to become overly reliant on this technology. Instead of critically thinking and working on a problem, it is so much easier to offload that work to an AI, meaning the quality of the content created is only as good as what AI can produce. In the same vein, there seems to be a growing sentiment that the content generated by AI does not have the same quality that a human can produce, and many organizations are pushing back against “AI Slop” (ref, ref). As a means to help balance the good with the bad, the Strands team is introducing an AI Usage Policy that is meant to help our community continue to reap the benefits of AI, while acknowledging and trying to limit the downsides that come along with it.
A big part of this quality gap comes from the “voice” of an Agent and the high volume of content it generates. Agents seem to follow a mantra of “More is More”; they write tons of information with long winded explanations which make it difficult to extract any real meaning from. Additionally, much of our work has turned into kicking off multiple agents in parallel to do some tasks: write code, investigate a topic, or write a document. The combination of lengthy responses coupled with parallel execution of agents is leading to tons of low-quality AI content being generated, many engineers burning themselves out trying to keep up, and impacts to products that are released. This bias towards doing more faster, instead of trying to do less better, means our bar for quality is dropping to the level of what Agents can generate.
The low-quality content that AI generates has a real impact beyond making an engineer’s work more frustrating. Vulnerabilities have surfaced in the Strands team’s code in part because of the large amounts of AI-generated code obfuscates real security issues hidden amidst the volume; a part of our investigation highlighted AI-generated code failed to account for security patterns unique to an agent SDK like Strands. Strands documentation (both in code and on the website) faces quality issues because it is riddled with AI content that we have no expectation a human will read. Our repos have low-quality content in pull requests, either in discussions (ref) or in the code submitted, that distract us from interacting with human submitted requests. The ways we let Agents participate today makes it harder to understand what is happening in our codebase, discourages humans from actually reading what is contributed, and means human interactions are getting drowned out by AI noise.
In contrast, we are seeing Human written content as being higher quality because it makes up for the failings of AI. Humans are much better at conveying ideas concisely and are often better at connecting with the reader through their own distinct “voice”. AWS VP and Distinguished Engineer Marc Brooker has a great blog post sharing his take on consuming AI-generated content:
… I think that asking people to read LLM-generated text breaks a kind of social contract. When I publish a piece of writing under my name (e.g. in my blog, or a document at work), I want the reader to know that I deeply understand and own what I wrote. That I respect their time and effort. In exchange, I want them to be fully and deeply engaged with reading.
There is clearly a negative sentiment around publishing AI-generated content, but we still see it replace human content in almost every space; the line for where a human or an agent should participate is blurring in frustrating ways. Like Marc mentioned, we need a kind of social contract to inform when humans and agents are meant to participate.
This contract can start to be defined by looking at the places where Agents have a negative impact on quality. For example, Agents are good at analyzing and reviewing code, but when they start to actively participate in discussions between people about that code, the conversation becomes bloated and indiscernible. Jynn Nelson, active contributor to multiple areas of the rust programming language, authored a blog post about how rust-lang/rust is struggling with LLM usage in its community, and shares their opinions on where Agent responses do not belong:
We will often get people who respond to review comments by copy-pasting them into their LLM, then copy-pasting its response back onto GitHub. Bluntly: this is a waste of everyone’s time. If we wanted an LLM’s opinion, we could have asked it ourselves. We want to hear your thoughts, not a machine’s.
The Strands team, our community, and the industry have been guilty of doing this. We have AI code review bots that post huge pull requests that are overly verbose and tiring to read through. Instead of engaging with a user’s question on an issue, bots are either asked to give a response, or we copy and paste an AI-generated response. Sometimes humans aren’t even included in the loop, many interactions are purely with someone’s autonomous Agent. And when we scale this problem up to hundreds of pull requests on a repository, many low-quality AI-generated, it drowns out the productive human conversations that would otherwise raise the bar on quality. At the end of the day, what we want is to produce the best software together, and have fun while doing it, and the way we utilize agents today is making this harder on both fronts.
To be clear, we do not want to stop the usage of AI in our day-to-day work, there are significant benefits to its use. But we want to be prescriptive with how it is used, make sure there is space for humans to converse, and raise the overall quality of what we produce. This leads to the introduction of our AI Usage Policy - a list of insights and tenets that outline how AI should be used in the Strands community. We plan to use this policy to influence the work we do and the products we make, and we intend to enforce this policy with our community as well. We are open to feedback and plan to revise this policy as we learn more and grow alongside AI.
We want our stance here to be obvious and well represented on all our surfaces, and we want to guide both humans and agents toward better interactions. With that in mind, we will introduce changes to interactions in our repositories to help encourage everyone to follow this policy:
- The addition of the new AI Usage Policy added to our repository, as well as updates to our documentation to reflect our new stance here
- Additions and updates to our code contribution documentation to make it easier for everyone to contribute high-quality content
- An expectation that when posting comments to issues in Github an Agent will include a footnote in their comment to identify themselves as AI
- Updates to Issue and Pull Request templates to include “Human Overview” sections where humans are expected to give a short overview of a change introduced by an Agent
- The addition of labels on Issues and Pull Request that we deem are submitted entirely by AI, with the expectation that we will prioritize human contributions over these
If we find that anyone is not adhering to our policy, we plan on stepping in - first to warn and then block - those who do not respect our stance here. As the stewards of our community, we want to ensure that humans are continually encouraged to share their voice.