Re: LLM Usage Policy

[C. Scott Andreas]

Dinesh, I follow the point that you are making, but your reply dismisses Jane’s central concern without addressing it: the review burden. “Software testing, certification, and verification” are orthogonal concerns to the burden on reviewers and the activity of review: ensuring harmony with the overall codebase and architecture, replicating knowledge of the implementation among maintainers, and refining software together. These are functions that aren’t and can’t be addressed by software. They are the “community” part of code. – Scott On Sep 29, 2026, at 8:39 PM, Dinesh Joshi <djo...@apache.org> wrote:  Jane, Everything that you just said is true about human generated code as well. Believe it or not every engineer that hand writes code also believes that their code is perfect and they understand it completely. Yet reviewers find issues with it (and even they miss things that later get caught in testing or production). The issue with AI is that we can produce a lot of code very quickly and create a lot of work for reviewers. The problem we should be addressing is that of software testing, verification and certification of the database. Thanks, Dinesh On Tue, Sep 29, 2026 at 4:58 PM Jane H <jan...@apache.org> wrote: I'd support detailed and explicit policies, like Benedict's, Rust's, or Caleb's, at least they can actually help the problem of AI assisted code increasing reviewers' burden, though they might introduce other problems. Blake, why I think your 3 point simple rules won't work as you wish: people have very different understandings in what's considered responsible enough or comprehensive enough understanding.  Everyone who has brought AI-coded PRs for me to review, new contributors or committers, who costs around 4x time from me to review their PRs compared to human-written ones (already more time than implementing on my own), believes that they are fully responsible for their code and fully understand their code. Every one of them believes so. I tried reminding them that they have to be fully responsible for the verification of their code. Didn't help in my own experience. (I don't blame them. We are all figuring out how to use this new tool. And I voluntarily pick up the tasks of reviewing those PRs. Not their fault. But that's why we need explicit guidance/policies.) I've already met people who believe one of the following is considered responsible enough: - Read every line of code and think it makes sense. - Pass other AI's code reviews - Pass all existing tests and AI-coded new unit tests and integration tests But I think none of them is enough. Therefore, when someone says they are fully responsible for/fully understanding their code, it doesn't really reduce my reviewer's load. Those detailed and explicit policies like Benedict's or Rust's, are largely just defining what's considered responsible enough. So they will help.