On Wed, Dec 30, 2020, 10:34 PM Tyler Compton <[email protected]> wrote:
> This is a very interesting project! I went ahead and labeled a few of the > issues. However, much as I hate to pour cold water on the idea, I have some > doubts about how effective this approach can be. The githubvet bot has > reported over ten thousand issues so far, which is quite a lot. > This is a valid concern but since the tool remains under active development, I would caution against making an assessment based on the number of issues currently reported in the tracker. Part of the problem is that the current tracker contains issues from multiple runs, since I've rescanned during testing, as I find and fixing issues. There are still some things that can be done to bring down the false-positive rate, but before addressing that I need to spend time on some of the false-negatives that I am currently aware of. My gut feeling is that if code does exist that correctly relies on the > current behavior, it would be extremely subtle. If we're going to get > through all of these issues, we won't be able to give each example a > complete deep dive which I think means we'll likely miss very subtle cases. > Not that I have any better ideas :) > Part of the hope has been that a large number of community participants would help parallelize the effort and (perhaps) even get multiple pairs of eyes on each issue. Whether that will work or not remains to be seen. I will also say that many of the issues I have looked at have seemed to be classifiable rather quickly. OTOH, that could also be an indication that the false-positive rate remains a little high. All that said, so far, the issues reported only focus on race-conditions or incorrectly storing a reference to a range-loop pointer. Tim King has just pointed out a few other concerns like pointer comparison and finalizers for which analyzers have yet to be written. My perception is that these two features aren't very widely used, so I wouldn't expect to find a large number of instances like these. > On Sun, Dec 20, 2020 at 12:38 PM K. Alex Mills <[email protected]> > wrote: > >> Hello Gophers! >> >> During a Q&A session at this year's GopherCon, some members of the Go >> language team indicated a willingness to modify the behavior of range >> loops <https://github.com/golang/go/issues/20733> *if* we can be >> reasonably certain that the change will not cause incorrect behavior in >> current programs. To make that determination, a large collection of >> real-world go programs would need to be vetted. If we can find that, in >> every case, modifying the compiler behavior does not lead to an undesirable >> outcome, we will have strong evidence that the change may only have a small >> impact. >> >> I've been working on a project to gather and crowdsource data for that >> sort of analysis. It has reached a point where I think that it's time to >> share it with the Go community. >> >> The GitHub Vet Project <https://github.com/github-vet> performs static >> analysis on public Go repositories hosted on GitHub in order to better >> understand the impact of this proposed language change >> <https://github.com/golang/go/issues/20733>. It consists of two bots. >> VetBot crawls GitHub to snippets of code it thinks could be interesting, it >> reports it as an issue in a separate repository >> <https://github.com/github-vet/rangeloop-pointer-findings>, for humans >> to analyze via crowdsourcing. TrackBot >> <https://github.com/github-vet/bots/tree/main/cmd/track-bot> manages the >> crowdsourcing workflow. >> >> At this point, all of the features I think are necessary for the >> project's success have been implemented. But I am only one Gopher, and so I >> would like to a) make the community aware of the project and b) ask the >> community to help review it in three ways: >> >> 1) Test out TrackBot and the project instructions by actually >> crowdsourcing through the issues found so far >> <https://github.com/github-vet/rangeloop-pointer-findings>. >> 2) Review the output >> <https://github.com/github-vet/rangeloop-pointer-findings/issues> and >> raise concern if anything that I claim ought to be excluded >> <https://github.com/github-vet/bots/tree/main/cmd/vet-bot#2-run-static-analysis> >> is somehow making it through. >> 3) Static analysis experts: review the analyzers >> <https://github.com/github-vet/bots/tree/main/cmd/vet-bot> and >> ruthlessly question anything that could lead to a false negative. I don't >> think anything exists, but one pair of eyes is often wrong, and there are >> many folks on this list with (much) more expertise than me. >> >> One final thing. The crowd-sourcing workflow I've designed relies on the >> opinion of experts to help estimate the reliability of the community >> assessment. In order for that to work, I need the help of some Go experts >> who are willing to commit some time to reviewing the findings. If you'd >> like to participate in this way, first, read the TrackBot documentation >> <https://github.com/github-vet/bots/tree/main/cmd/track-bot> to >> understand what being an expert entails. If you're still interested, email >> me with the subject line 'GitHub Vet Expert' and include your GitHub >> username and a brief outline of your experience with Go. >> >> It's my hope that this project can provide some data that can help to >> move Go forward. To that end, I'm also interested in any and all feedback >> and suggestions. Contributions are also welcome >> <https://github.com/github-vet/bots/issues>. >> >> Thanks for reading, >> >> K. Alex Mills, Ph.D >> >> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/golang-nuts/CALJzkY86ZgQMQBqiAyDOKHHionjLZJbB6f%2BEDcMp82ZaPCKmXw%40mail.gmail.com >> <https://groups.google.com/d/msgid/golang-nuts/CALJzkY86ZgQMQBqiAyDOKHHionjLZJbB6f%2BEDcMp82ZaPCKmXw%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CALJzkY95%3DJnp4rAXTOFm0%3DC7z1mMs7oJ_pJ8OHzfdAX1ow%2Bbmw%40mail.gmail.com.
