Let's not vote until there is a call to vote, folks...
On Thu, Jan 16, 2020, 18:31 Jacob Barrett <jbarr...@pivotal.io> wrote: > I would characterize my vote as 0. I really don’t care either way. Just > sharing I think they have no value in a release. > > > On Jan 16, 2020, at 6:08 PM, Owen Nichols <onich...@pivotal.io> wrote: > > > > Geode PMC has 52 members. If this were a vote, it looks like the > results would have been: > > +1: 2 (Anthony, Dan) > > -1: 1 (Jake) > > > > If the next release manager were to go ahead and put geode-benchmarks in > the Geode 1.12.0 source release, at least 3 PMC members would need to be > willing to vote +1. So it sounds like we need a few more of the other 49 > PMC members to weigh in on this discussion. > > > > To summarize so far: > > > > Proposal: > > - add a geode-benchmarks-n.n.n-src.tgz artifact to all Geode releases > going forward, starting with 1.12.0 > > > > Arguments in favor: > > - why not? > > - it’s already public > > - we should default to including all things > > - it might be of interest to the user community > > - it might encourage contributions back to further improve it > > - it is required by CI, which is already included > > - Apache mandates that source releases must include test code too > > > > Arguments against: > > - doing nothing is less work > > - it will burden PMC members with additional work to validate and vote > on RCs > > - nobody outside the dev community has asked for it to be included > > - maybe it’s not ready > > - maybe it’s not documented well enough > > - it’s not needed to use Geode > > - Apache's legal separation between dev stuff and public release stuff > > - legal or license review may be not have been conducted yet > > > > > >>> On Jan 16, 2020, at 4:48 PM, Dan Smith <dsm...@pivotal.io> wrote: > >>> > >>> If geode-benchmarks is included, that implies that an RC cannot be > >> approved until reviewers can successfully run the benchmark suite from > the > >> geode-benchmarks source distribution. Is that what we want? > >> > >> I think it would be sufficient to run the tests of the benchmarks, eg > >> ./gradlew test > >> > >>> Deploying CI pipelines and running Benchmarks seems like a prime > example > >> of things we’d be happy to help others in the community with on the dev > >> list — but not something we would expect questions about on the user > list. > >> > >> I think it would be valuable to share our benchmarks with the geode user > >> community. The benchmark framework itself (the harness module) is a > fairly > >> generic benchmarking framework than can be used to benchmark anything > that > >> can be spun up using java. The geode-benchmark module has geode > benchmarks > >> that could be used for testing specific hardware, for example. > >> > >> -Dan > >> > >>> On Thu, Jan 16, 2020 at 12:37 PM Owen Nichols <onich...@pivotal.io> > wrote: > >>> > >>> When voting on RC candidates, PMC members "are required to download the > >>> signed source code package, compile it as provided, and test the > resulting > >>> executable on their own platform”. > >>> > >>> If geode-benchmarks is included, that implies that an RC cannot be > >>> approved until reviewers can successfully run the benchmark suite from > the > >>> geode-benchmarks source distribution. Is that what we want? > >>> > >>> Similarly, if CI is included, that seems to imply that an RC cannot be > >>> approved until reviewers can stand up their own pipeline from the > geode/ci > >>> source distribution. Is that what we want? > >>> > >>> So far there doesn’t seem to be consensus on what to include in a Geode > >>> source release, but let’s keep in mind that anything we add to the > release > >>> becomes an Act Of The Foundation and is held to a higher standard. > Apache > >>> makes a clear distinction between between development activity and > official > >>> releases to the public. Development activity is anything that should > stay > >>> within the dev list. Deploying CI pipelines and running Benchmarks > seems > >>> like a prime example of things we’d be happy to help others in the > >>> community with on the dev list — but not something we would expect > >>> questions about on the user list. > >>> > >>>> On Jan 16, 2020, at 10:23 AM, Dan Smith <dsm...@pivotal.io> wrote: > >>>> > >>>> We are supposed to be including all of the source necessary to test > Geode > >>>> in the source release [1] - I think that would include benchmarks as > >>> well. > >>>> > >>>> I don't really see any compelling reason *not* to include the > benchmarks, > >>>> let's go ahead and get them into our source release! > >>>> > >>>> [1] > >>>> > >>> > http://www.apache.org/legal/release-policy.html#what-must-every-release-contain > >>>> > >>>> On Wed, Jan 15, 2020 at 10:26 PM Owen Nichols <onich...@pivotal.io> > >>> wrote: > >>>> > >>>>> +1 for no changes > >>>>> > >>>>> On Wed, Jan 15, 2020 at 5:57 PM Jacob Barrett <jbarr...@pivotal.io> > >>> wrote: > >>>>> > >>>>>> We can live in areas of gray that don’t require any changes. Nobody > is > >>>>>> asking for benchmarks so let’s not do work to add them. Nobody is > >>>>>> complaining they CI is included so let’s not do work to remove > them. Is > >>>>> it > >>>>>> ideal, meh... > >>>>>> > >>>>>>> On Jan 15, 2020, at 5:50 PM, Mark Hanson <mhan...@pivotal.io> > wrote: > >>>>>>> > >>>>>>> Just my two cents. > >>>>>>> > >>>>>>> I think that we should probably strip CI into a separate repo. The > key > >>>>>> indicator is that if something were wrong in the CI yaml, would I > hold > >>> a > >>>>>> release for that? I think no. So that suggests to me it is a > separate > >>>>>> thing. Same goes for benchmarks. If we were failing a benchmark I > would > >>>>> be > >>>>>> concerned, but if the script were broken, would I hold the release? > I > >>>>> think > >>>>>> no as well. > >>>>>>> > >>>>>>> I think that says that the CI code should also be a separate repo. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Mark > >>>>>>> > >>>>>>>> On Jan 14, 2020, at 10:21 PM, Jacob Barrett <jbarr...@pivotal.io> > >>>>>> wrote: > >>>>>>>> > >>>>>>>> Until someone outside of the geode ci community is asking for it I > >>>>> just > >>>>>> don’t see utility in going through the motions of making a release > for > >>>>> it. > >>>>>>>> > >>>>>>>>>> On Jan 14, 2020, at 10:13 PM, Owen Nichols <onich...@pivotal.io > > > >>>>>> wrote: > >>>>>>>>> > >>>>>>>>> The source is already public, so on some level a source release > is > >>>>> no > >>>>>> different from a git tag. Benchmarks has matured enough that I > think > >>> it > >>>>>> makes sense to at least start branching and tagging the > >>> geode-benchmarks > >>>>>> repo to capture exactly what was used in that Geode release. > >>>>>>>>> > >>>>>>>>> Others in the dev and user community may find the benchmarks > useful > >>>>> in > >>>>>> other ways than we use them. While our focus for CI is on tuning > for > >>>>>> repeatability, someone else might just want a load generator to > break > >>> in > >>>>> a > >>>>>> new cluster or get some rough numbers. Some might want to get under > >>> the > >>>>>> hood and tinker and tune, or contribute their own benchmarks, with > the > >>>>>> understanding that it’s not a turnkey or standalone product, but a > tool > >>>>>> that requires getting your hands dirty. > >>>>>>>>> > >>>>>>>>> Would a “1 page” readme with a few tips on “how to run on a > laptop” > >>>>> be > >>>>>> enough to let other interested contributors help get > geode-benchmarks > >>> to > >>>>> a > >>>>>> “better state”? > >>>>>>>>> > >>>>>>>>>> On Jan 14, 2020, at 9:38 PM, Jacob Barrett <jbarr...@pivotal.io > > > >>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>> I don’t think the benchmarks provide any material benefit to a > user > >>>>>> in their current state. They are heavily tuned for our CI process > which > >>>>>> relies on very beefy machines. Usage on other hardware will require > >>> more > >>>>>> tuning. I don’t think it’s worth putting the source in the release > >>> until > >>>>>> they are in a better state. > >>>>>>>>>> > >>>>>>>>>> -Jake > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>>> On Jan 14, 2020, at 4:14 PM, Dan Smith <dsm...@pivotal.io> > >>> wrote: > >>>>>>>>>>> > >>>>>>>>>>> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols < > >>> onich...@pivotal.io > >>>>>> > >>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> I believe the desire is to include the source code for > >>>>>> geode-benchmarks as > >>>>>>>>>>>> part of the official geode release, much like how we include > >>>>>> geode-examples. > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Oh! I thought you meant running the benchmarks in the release > >>>>>> pipeline - I > >>>>>>>>>>> think last release we were running them but decided they were > too > >>>>>> flaky to > >>>>>>>>>>> use. > >>>>>>>>>>> > >>>>>>>>>>> +1 to including the benchmark source in the source release. > >>>>>>>>>>> > >>>>>>>>>>> -Dan > >>>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > >>> > > >
