Re: [DISCUSS] Publish Builds, not Snapshots

2020-04-27 Thread Owen Nichols
Not sure that’s true. Internally maven snapshot repos are versioned, and the maven-resource is able to download any desired version still on the server, even if maven itself doesn’t expose that capability. You just need to ensure your job uses the version put into the local maven cache by the

Re: [DISCUSS] Publish Builds, not Snapshots

2020-04-27 Thread Robert Houghton
But the "maven-resource" type is still non-repeatable for SNAPSHOTs. Resource pinning will not work, and identifying the actual GEODE build causing the failure won't work. On Mon, Apr 27, 2020 at 4:39 PM Owen Nichols wrote: > That sounds like a problem with geode-examples regardless of whether w

Re: [DISCUSS] Publish Builds, not Snapshots

2020-04-27 Thread Owen Nichols
That sounds like a problem with geode-examples regardless of whether we use -SNAPSHOT or -build.123 … we should replace the 24hr trigger with a maven-resource trigger (nulldriver, linkyard, or similar) > On Apr 27, 2020, at 4:34 PM, Robert Houghton wrote: > > @Owen: What geode-examples is doin

Re: [DISCUSS] Publish Builds, not Snapshots

2020-04-27 Thread Anthony Baker
If I build the /develop branch and run `gfsh version` what will it print? If I build the soon-to-be /release/1.13 branch and run `gfsh version` what will it print? Anthony On 4/27/20, 4:32 PM, "Robert Houghton" wrote: The artifact would change from "1.13.0-SNAPSHOT" to "1.13.0-build.123

Re: [DISCUSS] Publish Builds, not Snapshots

2020-04-27 Thread Donal Evans
Do we know if this is an issue that other open source projects have dealt with? And if so, is this proposed solution similar to what they might have done to remedy it? From: Robert Houghton Sent: Monday, April 27, 2020 4:31 PM To: dev@geode.apache.org Subject: R

Re: [DISCUSS] Publish Builds, not Snapshots

2020-04-27 Thread Robert Houghton
@Owen: What geode-examples is doing, is a perfect case study of the wrong behaviors predicated on 'SNAPSHOT' with Concourse. Concourse CI is built on repeatable builds with defined input. The current CI for examples is using a 24hr trigger because it has no valid dependency on the Geode develop pi

Re: [DISCUSS] Publish Builds, not Snapshots

2020-04-27 Thread Robert Houghton
The artifact would change from "1.13.0-SNAPSHOT" to "1.13.0-build.123". The number after the "build" slug is auto-incremented by our CI system anyway, as the "geode-build-version" semver resource. We are actually doing *more* work in Gradle to truncate that number from the current "SNAPSHOT" value

Re: Dangers of using sockets in unit tests

2020-04-27 Thread Kirk Lund
The problems in WindowsUnitTest jobs in CI are caused by various unit tests that are either bugged or should be moved to IntegrationTests. I found 3 tests named *IntegrationTests in src/test that need to move: * geode-core/src/test/java/org/apache/geode/distributed/internal/InternalDistributedSys

Re: [DISCUSS] Publish Builds, not Snapshots

2020-04-27 Thread Owen Nichols
Sounds good to me, based on the repeatability argument. Will the build number output by gfsh version --full match the maven artifact spec? I am unclear what geode-examples has to do with this. Aside from repeatability, there is no reason geode-examples should be having any problem with -SNAPS

Re: [DISCUSS] Publish Builds, not Snapshots

2020-04-27 Thread Anthony Baker
@Robert, can you show some examples of what the build number would be under this proposal? Does 1.13.0-SNAPSHOT become 1.13.0.N where N increments every build? Seems reasonable. Since the consumers of pre-release artifacts are either a) this project or b) close related projects for integratio

Re: [DISCUSS] Publish Builds, not Snapshots

2020-04-27 Thread Jacob Barrett
It is unfortunate that the Maven/Gradle community hasn’t addressed this glaring issue with SNAPSHOT for decades now (well maybe not decades but certainly decade). It is also unfortunate that the Maven version coordinate is ugly. Aside from that I am totally onboard. Yay for reproducible builds a

[DISCUSS] Publish Builds, not Snapshots

2020-04-27 Thread Robert Houghton
Hello to the community, tl;dr - Lets publish builds, not snapshots, for repeatable CI builds, as GEODE-8016[1]. Communicate desired artifact version via the existing 'UpdatePassingTokens' job. I have been working on the Geode build and CI systems for a long time, and it has irked me that the geod

Re: [DISCUSS] should we add some Windows PR checks?

2020-04-27 Thread Jacob Barrett
I will toss out there again that there are a number of tests in this set of tests that should have no impact by Windows. If it is truly a Unit tests then windows should note have any impact but we aren’t good about pure unit tests. Integration tests that interact with OS level components, like d

Re: [DISCUSS] should we add some Windows PR checks?

2020-04-27 Thread Owen Nichols
If supporting Windows is costly, I’ll toss out the suggestion that we scale back Windows testing in our regular pipeline as well. For example, we could run the Windows tests only once a week, once a month, or once per release. > On Apr 27, 2020, at 2:00 PM, Robert Houghton wrote: > > I would

Re: Dangers of using sockets in unit tests

2020-04-27 Thread Kirk Lund
Yes, that's correct, SocketCreatorFactoryJUnitTest does not actually use any sockets. I assume some unit test that ran before it used sockets. What I really mean to say is: (1) singletons are bad, please let's keep working to get rid of them, (2) please be careful with unit tests -- I continue to o

Re: Dangers of using sockets in unit tests

2020-04-27 Thread Bruce Schuchardt
This seems more like it should be another call to remove all singletons. Tshe SocketCreatorFactoryJUnitTest that failed doesn't create a socket. It just configures a SocketCreator and then asserts that it was correctly configured. On 4/27/20, 1:58 PM, "Kirk Lund" wrote: This test starte

Re: [DISCUSS] should we add some Windows PR checks?

2020-04-27 Thread Robert Houghton
I would like to *not* include them, until they can take advantage of our run-in-docker behavior, and not be dog slow. Plus, they are *very* expensive instances. On Mon, Apr 27, 2020 at 1:25 PM Donal Evans wrote: > Having just hit this issue myself, and seen it crop up a few times when I > was CI

Dangers of using sockets in unit tests

2020-04-27 Thread Kirk Lund
This test started failing consistently on Windows (5 builds in a row so far!): org.apache.geode.internal.net.SocketCreatorFactoryJUnitTest > testNewSSLConfigSSLComponentLocator FAILED java.lang.AssertionError at org.junit.Assert.fail(Assert.java:86) at org.junit.Assert.assertTr

Re: Proposal to include fix for GEODE-8020 in support/1.12

2020-04-27 Thread Bruce Schuchardt
Thanks - this has been merged to the support branch > On Apr 27, 2020, at 9:12 AM, Bruce Schuchardt wrote: > > commit ec8db54ad7f342542762beb8f3e912dff44e3a53 (HEAD -> develop, origin/develop) > > Author: Bruce Schuchardt > > Date: Mon Apr 27 09:07:16 2020

Re: [DISCUSS] should we add some Windows PR checks?

2020-04-27 Thread Donal Evans
Having just hit this issue myself, and seen it crop up a few times when I was CIO, I think it would definitely be valuable to add some level of Windows testing to PR pre-checkin. It's always better to catch issues early rather than late, and anything that helps keep the pipeline green gets my ap

[DISCUSS] should we add some Windows PR checks?

2020-04-27 Thread Owen Nichols
Currently we have zero Windows tests in the PR pipeline, which has led to more than a handful of times this year where a PR passed all its checks then failed the main pipeline once merged. Adding these 3 jobs to the PR pipeline would have caught the majority of them: WindowsUnitTestOpenJDK11 Win

Re: Proposal to include fix for GEODE-8020 in support/1.12

2020-04-27 Thread Anthony Baker
+1 > On Apr 27, 2020, at 9:14 AM, Kirk Lund wrote: > > +1 > > On Mon, Apr 27, 2020 at 9:12 AM Bruce Schuchardt > wrote: > >> commit ec8db54ad7f342542762beb8f3e912dff44e3a53 (HEAD -> develop, >> origin/develop) >> >> Author: Bruce Schuchardt >> >> Date: Mon Apr 27 09:07:16 2020 -0700 >>

Re: Proposal to include fix for GEODE-8020 in support/1.12

2020-04-27 Thread Kirk Lund
+1 On Mon, Apr 27, 2020 at 9:12 AM Bruce Schuchardt wrote: > commit ec8db54ad7f342542762beb8f3e912dff44e3a53 (HEAD -> develop, > origin/develop) > > Author: Bruce Schuchardt > > Date: Mon Apr 27 09:07:16 2020 -0700 > > > > GEODE-8020: buffer corruption in SSL communications (#4994) > > >

Re: Proposal to include fix for GEODE-8020 in support/1.12

2020-04-27 Thread Owen Nichols
+1 > On Apr 27, 2020, at 9:12 AM, Bruce Schuchardt wrote: > > commit ec8db54ad7f342542762beb8f3e912dff44e3a53 (HEAD -> develop, > origin/develop) > > Author: Bruce Schuchardt > > Date: Mon Apr 27 09:07:16 2020 -0700 > > > > GEODE-8020: buffer corruption in SSL communications (#4994)

Proposal to include fix for GEODE-8020 in support/1.12

2020-04-27 Thread Bruce Schuchardt
commit ec8db54ad7f342542762beb8f3e912dff44e3a53 (HEAD -> develop, origin/develop) Author: Bruce Schuchardt Date:   Mon Apr 27 09:07:16 2020 -0700     GEODE-8020: buffer corruption in SSL communications (#4994)     revert change in GEODE-6661 that made NioSslEngine use a direct buffer.