Re: Lucene upgrade

2019-11-06 Thread Mario Kevo
Hi Dan, thanks for suggestions. I didn't found a way to write lucene in older format. They only support reading old format indexes with newer version by using lucene-backward- codec. Regarding to freeze writes to the lucene index, that means that we need to start locators and servers, create luce

Re: bug fix needed for release/1.11.0

2019-11-06 Thread Bruce Schuchardt
The fix for this problem is in the CI pipeline today: https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/Build/builds/1341 On 11/5/19 10:49 AM, Owen Nichols wrote: +1 for bringing this fix to release/1.11.0 (after it has passed Benchmarks on develop) On Nov 5

Re: bug fix needed for release/1.11.0

2019-11-06 Thread Mark Hanson
Any other votes? I have 2 people in favor. Voting will close at noon. Thanks, Mark > On Nov 6, 2019, at 8:00 AM, Bruce Schuchardt wrote: > > The fix for this problem is in the CI pipeline today: > https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/Build/builds

Odg: bug fix needed for release/1.11.0

2019-11-06 Thread Mario Ivanac
+1 for bringing this fix to release/1.11.0 Šalje: Mark Hanson Poslano: 6. studenog 2019. 18:28 Prima: dev@geode.apache.org Predmet: Re: bug fix needed for release/1.11.0 Any other votes? I have 2 people in favor. Voting will close at noon. Thanks, Mark > On No

Re: Odg: bug fix needed for release/1.11.0

2019-11-06 Thread Mark Hanson
Thanks Mario. Your vote reminded me not all voters are in the PST time zone.. Pardon my thoughtlessness.. Voting closes at 12pm PST > On Nov 6, 2019, at 9:33 AM, Mario Ivanac wrote: > > +1 for bringing this fix to release/1.11.0 > > Šalje: Mark Hanson > Posl

Re: Odg: bug fix needed for release/1.11.0

2019-11-06 Thread Owen Nichols
Also perhaps some people are waiting to see how the fix actually fares in the develop pipeline, which we won’t know until around 4 or 5PM today... > On Nov 6, 2019, at 9:44 AM, Mark Hanson wrote: > > Thanks Mario. Your vote reminded me not all voters are in the PST time zone.. > Pardon my thou

Re: Odg: bug fix needed for release/1.11.0

2019-11-06 Thread Ernest Burghardt
we could just use GMT (Geode Mean Time) On Wed, Nov 6, 2019 at 9:45 AM Mark Hanson wrote: > Thanks Mario. Your vote reminded me not all voters are in the PST time > zone.. Pardon my thoughtlessness.. > > Voting closes at 12pm PST > > > > On Nov 6, 2019, at 9:33 AM, Mario Ivanac wrote: > > > > +

IncrementalBackupDistributedTest.testMissingMemberInBaseline hangs

2019-11-06 Thread Kirk Lund
IncrementalBackupDistributedTest.testMissingMemberInBaseline is hanging intermittently in the DistributedTest job of CI and precheckin. I filed GEODE-7411 with all the involved thread stacks that I could find: https://issues.apache.org/jira/browse/GEODE-7411 If anyone knows of any recent changes

Re: IncrementalBackupDistributedTest.testMissingMemberInBaseline hangs

2019-11-06 Thread Jason Huynh
I'm working on a fix and have a PR up for another hang in the same test that I think fixes this issue. https://github.com/apache/geode/pull/4255 On Wed, Nov 6, 2019 at 10:47 AM Kirk Lund wrote: > IncrementalBackupDistributedTest.testMissingMemberInBaseline is hanging > intermittently in the Dis

Re: Lucene upgrade

2019-11-06 Thread Jason Huynh
Hi Mario, I think there are a few ways to accomplish what Dan was suggesting...Dan or other's, please chime in with more options/solutions. 1.) We add some product code/lucene listener to detect whether we have old versions of geode and if so, do not write to lucene on the newly updated node unti

Re: Lucene upgrade

2019-11-06 Thread Jacob Barrett
What about “versioning” the region that backs the indexes? Old servers with old license would continue to read/write to old region. New servers would start re-indexing with the new version. Given the async nature of the indexing would the mismatch in indexing for some period of time have an impa

Re: Lucene upgrade

2019-11-06 Thread Xiaojian Zhou
He tried to upgrade lucene version from current 6.6.4 to 8.2. There're some challenges. One challenge is the codec changed, which caused the format of index is also changed. That's why we did not implement it. If he resolved the coding challenges, then rolling upgrade will probably need option-2

Re: IncrementalBackupDistributedTest.testMissingMemberInBaseline hangs

2019-11-06 Thread Kirk Lund
Thanks Jason! I reviewed your PR and added a comment/question. On Wed, Nov 6, 2019 at 10:59 AM Jason Huynh wrote: > I'm working on a fix and have a PR up for another hang in the same test > that I think fixes this issue. > > https://github.com/apache/geode/pull/4255 > > On Wed, Nov 6, 2019 at 10

Re: Lucene upgrade

2019-11-06 Thread Jason Huynh
Jake, -from my understanding, the implementation details of geode-lucene is that we are using a partitioned region as a "file-system" for lucene files. As new servers are rolled, the issue is that the new servers have the new codec. As puts occur on the users data region, the async listeners are

Re: Lucene upgrade

2019-11-06 Thread Dan Smith
> > 1.) We add some product code/lucene listener to detect whether we have old > versions of geode and if so, do not write to lucene on the newly updated > node until all versions are up to date. Elaborating on this option a little more, this might be as simple as something like the below at the

Re: Lucene upgrade

2019-11-06 Thread Jacob Barrett
> On Nov 6, 2019, at 2:16 PM, Jason Huynh wrote: > > Jake, -from my understanding, the implementation details of geode-lucene is > that we are using a partitioned region as a "file-system" for lucene > files. Yeah, I didn’t explain well. I mean to say literally create a new region for the n

Re: Lucene upgrade

2019-11-06 Thread Jason Huynh
Dan - LGTM check it in! ;-) (kidding of course) Jake - there is a side effect to this in that the user would have to reimport all their data into the user defined region too. Client apps would also have to know which of the regions to put into.. also, I may be misunderstanding this suggestion, co

Re: Lucene upgrade

2019-11-06 Thread Jacob Barrett
> On Nov 6, 2019, at 3:36 PM, Jason Huynh wrote: > > Jake - there is a side effect to this in that the user would have to > reimport all their data into the user defined region too. Client apps > would also have to know which of the regions to put into.. also, I may be > misunderstanding this

Re: bug fix needed for release/1.11.0

2019-11-06 Thread Helena Bales
+1 to cherry-picking the fix. The sha hasn't made it to benchmarks yet due to an issue with CI losing resource refs that were needed to keep it moving through the pipeline. The next commit is still about an hour away from triggering benchmarks. In my manual benchmarking of this change, I found tha

Re: Lucene upgrade

2019-11-06 Thread Xiaojian Zhou
Usually re-creating region and index are expensive and customers are reluctant to do it, according to my memory. We do have an offline reindex scripts or steps (written by Barry?). If that could be an option, they can try that offline tool. I saw from Mario's email, he said: "I didn't found a way