upgrade of geode version

[Yossi Reginiano]

Hi team,

We are currently using geode version 1.4 and wish to upgrade to version 1.10
We are still using cpp client geode-native - 
pre-modernization
 (that was released on the 8th of may 2017), can we still work with this client 
after the upgrade to 1.10?
If not - which cpp client version is suitable?

Thanks,
Yossi Reginiano

This email and the information contained herein is proprietary and confidential 
and subject to the Amdocs Email Terms of Service, which you may review at 
https://www.amdocs.com/about/email-terms-of-service 

Re: upgrade of geode version

[Alberto Gomez]

Hi Yossi,

Version 1.4 of the cpp native client should work with Geode version 1.10. 
Client/server backward compatibility is a requirement in Geode as described 
here:

https://cwiki.apache.org/confluence/display/GEODE/Managing+Backward+Compatibility

-
Client/server backward compatibility

Client/server backward compatibility must work for all releases. All previous 
versions of clients must be supported because they may be embedded in 
applications that users would find difficult, if not impossible, to track down 
and upgrade.

-


Nevertheless, it is always advisable to use the client in the latest release 
available, so if it is possible for you to upgrade the clients, the 
recommendation would be to upgrade to cpp native client version 1.10.

Best regards,

/Alberto


From: Yossi Reginiano 
Sent: Thursday, December 12, 2019 9:24 AM
To: dev@geode.apache.org 
Cc: u...@geode.apache.org 
Subject: upgrade of geode version


Hi team,



We are currently using geode version 1.4 and wish to upgrade to version 1.10

We are still using cpp client geode-native - 
pre-modernization
 (that was released on the 8th of may 2017), can we still work with this client 
after the upgrade to 1.10?

If not – which cpp client version is suitable?



Thanks,

Yossi Reginiano



This email and the information contained herein is proprietary and confidential 
and subject to the Amdocs Email Terms of Service, which you may review at 
https://www.amdocs.com/about/email-terms-of-service

Odg: Odg: Odg: Odg: Lucene upgrade

[Mario Kevo]

Hi Jason,

Yes, the same tests failed:

RollingUpgradeQueryReturnsCorrectResultAfterTwoLocatorsWithTwoServersAreRolled

RollingUpgradeQueryReturnsCorrectResultsAfterServersRollOverOnPartitionRegion

Sometimes this tests passed but more times it failed.
As I said when change tests to put lower number of entries it passed every time 
or set to wait for repo in LuceneQueryFunction.java.

waitUntilFlushed is called by verifyLuceneQueryResults before executing 
queries. Also tried to wait until isIndexingInProgress return false, but 
reached timeout and failed.
In tests it tried to execute a query after all members are rolled.

BR,
Mario


Šalje: Jason Huynh 
Poslano: 11. prosinca 2019. 23:08
Prima: Mario Kevo 
Kopija: geode 
Predmet: Re: Odg: Odg: Odg: Lucene upgrade

Hi Mario,

Is the same test failing?  If it's a different test, could you tell us which 
one?
If it's a rolling upgrade test, then we might have to mark this as expected 
behavior and modify the tests to waitForFlush (wait until the queue is 
drained).  As long as the test is able to roll all the servers and not get 
stuck waiting for a queue to flush (which will only happen once all the servers 
are rolled now).

If the test hasn't rolled all the servers and is trying to execute a query, 
then we'd probably have to modify the test to not do the query in the middle or 
expect that exception to occur.

Thanks,
-Jason

On Wed, Dec 11, 2019 at 6:43 AM Mario Kevo  wrote:
Hi Jason,

This change fix IndexFormatTooNewException, but now we have

 org.apache.geode.cache.lucene.LuceneQueryException: Lucene Index is not 
available, currently indexing

So this means that query doesn't wait until all indexes are created.
In LuceneQueryFunction.java it is set to not wait for repo [execute(context, 
false)]. If we have a bigger queue(like in the test) it will failed as it will 
not wait until indexes are created. I also tried to put just few objects and it 
passed as it had enough time to create indexes.
Do we need to change this part to wait for repo, or put a lower number of 
entries in tests?

BR,
Mario




Šalje: Jason Huynh mailto:jhu...@pivotal.io>>
Poslano: 6. prosinca 2019. 20:53
Prima: Mario Kevo 
Kopija: geode mailto:dev@geode.apache.org>>
Predmet: Re: Odg: Odg: Lucene upgrade

Hi Mario,

I made a PR against your branch for some of the changes I had to do to get past 
the Index too new exception.  Summary - repo creation, even if no writes occur, 
appear to create some meta data that the old node attempts to read and blow up 
on.

The pr against your branch just prevents the repo from being constructed until 
all old members are upgraded.
This requires test changes to not try to validate using queries (since we 
prevent draining and repo creation, the query will just wait)

The reason why you probably were seeing unsuccessful dispatches, is because we 
kind of intended for that with the oldMember check.  In-between the server 
rolls, the test was trying to verify, but because not all servers had upgraded, 
the LuceneEventListener wasn't allowing the queue to drain on the new member.

I am not sure if the changes I added are acceptable or not -maybe if this ends 
up working then we can discuss on the dev list.

There will probably be other "gotcha's" along the way...


On Fri, Dec 6, 2019 at 1:12 AM Mario Kevo  wrote:
Hi Jason,

I tried to upgrade from 6.6.2 to 7.1.0 and got the following exception:

org.apache.lucene.index.IndexFormatTooNewException: Format version is not 
supported (resource BufferedChecksumIndexInput(segments_2)): 7 (needs to be 
between 4 and 6)

It looks like the fix is not good.

What I see (from 
RollingUpgradeQueryReturnsCorrectResultsAfterServersRollOverOnPartitionRegion.java)
 is when it doing upgrade of a locator it will shutdown and started on the 
newer version. The problem is that server2 become a lead and cannot read lucene 
index on the newer version(Lucene index format has changed between 6 and 7 
versions).

Another problem is after the rolling upgrade of locator and server1 when 
verifying region size on VMs. For example,

expectedRegionSize += 5;
putSerializableObjectAndVerifyLuceneQueryResult(server1, regionName, 
expectedRegionSize, 5,
15, server2, server3);

First it checks if region has expected size for VMs and it passed(has 15 
entries). The problem is while executing verifyLuceneQueryResults, for 
VM1(server2) it has 13 entries and assertion failed.
>From logs it can be seen that two batches are unsuccessfully dispatched:

[vm0] [warn 2019/12/06 08:31:39.956 CET  tid=0x42] During normal 
processing, unsuccessfully dispatched 1 events (batch #0)

[vm0] [warn 2019/12/06 08:31:40.103 CET  tid=0x46] During normal 
processing, unsuccessfully dispatched 1 events (batch #0)

For VM0(server1) and VM2(server3) it has 14 entries, one is unsuccessfully 
dispatched.

I don't know why some events are successfully dispatched, some not.
Do you have any idea

Re: upgrade of geode version

[Jacob Barrett]

Yossi,

While the statements about backwards compatibility are generally true please 
keep in mind that geode-native pre-modernization was never an official release. 
I strongly recommend you upgrade to an official release. Several bugs, memory 
leaks and security issues have been addressed in the official released versions 
of geode-native.

Unfortunately for you, the pre-modernization branch does not have a simple 
migration path since the official release of geode-native changes almost every 
API to use modern C++11 standards. It isn’t a terribly difficult task to make 
the conversion and this list will be happy to provide assistance with specific 
questions on changes.

A couple of hints:

There is no singleton cache anymore. Create your Cache instance and hold onto 
it. When it is destructed the cache will be closed and destroyed. You can now 
have more than one Cache in a process space. Cache is moveable so you can move 
it into a shared_ptr or unique_ptr.
SharedPtr is now std::shared_ptr.
Use auto keyword everywhere you can.
Nearly all raw pointers are now std::unique_ptr, std::shared_ptr, or references.
All char* have been converted to std::string.
All std::strings are expected to be UTF-8 encoded.
std::u16string are expected to be UTF-16
std::u32string are expected to be UTF-32
std::wstring are platform dependent, but generally UTF-16 or UTF-32.


-Jake



> On Dec 12, 2019, at 2:30 AM, Alberto Gomez  wrote:
> 
> Hi Yossi,
> 
> Version 1.4 of the cpp native client should work with Geode version 1.10. 
> Client/server backward compatibility is a requirement in Geode as described 
> here:
> 
> https://cwiki.apache.org/confluence/display/GEODE/Managing+Backward+Compatibility
> 
> -
> Client/server backward compatibility
> 
> Client/server backward compatibility must work for all releases. All previous 
> versions of clients must be supported because they may be embedded in 
> applications that users would find difficult, if not impossible, to track 
> down and upgrade.
> 
> -
> 
> 
> Nevertheless, it is always advisable to use the client in the latest release 
> available, so if it is possible for you to upgrade the clients, the 
> recommendation would be to upgrade to cpp native client version 1.10.
> 
> Best regards,
> 
> /Alberto
> 
> 
> From: Yossi Reginiano 
> Sent: Thursday, December 12, 2019 9:24 AM
> To: dev@geode.apache.org 
> Cc: u...@geode.apache.org 
> Subject: upgrade of geode version
> 
> 
> Hi team,
> 
> 
> 
> We are currently using geode version 1.4 and wish to upgrade to version 1.10
> 
> We are still using cpp client geode-native - 
> pre-modernization
>  (that was released on the 8th of may 2017), can we still work with this 
> client after the upgrade to 1.10?
> 
> If not – which cpp client version is suitable?
> 
> 
> 
> Thanks,
> 
> Yossi Reginiano
> 
> 
> 
> This email and the information contained herein is proprietary and 
> confidential and subject to the Amdocs Email Terms of Service, which you may 
> review at https://www.amdocs.com/about/email-terms-of-service