Re: DISCUSS: Deprecating and replacing current serializable string encodings

2017-12-05 Thread Jacob Barrett
On Mon, Dec 4, 2017 at 8:45 PM Michael Stolz wrote: > Anything that breaks data on disk is also a big PITA. This change would > break data on disk. Changing the on wire serialization format doesn't necessitate changing the on disk format. While it would be easier or more performant to have them

Re: DISCUSS: Deprecating and replacing current serializable string encodings

2017-12-05 Thread Jacob Barrett
On Mon, Dec 4, 2017 at 1:52 PM Dan Smith wrote: > The new protocol is currently translating from PDX->JSON before sending > results to the clients so the client doesn't have to understand PDX or > DataSerializable. > For now it does, but we all know that isn't a longer term viable solution. We w

Re: DISCUSS: Deprecating and replacing current serializable string encodings

2017-12-04 Thread Michael Stolz
Anything that breaks data on disk is also a big PITA. This change would break data on disk. -- Mike Stolz Principal Engineer, GemFire Product Lead Mobile: +1-631-835-4771 On Mon, Dec 4, 2017 at 1:52 PM, Dan Smith wrote: > The new protocol is currently translating from PDX->JSON before sending >

Re: DISCUSS: Deprecating and replacing current serializable string encodings

2017-12-04 Thread Dan Smith
The new protocol is currently translating from PDX->JSON before sending results to the clients so the client doesn't have to understand PDX or DataSerializable. There is a lot more to DataSerializable than just how a String is serialized. And it's not even documented that I am aware of. Just tweak

Re: DISCUSS: Deprecating and replacing current serializable string encodings

2017-12-04 Thread Michael William Dodge
I think there is value in having a single string encoding. Sarge > On 1 Dec, 2017, at 17:35, Jacob Barrett wrote: > > On Fri, Dec 1, 2017 at 4:59 PM Dan Smith wrote: > >> I think I'm kinda with Mike on this one. The existing string format does >> seem pretty gnarly. But the complexity of impl

Re: DISCUSS: Deprecating and replacing current serializable string encodings

2017-12-01 Thread Jacob Barrett
On Fri, Dec 1, 2017 at 4:59 PM Dan Smith wrote: > I think I'm kinda with Mike on this one. The existing string format does > seem pretty gnarly. But the complexity of implementing and testing all of > the backwards compatibility transcoding that would be required in order to > move to the new pro

Re: DISCUSS: Deprecating and replacing current serializable string encodings

2017-12-01 Thread Michael Stolz
It could always WRITE UTF-16 strings, but it will still need to be able to read the others. -- Mike Stolz Principal Engineer, GemFire Product Lead Mobile: +1-631-835-4771 On Fri, Dec 1, 2017 at 7:58 PM, Dan Smith wrote: > I think I'm kinda with Mike on this one. The existing string format does

Re: DISCUSS: Deprecating and replacing current serializable string encodings

2017-12-01 Thread Dan Smith
I think I'm kinda with Mike on this one. The existing string format does seem pretty gnarly. But the complexity of implementing and testing all of the backwards compatibility transcoding that would be required in order to move to the new proposed format seems to be way more work with much more poss

Re: DISCUSS: Deprecating and replacing current serializable string encodings

2017-12-01 Thread Michael Stolz
My opinion is that risk/reward on this on is not worth it -- Mike Stolz Principal Engineer - Gemfire Product Manager Mobile: 631-835-4771 On Dec 1, 2017 5:19 PM, "Jacob Barrett" wrote: > On Fri, Dec 1, 2017 at 1:48 PM Michael Stolz wrote: > > > There also would likely be Disk Stores that would

Re: DISCUSS: Deprecating and replacing current serializable string encodings

2017-12-01 Thread Jacob Barrett
On Fri, Dec 1, 2017 at 1:48 PM Michael Stolz wrote: > There also would likely be Disk Stores that would need to be converted. > That would be real ugly too. Disk store could transcode each entry on demand or all at once on first load. Not saying it will be easy but progress rarely is. -Jake

Re: DISCUSS: Deprecating and replacing current serializable string encodings

2017-12-01 Thread Jacob Barrett
On Fri, Dec 1, 2017 at 1:25 PM Bruce Schuchardt wrote: > I'm wondering how we would maintain backward compatibility with this > change? Geode accepts serialized data from a client and keeps it in > serialized form and might transmit this serialized data to an older > version peer or client. An

Re: DISCUSS: Deprecating and replacing current serializable string encodings

2017-12-01 Thread Michael Stolz
There also would likely be Disk Stores that would need to be converted. That would be real ugly too. -- Mike Stolz Principal Engineer - Gemfire Product Manager Mobile: 631-835-4771 On Dec 1, 2017 4:25 PM, "Bruce Schuchardt" wrote: > I'm wondering how we would maintain backward compatibility wit

Re: DISCUSS: Deprecating and replacing current serializable string encodings

2017-12-01 Thread Bruce Schuchardt
I'm wondering how we would maintain backward compatibility with this change?  Geode accepts serialized data from a client and keeps it in serialized form and might transmit this serialized data to an older version peer or client.  An older peer or client wouldn't be able to handle a new string

DISCUSS: Deprecating and replacing current serializable string encodings

2017-12-01 Thread Jacob Barrett
The current data serializer support 4 different encodings (in addition to the 3 or 4 other encodings I have found elsewhere in Geode). It supports encoding ASCII prefixed with an uint16 length (64K), ASCII prefixed with an uint32 length, Java Modified UTF-8 with unit16 length (64k) and finally UTF-