On Mon, Dec 4, 2017 at 1:52 PM Dan Smith <dsm...@pivotal.io> wrote: > The new protocol is currently translating from PDX->JSON before sending > results to the clients so the client doesn't have to understand PDX or > DataSerializable. >
For now it does, but we all know that isn't a longer term viable solution. We will either have to support PDX, at which time all clients will have to implement nightmare that is PDX, or replace it with something better. > There is a lot more to DataSerializable than just how a String is > serialized. And it's not even documented that I am aware of. Just tweaking > the string format is not going to make that much better. Your hypothetical > ruby developer is in trouble with or without this proposed change. > Very true but just because you can't fix everything doesn't mean you shouldn't try to fix something. > Breaking compatibility is a huge PITA for our users. We should do that when > we are actually giving them real benefits. In this case if we were > switching to some newer PDX format that was actually easy to implement > deserialization logic I could see the argument for breaking compatibility. > Just changing the string format without fixing the rest of the issues > around DataSerializable isn't providing real benefits. > Great, let's prioritize some new PDX format that doesn't have 4 different string encodings, 2 of which are incorrect and one that isn't a standard. > You can't assume that a client in one language will only be serializing > > strings for it's own consumption. > > > > I wasn't making that assumption. The suggestion is that the C++ client > would have to deserialize all 4 valid formats, but it could just always > serialize data using the valid UTF-16 format. All other clients should be > able to deserialize that. > Unfortunately its the Java Modified UTF-8 to anything that is the PITA since it doesn't comply with any standard it must be implemented by hand. Reading it and not writing it only makes it 1/2 a PITA. -Jake
