Re: New proposal for type definitons

I guess since JSON does not specify a storage type then yes, the least common denominator is BigDecimal in Java. Anything more than that may again be trying to add structure to something that isn't structured. On Tue, Jan 10, 2017 at 9:46 AM Bruce Schuchardt wrote: > Why don't we just support th

Re: New proposal for type definitons

Why don't we just support the Jackson DeserializationOptions? Some people are going to want BigDecimals instead of floating-point numbers because they're dealing with monitary values. Le 1/10/2017 à 6:40 AM, Jacob Barrett a écrit : On Mon, Jan 9, 2017 at 4:14 PM Darrel Schneider wrote:

Re: New proposal for type definitons

On Mon, Jan 9, 2017 at 4:14 PM Darrel Schneider wrote: > Pdx stores and index at the end of the blob so that it can lookup any of > the > fields without reading the preceding fields. > Thanks for that correction. If you stored every JSON number as a BigDecimal this would mean it would be > sto

Re: New proposal for type definitons

external tool or wrapper focused on sanitizing json documents without > > >> being > > >>> concerned with type ids or a central registry service. I could > picture > > >> just > > >>> having a single sanitize method that takes a json string and a >

Re: New proposal for type definitons

org/> and returns a cleaned up json > document. > >>> That seems like it would be a lot easier to implement and wouldn't > >> require > >>> the user to add typeIds to their json documents. > >>> > >>> I still feel like storing self desc

Re: New proposal for type definitons

cost. Dan, are you saying expose some interface to serialize/de and "query the some field in data - getFieldValue(fieldname)" dtata? Some sort of ExternalSerializer with getFieldValue() capability. From: Dan Smith To:dev@geode.apache.org Sent: Wednesday, December 21, 201

Re: New proposal for type definitons

can solve this issue by sorting the fields. Thanks.Hitesh. From: Dan Smith To: dev@geode.apache.org; Hitesh Khamesra Sent: Tuesday, January 3, 2017 4:46 PM Subject: Re: New proposal for type definitons Hi Hitesh, There are a few different ways to store self describing data. One wa

Re: New proposal for type definitons

gt; > > > wrote: > > > > > >>If we give people the option to store > > > and query self describing values, then users with inconsistent json > could > > > just use that option and pay the extra storage cost. > > > Dan, are you saying exp

Re: New proposal for type definitons

07 PM, Hitesh Khamesra > > > > wrote: > > > > > >>If we give people the option to store > > > and query self describing values, then users with inconsistent json > could > > > just use that option and pay the extra storage cost. > > > Dan,

Re: New proposal for type definitons

interface to serialize/de and "query the > > some field in data - getFieldValue(fieldname)" dtata? Some sort of > > ExternalSerializer with getFieldValue() capability. > > > > > > From: Dan Smith > > To: dev@geode.apache.org > > Sent: Wednesday,

Re: New proposal for type definitons

etFieldValue(fieldname)" dtata? Some sort of > ExternalSerializer with getFieldValue() capability. > > > From: Dan Smith > To: dev@geode.apache.org > Sent: Wednesday, December 21, 2016 6:20 PM > Subject: Re: New proposal for type definitons > > I'

Re: New proposal for type definitons

ame)" dtata?  Some sort of ExternalSerializer with getFieldValue() capability. From: Dan Smith To: dev@geode.apache.org Sent: Wednesday, December 21, 2016 6:20 PM Subject: Re: New proposal for type definitons I'm assuming the type ids here are a different set than the type

Re: New proposal for type definitons

+1 to what Jacob said. On Tue, Jan 3, 2017 at 11:39 AM, Jacob Barrett wrote: > A little late to the game here but I want to go back to Dan's idea of > storing the JSON or other self describing objects as a first class object > in Geode. As it stands right now an entry can be a POJO, Java seriali

Re: New proposal for type definitons

A little late to the game here but I want to go back to Dan's idea of storing the JSON or other self describing objects as a first class object in Geode. As it stands right now an entry can be a POJO, Java serialized object, or PDX, so why not other types? Seems perfectly reasonable to allow first

Re: New proposal for type definitons

You are correct here. Ordering the fields would be a simple solution IF the only problem was that fields were incorrectly ordered. In most cases not all fields are provided thus causing an explosion of type definitions that would be generated. On 12/22/16 16:11, Darrel Schneider wrote: When

Re: New proposal for type definitons

This is pretty interesting actually. It brings back the good parts of formal schema design. -- Mike Stolz Principal Engineer, GemFire Product Manager Mobile: 631-835-4771 On Fri, Dec 23, 2016 at 11:09 AM, Bruce Schuchardt wrote: > I wonder if it would be helpful to use JSON Schema < > http://js

Re: New proposal for type definitons

I wonder if it would be helpful to use JSON Schema as a starting point for this effort? Le 12/22/2016 à 6:45 PM, Udo Kohlmeyer a écrit : Ok, I will try and explain all of this better. --Udo On 12/22/16 16:42, Darrel Schneider wrote: The @refTypeId is hard to unders

Re: New proposal for type definitons

Ok, I will try and explain all of this better. --Udo On 12/22/16 16:42, Darrel Schneider wrote: The @refTypeId is hard to understand. It is unclear to me how it interacts with other things like "dataType" and "subType". I think you can either specify a dataType/subType OR a @refTypeId. Is this

Re: New proposal for type definitons

The thought was to process the type definition and optimize before generating the type def. Making it more generic or "wider" is a good idea. Maybe I try and keep to the already defined PDX types and see how they could map. --Udo On 12/22/16 16:11, Darrel Schneider wrote: When generating a

Re: New proposal for type definitons

I think the spec needs to be further extended. This was just draft 1 with the hope to spark some thought. I'll continue refining over time. The thought was to make it a generic "catalogue" which could potentially drive all type definitions, regardless of serialization mechanism. --Udo On 1

Re: New proposal for type definitons

Correct, The user will not be able to override an existing typeId + type definition. It will fail with the error message that an previous definition already exists. I imagine the process would be to remove a typeId (later feature) which could check if any data exists for that typeId. If no d

Re: New proposal for type definitons

The @refTypeId is hard to understand. It is unclear to me how it interacts with other things like "dataType" and "subType". I think you can either specify a dataType/subType OR a @refTypeId. Is this correct? The current spec makes it look like you can specify both but your example just show one or

Re: New proposal for type definitons

You proposal seems to be only handle types for JSON. I do not see that you support all the pdx field types. Also you have things like List and "subType" which currently have no support explicit support in the pdx type system. So do you intend this proposal to be specific to JSON? If so the gfsh an

Re: New proposal for type definitons

Something I did not see in your proposal was the rules that would be used when a JSON document uses "@typeId" to determine if that type is valid for the current document. For example I think you want to allow the type to have a field that does not exist in the document. I think you also want to say

Re: New proposal for type definitons

One danger of this solution is users may think they can modify a previously defined type. Since they specify the type they may think they can just edit the file and reload the types with modified definitions. In most cases if data has already been serialized using the old type then modifying the ty

Re: New proposal for type definitons

When generating a pdx type for a JSON document couldn't we sort the field names from the JSON document so that field order would not generated different pdx types? Also when choosing a pdx field type if we always picked a "wider" type then it would reduce the number of types generated because of di

Re: New proposal for type definitons

Hi there Dan, You are correct, the thought is there to add a flag to the registry to indicate that a definition is custom and thus should not conflict with the existing ids. Even if they types were to be stored with the current Pdx type definitions, upon loading/registration of the custom type

Re: New proposal for type definitons

+1 for self describing documents -- Mike Stolz Principal Engineer, GemFire Product Manager Mobile: 631-835-4771 On Wed, Dec 21, 2016 at 9:20 PM, Dan Smith wrote: > I'm assuming the type ids here are a different set than the type ids used > with regular PDX serialization so they won't conflict i

Re: New proposal for type definitons

I'm assuming the type ids here are a different set than the type ids used with regular PDX serialization so they won't conflict if the pdx registry assigns 1 to some class and a user puts @typeId: 1 in their json? I'm concerned that this won't really address the type explosion issue. Users that ar

New proposal for type definitons

Hey there, I've just completed a new proposal on the wiki for a new mechanism that could be used to define a type definition for an object. https://cwiki.apache.org/confluence/display/GEODE/Custom+External+Type+Definition+Proposal+for+JSON Primarily the new type definition proposal will hopef