+1 to what Anthony has laid out! I think this is a better way to handle value encodings, and it's also better to be putting message specific details like event id with those messages.
I do wonder whether this proposal actually needs metadata headers at all? What will eventually go in there? -Dan There’s some really good feedback and discussion in this thread! Here are > a few thoughts: > > 1) Optional metadata should be used for fields that are generally > applicable across all messages. If a metadata field is required or only > applies to a small set of messages, it should become part of a message > definition. Of course there’s some grey area here. > > 2) I think we should pull out the message fragmentation support to avoid > some significant complexity. We can later add a fragmentation / envelope > layer on top without disrupting the current proposal. I do think we should > add the capability for chunking data (more on that below). > > 3) I did not find any discussion of message pipelining (queuing multiple > requests on a socket without waiting for a response) or out-of-order > responses. What is the plan for these capabilities and how will that > affect consistency? What about retries? > > 4) Following is an alternative definition with these characteristics: > > - Serialized data can either be primitive or encoded values. Encoded > values are chunked as needed to break up large objects into a series of > smaller parts. > - Because values can be chunked, the size field is removed. This allows > the message to be streamed to the socket incrementally. > - The apiVersion is removed because we can just define a new body type > with a new apiId (e.g. GetRequest2 with aipId = 1292). > - The GetRequest tells the server what kind of encoding the client is able > to understand. > - The metadata map is not used for fields that belong in the message > body. I think it’s much easier to write a spec without if statements :-) > > Message => MessageHeader MessageBody > > MessageHeader => correlationId metadata > correlationId => integer > metadata => count (key value)* > count => integer > key => string > value => string > > MessageBody => apiId body > apiId => integer > body => (see specific definitions) > > GetRequest => 0 acceptEncoding key > 0 => the API id > acceptEncoding => (define some encodings for byte[], JSON, PDX, *, etc) > key => EncodedValue > > GetResponse => 1 value > 1 => the API id > value => EncodedValue > > PutRequest => 2 eventId key value > 2 => the API id > eventId => clientId threadId sequenceId > clientId => string > threadId => integer > sequenceId => integer > key => EncodedValue > value => EncodedValue > > EncodedValue => encoding (boolean | integer | number | string | ((length > bytes)* 0)) > encoding => (define some encodings for byte[], JSON, PDX, *, etc) > boolean => TRUE or FALSE > integer => a signed integer value > number => a decimal value corresponding to IEEE 754 > string => UTF-8 text > bytes => arbitrary byte[] that can be chunked > > > Anthony > >
