On 19 October 2017 at 14:05, Mikael Ståldal <mi...@apache.org> wrote:
> And don't forget a corresponding LogEventParser. Right. The basic idea here would be to provide both a reference encoder and decoder. If an existing binary format such as BSON, Avro, or Thrift were appropriate, then there wouldn't be much need here other than publishing a schema, but all 3 of those projects create non-trivial garbage during encoding and don't really have a use case in minimal memory usage or high performance encoding like we strive for in log4j. > > > > On 2017-10-19 18:56, Remko Popma wrote: > >> I agree with Gary: this is just another Layout. Should not need another >> repo... >> Prototyping on another branch makes sense. >> >> On Fri, Oct 20, 2017 at 1:23 AM, Gary Gregory <garydgreg...@gmail.com> >> wrote: >> >> Bleh, ANOTHER repo? We have so many already... but I see what the big >>> picture is. Would this add a new layout in Core? Maybe we should just >>> start >>> with that... then grow... >>> >>> Gary >>> >>> On Thu, Oct 19, 2017 at 9:57 AM, Matt Sicker <boa...@gmail.com> wrote: >>> >>> I mean in the logging services project. So it'd be a new repo. >>>> >>>> On 19 October 2017 at 10:48, Ralph Goers <ralph.go...@dslextreme.com> >>>> wrote: >>>> >>>> When you say “another component in the project” do you mean logging >>>>> services project or log4j? I’d prefer to see you do this in a separate >>>>> >>>> repo >>>> >>>>> or at least a branch until we understand what it looks like. If it is >>>>> >>>> going >>>> >>>>> to apply to many things it probably makes sense to be a separate repo. >>>>> >>>>> Ralph >>>>> >>>>> On Oct 19, 2017, at 8:26 AM, Matt Sicker <boa...@gmail.com> wrote: >>>>>> >>>>>> For generic structured records, I'd probably go with Avro or Thrift >>>>>> >>>>> since >>>> >>>>> LogEvents have a lot of standard fields with only a few optional >>>>>> >>>>> map-like >>>> >>>>> structures. For optimized log appending, the binary format was >>>>>> >>>>> proposed >>> >>>> as >>>>> >>>>>> a way to append quickly and without garbage IIRC. >>>>>> >>>>>> On 19 October 2017 at 10:21, Gary Gregory <garydgreg...@gmail.com> >>>>>> >>>>> wrote: >>>>> >>>>>> >>>>>> What about BSON? >>>>>>> >>>>>>> Gary >>>>>>> >>>>>>> On Oct 19, 2017 08:41, "Matt Sicker" <boa...@gmail.com> wrote: >>>>>>> >>>>>>> I don't have the ticket on hand, but a few months ago, Remko >>>>>>>> >>>>>>> suggested >>>> >>>>> a >>>>> >>>>>> binary logging format that would allow for super fast appends of >>>>>>>> log-specific information along with companion files for additional >>>>>>>> >>>>>>> metadata >>>>>>> >>>>>>>> not commonly used in log messages. I've been thinking about this >>>>>>>> >>>>>>> idea a >>>> >>>>> bit >>>>>>> >>>>>>>> in relation to existing structured layouts (both textual and >>>>>>>> >>>>>>> binary), >>> >>>> and I >>>>>>> >>>>>>>> was thinking that it might be a useful format to standardize on for >>>>>>>> >>>>>>> all >>>> >>>>> the >>>>>>> >>>>>>>> logging projects. >>>>>>>> >>>>>>>> What I'd like to propose is making another component in the project >>>>>>>> >>>>>>> that >>>>> >>>>>> would contain a reference implementation of encoding and decoding >>>>>>>> >>>>>>> the >>> >>>> format in Java, C++, .NET, and PHP (or as a C binding for PHP). >>>>>>>> Potentially, this format could be inclusive with other logging >>>>>>>> >>>>>>> projects >>>> >>>>> like Logstash, Logback, Splunk, etc. >>>>>>>> >>>>>>>> What do you all think? Is this a good idea? Or would this be >>>>>>>> >>>>>>> duplicating >>>>> >>>>>> effort from other standards already? >>>>>>>> >>>>>>>> -- >>>>>>>> Matt Sicker <boa...@gmail.com> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Matt Sicker <boa...@gmail.com> >>>>>> >>>>> >>>>> >>>>> >>>>> >>>> >>>> -- >>>> Matt Sicker <boa...@gmail.com> >>>> >>>> >>> >> > -- Matt Sicker <boa...@gmail.com>
