Okay, that works, so summary: 1) CEP-12 will be delivered WITHOUT interacting with Chronicle Queues, something else will be used (most probably just a simple logger logging to a text file, that is just enough for something like diagnostic logs which are logged very sparsely)
2) We will NOT use Chronicle in the future for any new implementations / solutions from now on. 3) We WILL be replacing loggers based on Chronicle Queues. 4) A new implementation replacing Chronicle Queues SHOULD take into account a heterogeneous environment when it comes to consumers of the events logged, being language-agnostic or supporting an open format which enables that. On Mon, Sep 30, 2024 at 5:07 PM Josh McKenzie <jmcken...@apache.org> wrote: > I say we do CEP-12 w/out using chronicle and have a follow up JIRA to > replace chronicle w/something else. > > Seems like there's a reasonable consensus around at least that subset of > things given the discussion here. As to what form that something else > takes, well, that's a topic for another day. :) > > On Mon, Sep 30, 2024, at 10:09 AM, Štefan Miklošovič wrote: > > "how complex should it be to rip out the chronicle format, insert some > other well defined and well known, and handle log rolling ourselves". > > I think that it will be actually easier to do after CEP-12 is in because > as I mentioned it does housekeeping of what is there rigth now and > refactors it a little bit so throwing away the guts of it should be > isolated only to actual BinLog class and stuff around that which is built > on top of Chronicle. > > "There a reason we can't move forward with CEP-12 w/out addressing the > chronicle stuff? i.e." > > I think there is not any. > > "Why would CEP-12 be heavily coupled with chronicle?" - it would not be > heavier from what is already there for audit of fql. Chronicle here > basically acts as a sink. > > Actually, that patch also makes the implementations of (diagnostic too) > loggers pluggable (via coding against an interface and putting that on the > class path) so people might already write it to whatever they want - even > to something protobuf-like. If they do not want to use Chronicle as a sink, > by implementing their own logger, they could just put it wherever they > want. I think that I forgot to mention this aspect. So the whole solution > we have is already not hard-wired to Chronicle necessarily. It is just > something we provide out of the box. > > On Mon, Sep 30, 2024 at 3:57 PM Josh McKenzie <jmcken...@apache.org> > wrote: > > > I hear you. Not trying to shoehorn a change in w/CEP-12, just thinking > through "how complex should it be to rip out the chronicle format, insert > some other well defined and well known, and handle log rolling ourselves". > My preference (which I didn't indicate earlier) would be to have that done > independently. > > There a reason we can't move forward with CEP-12 w/out addressing the > chronicle stuff? i.e. > > I would like to have this resolved because there is CEP-12 I plan to > deliver and I hit this and I do not want to base that work on something we > might eventually abandon. > > Why would CEP-12 be heavily coupled with chronicle? I would assume you'd > be able to make light use of the existing logging + log rolling, and then > someone else could come along and easily rip out chronicle and the rolling > and add in a different one with minimal code changes? > > On Mon, Sep 30, 2024, at 9:15 AM, Štefan Miklošovič wrote: > > I don't understand why CEP-12 should be a vehicle for introducing changes > like that. That is something totally unrelated. I am not going to be the > one to implement anything beyond CEP-12 and I am not the one who is going > to replace it either so if we make it a hard requirement for CEP-12 then > CEP-12 as it is will never be introduced. Just want to be clear about that. > > On Mon, Sep 30, 2024 at 3:09 PM Brandon Williams <dri...@gmail.com> wrote: > > On Mon, Sep 30, 2024 at 7:55 AM Josh McKenzie <jmcken...@apache.org> > wrote: > > > > I'd strongly support either rolling the format change into the CEP-12 > proposal or having another CEP for introducing protobuf, spark, etc - some > kind of more broadly adopted format, and removing chronicle from our stack. > > +1, I too would strongly support an open format and removing chronicle. > > Kind Regards, > Brandon > > > >
