Re: Lilith and Log4j 2

2017-08-06 Thread Mikael Ståldal
That test was added as part of LOG4J2-1961 by Gary. On 2017-08-03 16:00, Ralph Goers wrote: The error with the missing artifact happens when you skip tests. The test that failed for you never fails for me on OS X but fails randomly in Jenkins. I’m not sure why no one has looked at it yet. R

Re: Lilith and Log4j 2

2017-08-03 Thread Ralph Goers
The error with the missing artifact happens when you skip tests. The test that failed for you never fails for me on OS X but fails randomly in Jenkins. I’m not sure why no one has looked at it yet. Ralph > On Aug 3, 2017, at 6:52 AM, Jörn Huxhorn wrote: > > OK, everything works for me with 2

Re: Lilith and Log4j 2

2017-08-03 Thread Jörn Huxhorn
OK, everything works for me with 2.9-SNAPSHOT. I can receive JSON, YAML and XML events just fine. I had the following error while building: [ERROR] Failed to execute goal on project log4j-core-its: Could not resolve dependencies for project org.apache.logging.log4j:log4j-core-its:jar:2.9-SNAPSHO

Re: Lilith and Log4j 2

2017-08-03 Thread Mikael Ståldal
Yes, that test is known to be unstablle. Just build with -DskipTests On 2017-08-03 11:24, Jörn Huxhorn wrote: I get test failures building against HEAD (3ac727026f295941fe71a34d6c80afc242f38c0f). Environment: $ mvn -version Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-

Re: Lilith and Log4j 2

2017-08-03 Thread Jörn Huxhorn
I get test failures building against HEAD (3ac727026f295941fe71a34d6c80afc242f38c0f). Environment: $ mvn -version Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-03T21:39:06+02:00) Maven home: /Users/huxi/.sdkman/candidates/maven/current Java version: 1.8.0_144, vendor: Ora

Re: Lilith and Log4j 2

2017-08-02 Thread Mikael Ståldal
In 2.9-SNAPSHOT, you can use includeNullDelimiter="true" on the layout. On 2017-08-02 09:23, Jörn Huxhorn wrote: I started working on the Lilith side of this and I think I’m pretty much done. To test the new functionality in a sandbox application, I’d now need the ability to switch JsonLayout

Re: Lilith and Log4j 2

2017-08-02 Thread Jörn Huxhorn
I started working on the Lilith side of this and I think I’m pretty much done. To test the new functionality in a sandbox application, I’d now need the ability to switch JsonLayout, YamlLayout and XmlLayout into a mode were events are terminated with a 0-byte instead of being separated by “, ".

Re: Lilith and Log4j 2

2017-07-26 Thread Mikael Ståldal
That would make sense. On 2017-07-26 19:51, Jörn Huxhorn wrote: Isn’t this supposed to also use 0-byte terminated events in the future? Because in that case it would make sense to change from String to byte[] processing. On 26. July 2017 at 17:49:07, Mikael Ståldal (mi...@apache.org) wrote: T

Re: Lilith and Log4j 2

2017-07-26 Thread Jörn Huxhorn
Isn’t this supposed to also use 0-byte terminated events in the future? Because in that case it would make sense to change from String to byte[] processing. On 26. July 2017 at 17:49:07, Mikael Ståldal (mi...@apache.org) wrote: > They just need the String version: > > https://github.com/apache/

Re: Lilith and Log4j 2

2017-07-26 Thread Mikael Ståldal
They just need the String version: https://github.com/apache/logging-log4j-tools/blob/master/log4j-server/src/main/java/org/apache/logging/log4j/server/InputStreamLogEventBridge.java#L100 On 2017-07-26 17:40, Gary Gregory wrote: What do our XML and JSON server need? Let's make sure we handle t

Re: Lilith and Log4j 2

2017-07-26 Thread Gary Gregory
What do our XML and JSON server need? Let's make sure we handle those use cases. Gary On Jul 26, 2017 05:30, "Jörn Huxhorn" wrote: > +1 > > > On 26. July 2017 at 13:29:27, Mikael Ståldal (mi...@apache.org) wrote: > > Maybe I should remove both the Reader and the InputStream versions, > > since

Re: Lilith and Log4j 2

2017-07-26 Thread Jörn Huxhorn
+1 On 26. July 2017 at 13:29:27, Mikael Ståldal (mi...@apache.org) wrote: > Maybe I should remove both the Reader and the InputStream versions, > since they cannot parse multiple log events from a stream, and are thus > not very useful. > > On 2017-07-26 10:34, Jörn Huxhorn wrote: > > It would

Re: Lilith and Log4j 2

2017-07-26 Thread Jörn Huxhorn
That’s what I meant with too lenient. It was probably a bad idea. Might make more sense to think about which Mixins, if any, should get a @JsonIgnoreProperties(ignoreUnknown=true) annotation… but that is unrelated to the general parsing API. I kind of derailed the topic because this really does

Re: Lilith and Log4j 2

2017-07-26 Thread Mikael Ståldal
A slightly related question, what should the parser do about missing properties? Currently it is so lenient that it will successfully parse an empty JSON object "{}" and just assign null (or other default values) to all fields. On 2017-07-25 23:24, Jörn Huxhorn wrote: Hey Mikael, sorry for

Re: Lilith and Log4j 2

2017-07-26 Thread Mikael Ståldal
Maybe I should remove both the Reader and the InputStream versions, since they cannot parse multiple log events from a stream, and are thus not very useful. On 2017-07-26 10:34, Jörn Huxhorn wrote: It would maybe be a good idea to get rid of the Reader method altogether since it mainly introd

Re: Lilith and Log4j 2

2017-07-26 Thread Jörn Huxhorn
Whoops... yes, of course. false. m) I'd use the byte[] method in Lilith. It would maybe be a good idea to get rid of the Reader method altogether since it mainly introduces an unnecessary point of failure if the Reader is using an encoding other than UTF-8. Using a FileReader would be an examp

Re: Lilith and Log4j 2

2017-07-26 Thread Mikael Ståldal
I guess you mean: this.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false); Out of curiosity, which of the methods would you use? Reading InputStream, Reader, String or byte[] ? On 2017-07-25 23:24, Jörn Huxhorn wrote: Hey Mikael, sorry for the slow response time. GMail tho

Re: Lilith and Log4j 2

2017-07-25 Thread Jörn Huxhorn
Hey Mikael, sorry for the slow response time. GMail thought that - of all the dev mailing list mails - this specific mail looked like spam. I just found it hidden in the spam folder... The API looks good to me, assuming that I’ll be fine with the default c’tor versions of Log4jJsonObjectMapper,

Re: Lilith and Log4j 2

2017-07-25 Thread Mikael Ståldal
Jörn Huxhorn, would the API in Git branch LOG4J2-1986 work for you? On 2017-07-22 22:49, Mikael Ståldal wrote: Work in progress in Git branch LOG4J2-1986. On 2017-07-22 22:31, Mikael Ståldal wrote: Yes, I am putting it in its own Java package. On 2017-07-22 22:13, Gary Gregory wrote: Should

Re: Lilith and Log4j 2

2017-07-22 Thread Mikael Ståldal
Work in progress in Git branch LOG4J2-1986. On 2017-07-22 22:31, Mikael Ståldal wrote: Yes, I am putting it in its own Java package. On 2017-07-22 22:13, Gary Gregory wrote: Should this API sit in its own package? On Jul 22, 2017 11:41, "Mikael Ståldal" wrote: I'll give it a shot: https:

Re: Lilith and Log4j 2

2017-07-22 Thread Mikael Ståldal
Yes, I am putting it in its own Java package. On 2017-07-22 22:13, Gary Gregory wrote: Should this API sit in its own package? On Jul 22, 2017 11:41, "Mikael Ståldal" wrote: I'll give it a shot: https://issues.apache.org/jira/browse/LOG4J2-1986 On 2017-07-22 20:28, Ralph Goers wrote: Y

Re: Lilith and Log4j 2

2017-07-22 Thread Gary Gregory
Should this API sit in its own package? On Jul 22, 2017 11:41, "Mikael Ståldal" wrote: > I'll give it a shot: > > https://issues.apache.org/jira/browse/LOG4J2-1986 > > > On 2017-07-22 20:28, Ralph Goers wrote: > >> Yes, that is a good idea. Probably XML and YAML as well. >> >> Ralph >> >> On Jul

Re: Lilith and Log4j 2

2017-07-22 Thread Mikael Ståldal
I'll give it a shot: https://issues.apache.org/jira/browse/LOG4J2-1986 On 2017-07-22 20:28, Ralph Goers wrote: Yes, that is a good idea. Probably XML and YAML as well. Ralph On Jul 22, 2017, at 11:21 AM, Mikael Ståldal wrote: But here we have a concrete use case: a third-party tool, Lilit

Re: Lilith and Log4j 2

2017-07-22 Thread Ralph Goers
Yes, that is a good idea. Probably XML and YAML as well. Ralph > On Jul 22, 2017, at 11:21 AM, Mikael Ståldal wrote: > > But here we have a concrete use case: a third-party tool, Lilith, needs to > parse log events from our JsonLayout. (Our SocketServer in log4j-server have > the same need, a

Re: Lilith and Log4j 2

2017-07-22 Thread Mikael Ståldal
But here we have a concrete use case: a third-party tool, Lilith, needs to parse log events from our JsonLayout. (Our SocketServer in log4j-server have the same need, and it uses this class.) Should we provide a public API in log4j-core for doing so, which Lilith, our SocketServer, and possibl

Re: Lilith and Log4j 2

2017-07-22 Thread Gary Gregory
Hi All, Just like anything in log4j-core, we try not break binary compatibility when we can but we do not bend over backwards like we do for log4j-api. This class is public probably because Jackson requires it to be, otherwise I would have made it package private. The fact that we use Jackson for

Re: Lilith and Log4j 2

2017-07-22 Thread Mikael Ståldal
Gary, what do you say? I think you once added that comment. On 2017-07-22 16:27, Jörn Huxhorn wrote: Well… it says /** * A Jackson JSON {@link ObjectMapper} initialized for Log4j. * * Consider this class private. * */ so I wanted to holler about how to continue. I’ll give it a sh

Re: Lilith and Log4j 2

2017-07-22 Thread Jörn Huxhorn
Well… it says /**  * A Jackson JSON {@link ObjectMapper} initialized for Log4j.  *  * Consider this class private.  *  */ so I wanted to holler about how to continue. I’ll give it a shot. On 22. July 2017 at 15:06:28, Mikael Ståldal (mi...@apache.org) wrote: > Would it work to use > org.apac

Re: Lilith and Log4j 2

2017-07-22 Thread Mikael Ståldal
Would it work to use org.apache.logging.log4j.core.jackson.Log4jJsonObjectMapper, which is public? See here how it is used: https://github.com/apache/logging-log4j-tools/blob/master/log4j-server/src/main/java/org/apache/logging/log4j/server/JsonInputStreamLogEventBridge.java On 2017-07-22 13

Re: Lilith and Log4j 2

2017-07-22 Thread Jörn Huxhorn
It seems like I don’t get access to the log4j2 ObjectMapper or Jackson Module used for JSON serialization, i.e. I can’t do something like this: new JacksonFactory.JSON(encodeThreadContextAsList, includeStacktrace, false) It would make sense to reuse your ObjectMapper/Module in my code instead of

Re: Lilith and Log4j 2

2017-07-19 Thread Jörn Huxhorn
I’ll give this a shot over the next view days. Thanks! On 19. July 2017 at 21:33:49, Mikael Ståldal (mi...@apache.org) wrote: > JsonLayout (and XmlLayout and YamlLayout) now supports 0-byte > termination of log events. Will be part of the upcoming 2.9 release. > > > On 2017-07-18 11:48, Jörn

Re: Lilith and Log4j 2

2017-07-19 Thread Mikael Ståldal
JsonLayout (and XmlLayout and YamlLayout) now supports 0-byte termination of log events. Will be part of the upcoming 2.9 release. On 2017-07-18 11:48, Jörn Huxhorn wrote: A general event header and footer would probably be overkill. I’m using the following interface in Lilith: public interf

Re: Lilith and Log4j 2

2017-07-18 Thread Jörn Huxhorn
A general event header and footer would probably be overkill. I’m using the following interface in Lilith: public interface WriteByteStrategy {     void writeBytes(DataOutputStream dataOutputStream, byte[] bytes)         throws IOException; } One implementation writes an int with the length of b

Re: Lilith and Log4j 2

2017-07-17 Thread Ralph Goers
Right now that is all handled by the specific layouts. For example, by default the RFC5424Layout doesn’t append newlines so when writing to a file they will all be on the same “line”, but it has an option to append one if needed. Doing the same would be another option for the JSONLayout. I’d be

Re: Lilith and Log4j 2

2017-07-17 Thread Gary Gregory
Do we want a general event header and footer then? Gary On Jul 17, 2017 14:43, "Ralph Goers" wrote: > No. A Footer is only used at end of file. He needs to know how long each > event is or when it is the start of a new event. > > Ralph > > > On Jul 17, 2017, at 12:32 PM, Gary Gregory > wrote:

Re: Lilith and Log4j 2

2017-07-17 Thread Ralph Goers
From comments from Scott Deboy I believe Chainsaw also supports Log4j 2 but not in an official release. Ralph > On Jul 17, 2017, at 12:13 PM, Mikael Ståldal wrote: > > Hi. > > (Moving this discussion to logging dev mailing list.) > > Have you tried to use: > > > Then each log event will be

Re: Lilith and Log4j 2

2017-07-17 Thread Ralph Goers
No. A Footer is only used at end of file. He needs to know how long each event is or when it is the start of a new event. Ralph > On Jul 17, 2017, at 12:32 PM, Gary Gregory wrote: > > Can't you use a footer for any terminator you wish? > > Gary > > On Mon, Jul 17, 2017 at 12:13 PM, Mikael St

Re: Lilith and Log4j 2

2017-07-17 Thread Gary Gregory
Can't you use a footer for any terminator you wish? Gary On Mon, Jul 17, 2017 at 12:13 PM, Mikael Ståldal wrote: > Hi. > > (Moving this discussion to logging dev mailing list.) > > Have you tried to use: > > > Then each log event will be terminated by end-of-line (\r\n). > > I think it would b

Re: Lilith and Log4j 2

2017-07-17 Thread Mikael Ståldal
Hi. (Moving this discussion to logging dev mailing list.) Have you tried to use: Then each log event will be terminated by end-of-line (\r\n). I think it would be easy to implement 0-byte terminated log events in JsonLayout, and that would make sense since we have implemented support for th