Re: Running under a SecurityManager
I we don’t have permission to use ServiceLoader Log4j will simply fail to initialize with anything other than SimpleLoggerContextFactory (i.e. - nothing in log4j-core will work). It looks to me that the issue in 2.18.0 is that ServiceLoaderUtil is accessing ServiceLoader via a MethodHandle and that must be requiring the SecurityManager. We are using MethodHandlers so that it will work properly in a JPMS environment. One solution that I think would work for this would be to check if JPMS is active and if not directly call ServiceLoader. Ralph > On Aug 21, 2022, at 9:31 AM, . . wrote: > > Hello all, > > thanks Piotr to take care for the topic. One thing to consider: > >> The environment and system properties sources are protected by internal Java >> security checks,... > > Unfortunately not after applying the fix: PropertiesUtil[1] loads all the > services which provide a PropertySource inside the doPrivileged including the > default log4j2 implementations[2] which include the system properties [3]. > Both fix approaches are not good at the moment. In practice nearly all > frameworks require Property* permissions, due to caching / loading all etc. > But that is not a good reason to introduce a leak. Maybe a alternative with > more refactoring: Only the really needed properties are loaded, without a > util method, without a service in between. The SecurityExceptions are thrown > and not silently ignored. Any service implementation has to care itself. > > I don't know enough about service loading: Would any service lookup inside a > doPrivileged block cause a constructor to be called inside the same block? :-( > > One thing in general: Could someone explain the usecase behind catching > SecurityExceptions and silently dropping them? [4][5][6][7][8] Please explain > it for the case that an authorized administrator with the knowledge and right > to grant permissions wants to set the permissions correct. Please explain it > for an monitoring system (ELK or something) which is configured to alert for > SecurityExceptions. > > The hope of a near end of the SecurityManager must be delayed till December > 2030[9] as long as you want to support Java 8 with log4j2 ;-) Due to very > positive experience in application testing and production I'm not happy about > the deprecation, but that is offtopic. > > Regards > Boris > > [1] > https://github.com/apache/logging-log4j2/blob/b734a4f66af868f03dafafe5de92999058096eca/log4j-api/src/main/java/org/apache/logging/log4j/util/PropertiesUtil.java#L477 > [2] > https://github.com/apache/logging-log4j2/blob/release-2.x/log4j-api/src/main/resources/META-INF/services/org.apache.logging.log4j.util.PropertySource > [3] > https://github.com/apache/logging-log4j2/blob/release-2.x/log4j-api/src/main/java/org/apache/logging/log4j/util/SystemPropertiesPropertySource.java#L44 > [4] > https://github.com/apache/logging-log4j2/blob/release-2.x/log4j-api/src/main/java/org/apache/logging/log4j/util/SystemPropertiesPropertySource.java#L45 > [5] > https://github.com/apache/logging-log4j2/blob/release-2.x/log4j-api/src/main/java/org/apache/logging/log4j/util/SystemPropertiesPropertySource.java#L76 > [6] > https://github.com/apache/logging-log4j2/blob/release-2.x/log4j-api/src/main/java/org/apache/logging/log4j/util/SystemPropertiesPropertySource.java#L85 > [7] > https://github.com/apache/logging-log4j2/blob/release-2.x/log4j-api/src/main/java/org/apache/logging/log4j/util/PropertiesUtil.java#L405 > [8] > https://github.com/apache/logging-log4j2/blob/release-2.x/log4j-api/src/main/java/org/apache/logging/log4j/util/PropertiesUtil.java#L456 > [9] https://www.oracle.com/java/technologies/java-se-support-roadmap.html
JDK 19 first Release Candidates!
Greetings! I hope you had a chance to take some time off. On our side, and despite the summer vacation, everything is on track for the Java 19 GA release on September 20th with JDK 19 now in the Release Candidate Phase [1]. If you haven't done so yet, it is time to start testing your project(s) using the JDK 20 Early-Access builds. Speaking of Early-Access builds, there is now a new set of EA builds, i.e., the jextract EA builds. jextract is a tool developed under the Project Panama umbrella whose goal is to mechanically generates Java bindings from native library headers. If you are using the Foreign Function & Memory API (Preview Feature in JDK 19), make sure to check jextract too (see the jextract section below). [1] https://mail.openjdk.org/pipermail/jdk-dev/2022-August/006861.html ## Heads-up - New system properties for `System.out` and `System.err` in JDK 19 Two new system properties, `stdout.encoding` and `stderr.encoding`, have been introduced. The value of these system properties is the encoding used by the standard output (`System.out`) and standard error (`System.err`) streams. The default values of these system properties depend on the platform. The values take on the value of the `native.encoding` property when the platform does not provide streams for the console. The properties can be overridden on the launcher's command line option, with `-D`, to set them to UTF-8 where required. For more details see https://bugs.openjdk.org/browse/JDK-8283620 ## Heads-up - SSLSocketImpl finalizer implementation removed in JDK 19 The finalizer implementation in SSLSocket has been removed, with the underlying native resource releases now done by the Socket implementation. With this update, the TLS close_notify messages will no longer be emitted if SSLSocket is not explicitly closed. Not closing Sockets properly is an error condition that should be avoided. Applications should always close sockets and not rely on garbage collection. For more details see https://bugs.openjdk.org/browse/JDK-8212136 ## Heads-up - New providerPath jarsigner option in JDK 19 A new `-providerPath` option has been added to the jarsigner. This option is used to specify the class path of an alternate keystore implementation, it can be used together with the -providerClass option. For more details see https://bugs.openjdk.org/browse/JDK-8281175 ## JDK 19 Release Candidate builds JDK 19 first Release Candidates (builds 36) are now available [2], and are provided under the GNU General Public License v2, with the Classpath Exception. The Release Notes are available here [3]. [2] https://jdk.java.net/19/ [3] https://jdk.java.net/19/release-notes ## JDK 20 Early-Access builds JDK 20 Early-Access builds 11 are now available [4], and are provided under the GNU General Public License v2, with the Classpath Exception. The Release Notes are available here [5]. [4] https://jdk.java.net/20/ [5] https://jdk.java.net/20/release-notes ### Recent changes that maybe of interest: - JDK-8282730: LdapLoginModule throw NPE from logout method after login failure - JDK-8290706: Remove the support for inline contiguous allocations - JDK-8289551: Conversions between bit representations of half precision values and floats - JDK-8290485: [vector] REVERSE_BYTES for byte type should not emit any instructions - JDK-8289137: Automatically adapt Young/OldPLABSize and when setting only MinTLABSize - JDK-8290034: Auto vectorize reverse bit operations. - JDK-8290868: NMT: MallocSiteTable statistics improvements - JDK-8291822: ARM32: Build errors with GCC 11 in frame::saved_oop_result [Reported by JaCoCo] - JDK-8289249: Add methods to Elements for record constructors - JDK-8283232: x86: Improve vector broadcast operations - JDK-8288327: Executable.hasRealParameterData should not be volatile - JDK-8291360: Create entry points to expose low-level class file information - JDK-8290840: Refactor the "os" class - JDK-8292327: InflaterInputStream.read throws EOFException - JDK-8155246: Throw error if default java.security file is missing - JDK-8289332: Auto-generate ids for user-defined headings - JDK-8292153: x86: Represent Registers as values ## Jextract Early-Access Builds Early Access Builds 19-jextract+2-3 (2022/7/19) are now available [6]. These open-source builds are provided under the GNU General Public License, version 2, with the Classpath Exception. These builds are from the OpenJDK jextract project [7] which is part of Code Tools [8]. jextract is a tool developed under the Panama umbrealla whose goal is to mechanically generate Java bindings from native library headers. These EA builds are intended for advanced users, and are provided as a convenience so that they don't need to build it from the sources. Additional notes on builds, documentation and known issues are available at [6]. Please subscribe to the jextract mailing list [9] to share feedback. [6] https://jdk.java.net/jext
Re: Stack valued MDC
Hi all, On Sat, 16 Apr 2022 at 00:17, Carter Kozak wrote: > I can understand how the stack-based mdc might be convenient and useful, but > I don't think it fits my use-cases. That said, I wonder if the API could be > improved in such a way that it could leverage the application stack instead > of maintaining its own -- this is an issue that I've encountered in tracing > implementations as well, where asymmetric interactions to cause the > application stack and internal stack to get out of sync. Perhaps using > something like putCloseable[1] would allow the data needed to reset to be > stored in the closeable without maintaining a standalone stack (at the cost > of the ability to support getCopyOfDequeByKey[2]). Since Ceki announced the release of SLF4J 2.0.0, the topic is back. We need to decide whether: 1. we extend the Log4j2 API to support the "enhanced" NDC version of SLF4J2, 2. we use the default implementation provided by Ceki, 3. we hack around it (e.g. encoding a list of values to a JSON-like structure) like in https://github.com/apache/logging-log4j2/pull/820. I would use Ceki's implementation and provide a `PatternConverter` for those hordes of users that will use it. Alternatively we could inject the top values from Ceki's NDC into the usual ThreadContextMap. What do you think?
Re: Stack valued MDC
It would be weird IMO to provide a PatternConverter for an NDC stack that is supported by SLF4J but not Log4j. Unfortunately, I think the only sane thing to do is option 1. Ralph > On Aug 22, 2022, at 8:45 AM, Piotr P. Karwasz wrote: > > Hi all, > > On Sat, 16 Apr 2022 at 00:17, Carter Kozak wrote: >> I can understand how the stack-based mdc might be convenient and useful, but >> I don't think it fits my use-cases. That said, I wonder if the API could be >> improved in such a way that it could leverage the application stack instead >> of maintaining its own -- this is an issue that I've encountered in tracing >> implementations as well, where asymmetric interactions to cause the >> application stack and internal stack to get out of sync. Perhaps using >> something like putCloseable[1] would allow the data needed to reset to be >> stored in the closeable without maintaining a standalone stack (at the cost >> of the ability to support getCopyOfDequeByKey[2]). > > Since Ceki announced the release of SLF4J 2.0.0, the topic is back. We > need to decide whether: > > 1. we extend the Log4j2 API to support the "enhanced" NDC version of SLF4J2, > 2. we use the default implementation provided by Ceki, > 3. we hack around it (e.g. encoding a list of values to a JSON-like > structure) like in https://github.com/apache/logging-log4j2/pull/820. > > I would use Ceki's implementation and provide a `PatternConverter` for > those hordes of users that will use it. Alternatively we could inject > the top values from Ceki's NDC into the usual ThreadContextMap. > > What do you think?
Re: Running under a SecurityManager
Hi Ralph, On Mon, 22 Aug 2022 at 09:50, Ralph Goers wrote: > It looks to me that the issue in 2.18.0 is that ServiceLoaderUtil is > accessing ServiceLoader via a MethodHandle and that must be requiring the > SecurityManager. We are using MethodHandlers so that it will work properly in > a JPMS environment. One solution that I think would work for this would be to > check if JPMS is active and if not directly call ServiceLoader. I did some more digging and `ServiceLoaderUtil` is failing, because the call stack contains classes that do not have access to `META-INF/services` resources (`log4j-api` of course has access to it). I added some reflection magic to PR #1008 to perform an `AccessController.doPriviledged(() -> ServiceLoader.load(serviceType. classLoader)` using the caller's domain. This should prevent other code from loading services using Log4j2 permissions, but at the same time requires a lot of reflection permissions from the Log4j2 API itself. I believe the Log4j2 API should at least have an "accessInPackage" permission for each caller's package. Piotr [1] https://github.com/apache/logging-log4j2/pull/1008
Re: Stack valued MDC
I don’t think that’s the only sane option. Another sane option is to ignore it. There’s nothing that says Log4j has to implement any enhanced SLF4J APIs. He wants to be the lowest common denominator in logging APIs, then he can’t add APIs that aren’t supported in the major implementations. Just throw an UnsupportedOperationException. > On Aug 23, 2022, at 2:04, Ralph Goers wrote: > > It would be weird IMO to provide a PatternConverter for an NDC stack that is > supported by SLF4J but not Log4j. > > Unfortunately, I think the only sane thing to do is option 1. > > Ralph > >> On Aug 22, 2022, at 8:45 AM, Piotr P. Karwasz >> wrote: >> >> Hi all, >> >>> On Sat, 16 Apr 2022 at 00:17, Carter Kozak wrote: >>> I can understand how the stack-based mdc might be convenient and useful, >>> but I don't think it fits my use-cases. That said, I wonder if the API >>> could be improved in such a way that it could leverage the application >>> stack instead of maintaining its own -- this is an issue that I've >>> encountered in tracing implementations as well, where asymmetric >>> interactions to cause the application stack and internal stack to get out >>> of sync. Perhaps using something like putCloseable[1] would allow the data >>> needed to reset to be stored in the closeable without maintaining a >>> standalone stack (at the cost of the ability to support >>> getCopyOfDequeByKey[2]). >> >> Since Ceki announced the release of SLF4J 2.0.0, the topic is back. We >> need to decide whether: >> >> 1. we extend the Log4j2 API to support the "enhanced" NDC version of SLF4J2, >> 2. we use the default implementation provided by Ceki, >> 3. we hack around it (e.g. encoding a list of values to a JSON-like >> structure) like in https://github.com/apache/logging-log4j2/pull/820. >> >> I would use Ceki's implementation and provide a `PatternConverter` for >> those hordes of users that will use it. Alternatively we could inject >> the top values from Ceki's NDC into the usual ThreadContextMap. >> >> What do you think? >
Re: Stack valued MDC
Hi everybody, Besides the enhanced NDC, there is another SLF4J 2.0 feature I don't really know how to deal with: https://www.slf4j.org/api/org/slf4j/spi/LoggingEventBuilder.html#addKeyValue-java.lang.String-java.lang.Object- These key/value pairs in the `LoggingEventBuilder` seem to be separate from the MDC, so we can not inject them into the ThreadContextMap. The reference implementation of SLF4J adds them to its `LoggingEvent`, which suggests that we might create a new `Slf4jMessage` and add them there. The problem would be how to format such a message. What do you think? Piotr
Re: Stack valued MDC
I don’t see that as a problem at all. Log4jLogEvent’s constructor accepts a List of properties. We use this to inject properties configured on Loggers into the Context Map of the LogEvent. I see no reason why these key/value pairs shouldn’t be treated the same way. Ralph > On Aug 22, 2022, at 10:51 PM, Piotr P. Karwasz > wrote: > > Hi everybody, > > Besides the enhanced NDC, there is another SLF4J 2.0 feature I don't > really know how to deal with: > > https://www.slf4j.org/api/org/slf4j/spi/LoggingEventBuilder.html#addKeyValue-java.lang.String-java.lang.Object- > > These key/value pairs in the `LoggingEventBuilder` seem to be separate > from the MDC, so we can not inject them into the ThreadContextMap. The > reference implementation of SLF4J adds them to its `LoggingEvent`, > which suggests that we might create a new `Slf4jMessage` and add them > there. The problem would be how to format such a message. > > What do you think? > > Piotr
