Re: Extend Native Memory Tracking over the JDK ? (was: Proposal: track zlib native memory usage with NMT)

2022-12-05 Thread Kennke, Roman
Hi Thomas, I very much like the idea and also your proposals how to do it. Insights in JDK's native memory usage is sorely lacking and would be very useful! I don't have all that much to add about the details beyond what you already covered, though :-) Cheers, Roman Are there any opinions

Re: Extend Native Memory Tracking over the JDK ? (was: Proposal: track zlib native memory usage with NMT)

2022-12-05 Thread Thomas Stüfe
Thank you for the positive encouragement, Roman :-) Cheers, Thomas On Mon, Dec 5, 2022 at 12:03 PM Kennke, Roman wrote: > Hi Thomas, > > I very much like the idea and also your proposals how to do it. Insights > in JDK's native memory usage is sorely lacking and would be very useful! > I don't

Re: Extend Native Memory Tracking over the JDK ? (was: Proposal: track zlib native memory usage with NMT)

2022-12-01 Thread Thomas Stüfe
Are there any opinions about whether or not to extend NMT across the JDK? This blocks https://bugs.openjdk.org/browse/JDK-8296360, and I had a PR prepared as https://github.com/openjdk/jdk/pull/10988. Originally I was hoping to get this into JDK 20, but I don't think that is realistic anymore. I a

Re: [External] : Re: Extend Native Memory Tracking over the JDK ? (was: Proposal: track zlib native memory usage with NMT)

2022-12-01 Thread Stefan Johansson
Hi Thomas and Carter, I opened up a PR for this to allow more specific comments on the implementation: https://github.com/openjdk/jdk/pull/11449 If this discussion leads to us not wanting to proceed with the change I will withdraw the PR. Some more comments below. On 2022-12-01 08:26, Thom

Re: Extend Native Memory Tracking over the JDK ? (was: Proposal: track zlib native memory usage with NMT)

2022-11-30 Thread Thomas Stüfe
Hi Carter, Stefan, thank you, I think it is good to have this discussion, it is important. Side note, the discussion steered away from my original question - whether to instrument the JDK with NMT. I still would love to discuss that, too. About opening NMT up for user consumption, that is of cou

Re: Extend Native Memory Tracking over the JDK ? (was: Proposal: track zlib native memory usage with NMT)

2022-11-30 Thread Carter Kozak
This looks fantastic, thank you so much! I can confirm that the proposed design would solve my use-case. I'd enjoy discussing the NMT event contract somewhere more specific to the implementation, but I don't want to muddle this thread with implementation details. Carter Kozak On Wed, Nov 30,

Re: Extend Native Memory Tracking over the JDK ? (was: Proposal: track zlib native memory usage with NMT)

2022-11-30 Thread Stefan Johansson
Hi Carter, Your mail made me pick up an old item from my wishlist: to have native memory tracking information available in JFR recordings. When we, in GC, do improvements to decrease the native memory overhead of our algorithms, NMT is a very good tool to track the progress. We have scripts t

Extend Native Memory Tracking over the JDK ? (was: Proposal: track zlib native memory usage with NMT)

2022-11-10 Thread Carter Kozak
*+serviceability-dev* Firstly, thank you both for your time and work in this space. Apologies if this should be a separate thread, but the new title “Extend Native Memory Tracking over the JDK” aligns directly with some work I’ve been investigating, and I hope my feedback will be helpful for pr

Extend Native Memory Tracking over the JDK ? (was: Proposal: track zlib native memory usage with NMT)

2022-11-08 Thread Thomas Stüfe
Hi Alan, (replaced hotspot-runtime-dev with hotspot-dev, since its more of a general topic) thank you for your time! I am very happy to talk this through. I think native memory observability in the JDK (and customer code!) is sorely lacking. Witness the countless "where did my native memory go"

Re: Proposal: track zlib native memory usage with NMT

2022-11-06 Thread Alan Bateman
On 04/11/2022 16:54, Thomas Stüfe wrote: Hi all, I am currently working on https://bugs.openjdk.org/browse/JDK-8296360; I was preparing the final PR [1], but then Alan did ask me to discuss this on core-libs first. Backstory: NMT tracks hotspot native allocations but does not cover the JDK

Proposal: track zlib native memory usage with NMT

2022-11-04 Thread Thomas Stüfe
Hi all, I am currently working on https://bugs.openjdk.org/browse/JDK-8296360; I was preparing the final PR [1], but then Alan did ask me to discuss this on core-libs first. Backstory: NMT tracks hotspot native allocations but does not cover the JDK libraries (small exception: Unsafe.AllocateMem