Re: Parallel classloading, need review...

2013-08-10 Thread Kristian Rosenvold
Just did the quick POC (without red wine!!) at https://github.com/krosenvold/plexus-classworlds/commit/f1059c904788eb96e22a9a6a1ead6b3febe57948 By recording the classloading and re-running in reverse order in a (single) separate thread on subsequent reloads, it would appear I almost cut the class

Re: Parallel classloading, need review...

2013-08-09 Thread Andreas Gudian
2013/8/9 Kristian Rosenvold > 2013/8/9 Andreas Gudian : > > But now you need a JDK 7 to build that project, right? I didn't see that > in > > the pom. Is it defined somewhere else? > > That's because you're not looking at the latest commit I did :) > I'm not going to compare timestamps now... ;-

Re: Parallel classloading, need review...

2013-08-09 Thread Kristian Rosenvold
2013/8/9 Andreas Gudian : > But now you need a JDK 7 to build that project, right? I didn't see that in > the pom. Is it defined somewhere else? That's because you're not looking at the latest commit I did :) Kristian - To unsub

Re: Parallel classloading, need review...

2013-08-09 Thread Kristian Rosenvold
I think we have to be quite smart to get any gains. Once a plugin class starts loading in the classloader, the first class referenced "A", will most likely load a whole tree. Sometimes a huge tree. If we recorded the class loading order when building the plugin descriptor, we could preemptively lo

Re: Parallel classloading, need review...

2013-08-09 Thread Andreas Gudian
2013/8/8 Kristian Rosenvold > I just committed an update to plexus-classworlds that permits > concurrent classloading under jdk7. > > I would really appreciate it if anyone would care to review this > patch, especially regarding thread safety (esp the use of guards > to access the protected metho

Re: Parallel classloading, need review...

2013-08-09 Thread Romain Manni-Bucau
Yeah the main issues were 1) you need to // the whole process 2) your hard drive needs to be parallized (didnt find a free solution to this one) 3) you need to load independent classes So generally gain is not as impressive as parallelization sounds Le 9 août 2013 19:52, "Kristian Rosenvold" a é

Re: Parallel classloading, need review...

2013-08-09 Thread Kristian Rosenvold
2013/8/9 Romain Manni-Bucau : > When i tested on tomee gain was ridiculous too so maybe not the first place > to hack on to make maven fast ;) > Le 9 août 2013 18:36, "Jason van Zyl" a écrit : >> And what's the net difference then before after trying to parallelize the >> classloading? I'll read

Re: Parallel classloading, need review...

2013-08-09 Thread Kristian Rosenvold
surefire and maven core itself. I run with -DskipTests since I dont want to measure "that" right now :) Kristian 2013/8/9 Stuart McCulloch : > On 9 Aug 2013, at 17:21, Kristian Rosenvold wrote: > >> I just did some manual instrumentation to classworlds since I'm not >> trusting the profiler due

Re: Parallel classloading, need review...

2013-08-09 Thread Stuart McCulloch
On 9 Aug 2013, at 17:21, Kristian Rosenvold wrote: > I just did some manual instrumentation to classworlds since I'm not > trusting the profiler due to native code. For my 2 test projects, > classloading is pretty close to 10% of the total build time (with > -DskipTests). Btw, which test projects

Re: Parallel classloading, need review...

2013-08-09 Thread Romain Manni-Bucau
When i tested on tomee gain was ridiculous too so maybe not the first place to hack on to make maven fast ;) Le 9 août 2013 18:36, "Jason van Zyl" a écrit : > And what's the net difference then before after trying to parallelize the > classloading? I'll read up on the Java7 classloading this week

Re: Parallel classloading, need review...

2013-08-09 Thread Jason van Zyl
And what's the net difference then before after trying to parallelize the classloading? I'll read up on the Java7 classloading this weekend. I'm in transport land at the moment. On Aug 9, 2013, at 12:21 PM, Kristian Rosenvold wrote: > I just did some manual instrumentation to classworlds sinc

Re: Parallel classloading, need review...

2013-08-09 Thread Kristian Rosenvold
I just did some manual instrumentation to classworlds since I'm not trusting the profiler due to native code. For my 2 test projects, classloading is pretty close to 10% of the total build time (with -DskipTests). As I suspected yesterday, this confirms that at least YJP reports classloading time

Re: Parallel classloading, need review...

2013-08-08 Thread Kristian Rosenvold
The 2-4% measure is taken with jprofiler; there's a fair amount of native code within classloading - does anyone know how that affects profiler measurement ? (I would suspect that since the profiler is incapable of instrumenting native code it will report execution times without "profiler skew"

Re: Parallel classloading, need review...

2013-08-08 Thread Kristian Rosenvold
While I was running the other day I had this crazy idea of making classworlds record the class loading sequence for each class realm and save the results somewhere. Next time the same realm is created we could have one or two threads that preemptively load all the same classes. This is what I get

Re: Parallel classloading, need review...

2013-08-08 Thread Kristian Rosenvold
Not much :) In the test builds I'm measuring (entire builds, not just core), class loading is 2-4% of the build CPU time (with -DskipTests). I have not (yet) done any empirical measurements to determine how much of that we could actually save. The current synchronzation in core means this change in

Re: Parallel classloading, need review...

2013-08-08 Thread Jason van Zyl
What the performance difference in loading classes in your stand-alone tests? On Aug 8, 2013, at 2:12 PM, Kristian Rosenvold wrote: > I just committed an update to plexus-classworlds that permits > concurrent classloading under jdk7. > > I would really appreciate it if anyone would care to rev

Parallel classloading, need review...

2013-08-08 Thread Kristian Rosenvold
I just committed an update to plexus-classworlds that permits concurrent classloading under jdk7. I would really appreciate it if anyone would care to review this patch, especially regarding thread safety (esp the use of guards to access the protected methods of the base classes, which is where th