Re: [math] threading redux

2015-04-23 Thread Gilles
On Thu, 23 Apr 2015 09:15:47 -0500, Ole Ersoy wrote: It's the case for the only example that was mentioned in this thread with sufficient level of details so as to permit concrete statements: it's the SOFM implementation (in package "o.a.c.m.ml.neuralnet.sofm"). The shared structure is the "Ne

Re: [math] threading redux

2015-04-23 Thread Ole Ersoy
It's the case for the only example that was mentioned in this thread with sufficient level of details so as to permit concrete statements: it's the SOFM implementation (in package "o.a.c.m.ml.neuralnet.sofm"). The shared structure is the "Network" instance. Can the KohonenTrainingTask be decomp

Re: [math] threading redux

2015-04-23 Thread Phil Steitz
On 4/22/15 6:25 PM, Gilles wrote: > On Wed, 22 Apr 2015 11:33:30 -0500, Ole Ersoy wrote: >> On Mon, Apr 20, 2015 at 6:05 PM Phil Steitz >> wrote: There are lots of ways to allow distributed processes to share common data. Spark has a very nice construct called a Resilient Dist

Re: [math] threading redux

2015-04-22 Thread Gilles
On Wed, 22 Apr 2015 11:33:30 -0500, Ole Ersoy wrote: On Mon, Apr 20, 2015 at 6:05 PM Phil Steitz wrote: There are lots of ways to allow distributed processes to share common data. Spark has a very nice construct called a Resilient Distributed Dataset (RDD) designed for exactly this purpose.

Re: [math] threading redux

2015-04-22 Thread Ole Ersoy
On Mon, Apr 20, 2015 at 6:05 PM Phil Steitz wrote: There are lots of ways to allow distributed processes to share common data. Spark has a very nice construct called a Resilient Distributed Dataset (RDD) designed for exactly this purpose. Are there any examples of a class in commons math wher

Re: [math] threading redux

2015-04-22 Thread James Carman
On Mon, Apr 20, 2015 at 6:05 PM Phil Steitz wrote: > > > There are lots of ways to allow distributed processes to share > common data. Spark has a very nice construct called a Resilient > Distributed Dataset (RDD) designed for exactly this purpose. > To take the abstraction layer a step further

Re: [math] threading redux

2015-04-22 Thread Ole Ersoy
That is one way to achieve parallelism. The Executor is one way to manage concurrently executing threads in a single process. There are other ways to do this. My challenge is to find a way to make it possible for users to plug in alternatives. Some of the methods on CompletableFuture allow th

Re: [math] threading redux

2015-04-20 Thread Phil Steitz
On 4/19/15 6:08 AM, Gilles wrote: > Hello. > > On Sat, 18 Apr 2015 22:25:20 -0400, James Carman wrote: >> I think I got sidetracked when typing that email. I was trying >> to say that >> we need an abstraction layer above raw threads in order to allow for >> different types of parallelism. The F

Re: [math] threading redux

2015-04-19 Thread Gilles
Hello. On Sat, 18 Apr 2015 22:25:20 -0400, James Carman wrote: I think I got sidetracked when typing that email. I was trying to say that we need an abstraction layer above raw threads in order to allow for different types of parallelism. The Future abstraction is there in order to support

Re: [math] threading redux

2015-04-18 Thread James Carman
I think I got sidetracked when typing that email. I was trying to say that we need an abstraction layer above raw threads in order to allow for different types of parallelism. The Future abstraction is there in order to support remote execution where side effects aren't good enough. As for a co

Re: [math] threading redux

2015-04-18 Thread Gilles
On Fri, 17 Apr 2015 16:53:56 -0500, James Carman wrote: Do you have any pointers to code for this ForkJoin mechanism? I'm curious to see it. The key thing you will need in order to support parallelization in a generic way What do you mean by "generic way"? I'm afraid that we may be trying to

Re: [math] threading redux

2015-04-18 Thread Phil Steitz
On 4/17/15 5:20 PM, Gary Gregory wrote: > I thought I'd share this read with you guys: > http://coopsoft.com/ar/CalamityArticle.html > > I'm not sure how closely these problems relate with what [math] is trying > to do, but it's a interesting read. Thanks! Kind of supports the idea that somehow a

Re: [math] threading redux

2015-04-17 Thread Ole Ersoy
This is a pretty good read as well: http://stackoverflow.com/questions/21163108/custom-thread-pool-in-java-8-parallel-stream A concern in earlier discussions focused on controlling the number of threads that the job consumes. Theres one example of using a custom thread pool for that. Users co

Re: [math] threading redux

2015-04-17 Thread Gary Gregory
I thought I'd share this read with you guys: http://coopsoft.com/ar/CalamityArticle.html I'm not sure how closely these problems relate with what [math] is trying to do, but it's a interesting read. Gary On Fri, Apr 17, 2015 at 9:01 AM, Gilles wrote: > On Fri, 17 Apr 2015 08:35:42 -0700, Phil

Re: [math] threading redux

2015-04-17 Thread James Carman
Do you have any pointers to code for this ForkJoin mechanism? I'm curious to see it. The key thing you will need in order to support parallelization in a generic way is to not tie it directly to threads, but use some abstraction layer above threads, since that may not be the "worker" method you'r

Re: [math] threading redux

2015-04-17 Thread Thomas Neidhart
On 04/17/2015 05:35 PM, Phil Steitz wrote: > On 4/17/15 3:14 AM, Gilles wrote: >> Hello. >> >> On Thu, 16 Apr 2015 17:06:21 -0500, James Carman wrote: >>> Consider me poked! >>> >>> So, the Java answer to "how do I run things in multiple threads" >>> is to >>> use an Executor (java.util). This doe

Re: [math] threading redux

2015-04-17 Thread Phil Steitz
On 4/17/15 9:01 AM, Gilles wrote: > On Fri, 17 Apr 2015 08:35:42 -0700, Phil Steitz wrote: >> On 4/17/15 3:14 AM, Gilles wrote: >>> Hello. >>> >>> On Thu, 16 Apr 2015 17:06:21 -0500, James Carman wrote: Consider me poked! So, the Java answer to "how do I run things in multiple thread

Re: [math] threading redux

2015-04-17 Thread Gilles
On Fri, 17 Apr 2015 08:35:42 -0700, Phil Steitz wrote: On 4/17/15 3:14 AM, Gilles wrote: Hello. On Thu, 16 Apr 2015 17:06:21 -0500, James Carman wrote: Consider me poked! So, the Java answer to "how do I run things in multiple threads" is to use an Executor (java.util). This doesn't necessar

Re: [math] threading redux

2015-04-17 Thread Phil Steitz
On 4/17/15 3:14 AM, Gilles wrote: > Hello. > > On Thu, 16 Apr 2015 17:06:21 -0500, James Carman wrote: >> Consider me poked! >> >> So, the Java answer to "how do I run things in multiple threads" >> is to >> use an Executor (java.util). This doesn't necessarily mean that you >> *have* to use a sep

Re: [math] threading redux

2015-04-17 Thread Gilles
Hello. On Thu, 16 Apr 2015 17:06:21 -0500, James Carman wrote: Consider me poked! So, the Java answer to "how do I run things in multiple threads" is to use an Executor (java.util). This doesn't necessarily mean that you *have* to use a separate thread (the implementation could execute inlin

Re: [math] threading redux

2015-04-16 Thread James Carman
Consider me poked! So, the Java answer to "how do I run things in multiple threads" is to use an Executor (java.util). This doesn't necessarily mean that you *have* to use a separate thread (the implementation could execute inline). However, in order to accommodate the separate thread case, you

Re: [math] threading redux

2015-04-16 Thread Phil Steitz
On 4/16/15 5:33 AM, Hank Grabowski wrote: > I've seen some ApacheCon North America videos on YouTube (9) but not this > one. Will a video of it be posted at some point or were those only for the > keynote type presentations? Unfortunately only the keynotes. Phil > > On Wed, Apr 15, 2015 at 11:33

Re: [math] threading redux

2015-04-16 Thread Hank Grabowski
I've seen some ApacheCon North America videos on YouTube (9) but not this one. Will a video of it be posted at some point or were those only for the keynote type presentations? On Wed, Apr 15, 2015 at 11:33 AM, Phil Steitz wrote: > James Carman and I had a brief conversation following my Apache

[math] threading redux

2015-04-15 Thread Phil Steitz
James Carman and I had a brief conversation following my Apachecon talk, where I mentioned the challenge we have around deciding what to do about supporting multiple threads / processes. He has some good ideas. This is really just a poke to get him to post those ideas :) The final presented slid