Interesting blog post, Chas, which led me to doall some of my lazy sequences (not 100% sure I caught them all) and in the process I also found another source of contention that I removed (over a Date object).
But I'm still having contention somewhere that I can't find. Is there a tool that will help me to see what is being contended for, and/or who is contending for it? Anything would be helpful. Also, is it possible that there is contention over the reading, by many different threats (from 10000 agents), of the same parts of many (10000+) nested immutable lists with lots of shared structure? I know this immutable forrest is *safe* for reading by many threads, but I don't know if there's a performance implication. I've tried to make copies of list structures just to explore this, but it's not obvious how to do that in clojure and I'm not sure I've been doing it right. Thanks, -Lee On Mar 26, 2010, at 8:37 AM, Chas Emerick wrote: > Just as long as your binding form is within the function body, you're good to > go. e.g., you need to do this (or its equivalent): > > (send some-agent #(binding [random-state (Random.)] (other-fns %&))) > > *not* this: > > (binding [random-state (Random.)] > (send some-agent other-fns)) > > The latter will cause the sent fn to be invoked outside the scope of the > binding. > > I wrote up a little guide on some of the wrinkles of binding some months ago: > > http://muckandbrass.com/web/display/~cemerick/2009/11/03/Be+mindful+of+Clojure%27s+binding > > On a very minor note, these forms are more syntactically idiomatic: > > (java.util.Random.) > (.nextFloat random-state) > (.nextInt random-state n) > > Cheers, > > - Chas > > On Mar 26, 2010, at 8:14 AM, Lee Spector wrote: > >> >> Thanks all for the quick and helpful responses on the issues with >> rand-int/rand and concurrency. >> >> I probably have some other issues that are also gumming up some of my >> concurrency, but following the advice on of creating per-thread >> java.util.Random objects *seems* to have helped, although it's a little hard >> to tell and I'm not sure I did it right. >> >> What I did was: >> >> (def thread-local-random-state (new java.util.Random)) ;; just an initial >> global binding >> >> (defn lrand-int >> "Return a random integer using the thread-local random state." >> [n] >> (if (< n 1) >> 0 >> (. thread-local-random-state (nextInt n)))) >> >> (defn lrand >> "Return a random float between 0 and 1 usng the thread-local random state." >> [] >> (. thread-local-random-state (nextFloat))) >> >> And then I wrap the bodies of the the functions that I pass to send (which >> is the only way I launch threads) with: >> >> (binding [thread-local-random-state (new java.util.Random)] >> <everything else goes here, with calls that make calls that eventually >> call lrand-int and lrand> >> ) >> >> Is that the right way to do it? >> >> Thanks, >> >> -Lee >> >> >> >> On Mar 26, 2010, at 3:30 AM, mac wrote: >> >>> There is a fast Java version of Mersenne Twister here, if you feel >>> like compiling a java file: >>> http://cs.gmu.edu/~sean/research/ >>> >>> >>> On 26 mar, 05:43, Chas Emerick <[email protected]> wrote: >>>> I was going to suggest something similar using seque in an atom, but >>>> in neither case (using an atom or a ref) is the contention going to be >>>> minimized -- just shifted from the AtomicLong in java.util.Random to >>>> the now-app-level atom or ref. >>>> >>>> - Chas >>>> >>>> On Mar 26, 2010, at 12:30 AM, Andrzej wrote: >>>> >>>> >>>> >>>>> As others have pointed out using per-thread java.util.Random objects >>>>> is probably the best way to go in this particular case. However, I'm >>>>> curious if the following code could give any speed gain on your >>>>> machine: >>>> >>>>> (defn rand-seq [] (repeatedly #(. Math (random)))) >>>> >>>>> (def rand-seq-ref (ref (rand-seq))) >>>>> (nth @rand-seq-ref 100) ;; pre-cache random values; evaluate it >>>>> every some time >>>>> ;;btw, how to do it automatically? >>>> >>>>> (defn next-rand-val [] >>>>> (dosync (commute rand-seq-ref next) (first @rand-seq-ref))) >>>> >>>>> user=> (next-random-val) >>>>> 0.5558267606843464 >>>>> user=> (next-random-val) >>>>> 0.32353157456467474 >>>> >>>>> Cheers, >>>>> Andrzej >>>> >>>>> On Fri, Mar 26, 2010 at 11:35 AM, Lee Spector >>>>> <[email protected]> wrote: >>>> >>>>>> I'm trying to track down the reason that I sometimes see a lot of >>>>>> concurrency in my system (up to 1200% CPU utilization on a dual >>>>>> quadcore mac that also has some kind of hyperthreading, allegedly >>>>>> allowing a maximum of 1600% CPU) while other times it gets stuck at >>>>>> around 100-200%. My system (a genetic programming system) has a >>>>>> *lot* of randomness in it, so it's hard to repeat runs and get a >>>>>> firm handle on what's going on. >>>> >>>>>> But after a bunch of testing I'm beginning to suspect that it might >>>>>> be the random number generator itself (clojure-core/rand-int in >>>>>> this case, which calls (. Math (random))). This seems at least >>>>>> somewhat plausible to me because I guess that the underlying Java >>>>>> random method must be accessing and updating a random number >>>>>> generator state, and so this would be a concurrency bottleneck. So >>>>>> if I'm in a condition in which lots of concurrent threads are all >>>>>> calling rand-int a lot then all of the accesses to the state have >>>>>> to be serialized and my concurrency suffers (a lot). >>>> >>>>>> Does this sound plausible to you? If so, is there a straightforward >>>>>> way to avoid it? It is not important to me that the random numbers >>>>>> being generated in different threads be generated from the same >>>>>> generator or coordinated/seeded in any way. I just need lots of >>>>>> numbers that are "random enough." I guess I could roll my own >>>>>> random number generator(s) and either have a lot of them with >>>>>> independent states or maybe even make them stateless (always >>>>>> generating numbers by scrambling the clock?). But I would hope >>>>>> there would be something simpler. >>>> >>>>>> Thanks, >>>> >>>>>> -Lee >>>> >>>>>> -- >>>>>> Lee Spector, Professor of Computer Science >>>>>> School of Cognitive Science, Hampshire College >>>>>> 893 West Street, Amherst, MA 01002-3359 >>>>>> [email protected],http://hampshire.edu/lspector/ >>>>>> Phone: 413-559-5352, Fax: 413-559-5438 >>>> >>>>>> Check out Genetic Programming and Evolvable Machines: >>>>>> http://www.springer.com/10710-http://gpemjournal.blogspot.com/ >>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Clojure" group. >>>>> To post to this group, send email to [email protected] >>>>> Note that posts from new members are moderated - please be patient >>>>> with your first post. >>>>> To unsubscribe from this group, send email to >>>>> [email protected] >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/clojure?hl=en >>>> >>>>> To unsubscribe from this group, send email to clojure >>>>> +unsubscribegooglegroups.com or reply to this email with the words >>>>> "REMOVE ME" as the subject. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To post to this group, send email to [email protected] >>> Note that posts from new members are moderated - please be patient with >>> your first post. >>> To unsubscribe from this group, send email to >>> [email protected] >>> For more options, visit this group at >>> http://groups.google.com/group/clojure?hl=en >>> >>> To unsubscribe from this group, send email to >>> clojure+unsubscribegooglegroups.com or reply to this email with the words >>> "REMOVE ME" as the subject. >> >> -- >> Lee Spector, Professor of Computer Science >> School of Cognitive Science, Hampshire College >> 893 West Street, Amherst, MA 01002-3359 >> [email protected], http://hampshire.edu/lspector/ >> Phone: 413-559-5352, Fax: 413-559-5438 >> >> Check out Genetic Programming and Evolvable Machines: >> http://www.springer.com/10710 - http://gpemjournal.blogspot.com/ >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to [email protected] >> Note that posts from new members are moderated - please be patient with your >> first post. >> To unsubscribe from this group, send email to >> [email protected] >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> >> To unsubscribe from this group, send email to >> clojure+unsubscribegooglegroups.com or reply to this email with the words >> "REMOVE ME" as the subject. > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to [email protected] > Note that posts from new members are moderated - please be patient with your > first post. > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > > To unsubscribe from this group, send email to > clojure+unsubscribegooglegroups.com or reply to this email with the words > "REMOVE ME" as the subject. -- Lee Spector, Professor of Computer Science School of Cognitive Science, Hampshire College 893 West Street, Amherst, MA 01002-3359 [email protected], http://hampshire.edu/lspector/ Phone: 413-559-5352, Fax: 413-559-5438 Check out Genetic Programming and Evolvable Machines: http://www.springer.com/10710 - http://gpemjournal.blogspot.com/ -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to [email protected] Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/clojure?hl=en To unsubscribe from this group, send email to clojure+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.
