Thanks all for the quick and helpful responses on the issues with rand-int/rand
and concurrency.
I probably have some other issues that are also gumming up some of my
concurrency, but following the advice on of creating per-thread
java.util.Random objects *seems* to have helped, although it's a little hard to
tell and I'm not sure I did it right.
What I did was:
(def thread-local-random-state (new java.util.Random)) ;; just an initial
global binding
(defn lrand-int
"Return a random integer using the thread-local random state."
[n]
(if (< n 1)
0
(. thread-local-random-state (nextInt n))))
(defn lrand
"Return a random float between 0 and 1 usng the thread-local random state."
[]
(. thread-local-random-state (nextFloat)))
And then I wrap the bodies of the the functions that I pass to send (which is
the only way I launch threads) with:
(binding [thread-local-random-state (new java.util.Random)]
<everything else goes here, with calls that make calls that eventually
call lrand-int and lrand>
)
Is that the right way to do it?
Thanks,
-Lee
On Mar 26, 2010, at 3:30 AM, mac wrote:
> There is a fast Java version of Mersenne Twister here, if you feel
> like compiling a java file:
> http://cs.gmu.edu/~sean/research/
>
>
> On 26 mar, 05:43, Chas Emerick <[email protected]> wrote:
>> I was going to suggest something similar using seque in an atom, but
>> in neither case (using an atom or a ref) is the contention going to be
>> minimized -- just shifted from the AtomicLong in java.util.Random to
>> the now-app-level atom or ref.
>>
>> - Chas
>>
>> On Mar 26, 2010, at 12:30 AM, Andrzej wrote:
>>
>>
>>
>>> As others have pointed out using per-thread java.util.Random objects
>>> is probably the best way to go in this particular case. However, I'm
>>> curious if the following code could give any speed gain on your
>>> machine:
>>
>>> (defn rand-seq [] (repeatedly #(. Math (random))))
>>
>>> (def rand-seq-ref (ref (rand-seq)))
>>> (nth @rand-seq-ref 100) ;; pre-cache random values; evaluate it
>>> every some time
>>> ;;btw, how to do it automatically?
>>
>>> (defn next-rand-val []
>>> (dosync (commute rand-seq-ref next) (first @rand-seq-ref)))
>>
>>> user=> (next-random-val)
>>> 0.5558267606843464
>>> user=> (next-random-val)
>>> 0.32353157456467474
>>
>>> Cheers,
>>> Andrzej
>>
>>> On Fri, Mar 26, 2010 at 11:35 AM, Lee Spector
>>> <[email protected]> wrote:
>>
>>>> I'm trying to track down the reason that I sometimes see a lot of
>>>> concurrency in my system (up to 1200% CPU utilization on a dual
>>>> quadcore mac that also has some kind of hyperthreading, allegedly
>>>> allowing a maximum of 1600% CPU) while other times it gets stuck at
>>>> around 100-200%. My system (a genetic programming system) has a
>>>> *lot* of randomness in it, so it's hard to repeat runs and get a
>>>> firm handle on what's going on.
>>
>>>> But after a bunch of testing I'm beginning to suspect that it might
>>>> be the random number generator itself (clojure-core/rand-int in
>>>> this case, which calls (. Math (random))). This seems at least
>>>> somewhat plausible to me because I guess that the underlying Java
>>>> random method must be accessing and updating a random number
>>>> generator state, and so this would be a concurrency bottleneck. So
>>>> if I'm in a condition in which lots of concurrent threads are all
>>>> calling rand-int a lot then all of the accesses to the state have
>>>> to be serialized and my concurrency suffers (a lot).
>>
>>>> Does this sound plausible to you? If so, is there a straightforward
>>>> way to avoid it? It is not important to me that the random numbers
>>>> being generated in different threads be generated from the same
>>>> generator or coordinated/seeded in any way. I just need lots of
>>>> numbers that are "random enough." I guess I could roll my own
>>>> random number generator(s) and either have a lot of them with
>>>> independent states or maybe even make them stateless (always
>>>> generating numbers by scrambling the clock?). But I would hope
>>>> there would be something simpler.
>>
>>>> Thanks,
>>
>>>> -Lee
>>
>>>> --
>>>> Lee Spector, Professor of Computer Science
>>>> School of Cognitive Science, Hampshire College
>>>> 893 West Street, Amherst, MA 01002-3359
>>>> [email protected],http://hampshire.edu/lspector/
>>>> Phone: 413-559-5352, Fax: 413-559-5438
>>
>>>> Check out Genetic Programming and Evolvable Machines:
>>>> http://www.springer.com/10710-http://gpemjournal.blogspot.com/
>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to [email protected]
>>> Note that posts from new members are moderated - please be patient
>>> with your first post.
>>> To unsubscribe from this group, send email to
>>> [email protected]
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>
>>> To unsubscribe from this group, send email to clojure
>>> +unsubscribegooglegroups.com or reply to this email with the words
>>> "REMOVE ME" as the subject.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to [email protected]
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to
> [email protected]
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
>
> To unsubscribe from this group, send email to
> clojure+unsubscribegooglegroups.com or reply to this email with the words
> "REMOVE ME" as the subject.
--
Lee Spector, Professor of Computer Science
School of Cognitive Science, Hampshire College
893 West Street, Amherst, MA 01002-3359
[email protected], http://hampshire.edu/lspector/
Phone: 413-559-5352, Fax: 413-559-5438
Check out Genetic Programming and Evolvable Machines:
http://www.springer.com/10710 - http://gpemjournal.blogspot.com/
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
To unsubscribe from this group, send email to
clojure+unsubscribegooglegroups.com or reply to this email with the words
"REMOVE ME" as the subject.
