On Dec 8, 2012, at 9:36 AM, Marshall Bockrath-Vandegrift wrote:
>
> Although it doesn’t impact your benchmark, `pmap` may be further
> adversely affecting the performance of your actual program. There’s a
> open bug regarding `pmap` and chunked seqs:
>
> http://dev.clojure.org/jira/browse/CLJ-862
>
> The impact is that `pmap` with chunked seq input will spawn futures for
> its function applications in flights of 32, spawning as many flights as
> necessary to reach or exceed #CPUS + 2. On a 48-way system, it will
> initially launch 64 futures, then spawn an additional 32 every time the
> number of active unrealized futures drops below 50, leading to
> significant contention for a CPU-bound application.
>
> I hope it can be made useful in a future version of Clojure, but right
> now `pmap` is more of an attractive nuisance than anything else.
(I said this would be in a reply to Jim, but now I see it makes more sense in a
reply to this message of Marshall's.)
In order to avoid issues with pmap we've also run tests using "pmapall",
defined as follows:
(defn pmapall
"Like pmap but: 1) coll should be finite, 2) the returned sequence
will not be lazy, 3) calls to f may occur in any order, to maximize
multicore processor utilization, and 4) takes only one coll so far."
[f coll]
(let [agents (map agent coll)]
(dorun (map #(send % f) agents))
(apply await agents)
(doall (map deref agents))))
The results are more or less the same (awful).
-Lee
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
