Hi all,
I'd like to get some feedback on an approach I took to limit the
concurrency of parallel processing in some of my Clojure codebase.
I've often found that I want to split work over some number of threads
greater than CPUs + 2, but less than infinity. In the past I had used
promises that were fulfilled by a FixedThreadPool. But that's a lot of
setup and teardown that distracts from what you're actually trying to
achieve.
My new approach was to leverage a Semaphore:
(defn bounded-pmap [limit f coll]
(let [sem (Semaphore. limit true)
step (fn [x]
(.acquire sem)
(future
(try (f x)
(finally (.release sem)))))]
(->> (map step coll)
(doall)
(map deref))))
It's not lazy, but in my case the collection is already fully realized. I'm
interested in constructive criticism or other approaches. Thanks!
Best,
Brian
--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
