I noticed recently that clojure.core/memoize does not promise that
memoized calls will occur only once in the presence of multiple
threads. i.e:
user=> (dorun (map (memoize (fn [x] (Thread/sleep 1000) (print x)))
(repeat 10 1)))
1nil
However:
user=> (dorun (pmap (memoize (fn [x] (Thread/sleep 1000) (print x)))
(repeat 10 1)))
1111111111nil
The program I'm writing reads a number of input files to produce java
code. I've memoized the function that reads and parses the input files
since some of them are used multiple times and it's wasteful to read
and parse them multiple times. This works as expected when I run the
jobs on a single thread. When I use pcalls, however, performance drops
far below the single threaded case. This is because the memoized input
function will happily read the same input file more than once. This
does no harm to correctness, but does great harm to performance.
Delays do guarantee this kind of once-and-only-once that I'm looking
for, so I've redesigned the program to compute a map of delays for the
possible input files and use those to configure all jobs before they
are run. I'm not quite done with that redesign, and even when it
works, I'll need to go back and clean things up.
It occurs to me that there might be a way to write memoize
("memoize-once") that has the property I'm looking for. It would be
easier to just use such a memoize-once than to continue along the road
I've started down. Does such a memoize already exist?
// Ben
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
