Hi,
I've a head holding problem that I believe is a bug in clojure 1.3. I
wrote the following function to split a a lazy seq of strings across
files of x size:
(defn split-file
([path strs size]
(trampoline split-file path (seq strs) size 0))
([path strs size part]
(with-open [f (clojure.java.io/writer (str path "." part))]
(loop [written 0, ss strs]
(when ss
(if (>= written size)
#(split-file path ss size (inc part))
(let [s (first ss)]
(.write f s)
(recur (+ written (.length s)) (next ss)))))))))
If I call the 3 arg version of the function:
(split-file "foo" (repeat 100000000 "blah blah blah") 100000000)
I see memory usage increases as I'm writing each file with the usual
gc slow down, then memory usage goes back down again as I get to a new
split file.
Memory usage is fine if I call the 4 arg version (which only writes
one part of the split file):
(split-file "foo" (repeat 100000000 "blah blah blah") 100000000 0)
I can also avoid the head holding problem by removing trampoline and
recursively calling split-file directly, but then those recursive
calls use up stack and don't close files until all calls complete
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
