Hello!
I've recently started playing with Clojure, and a couple of days ago I've
been pointed to Reducers in Clojure 1.5 after a discussion on #clojure at
Freenode.
I've read Rich's posts announcing the Reducers library, and he says that
there's a ***lack of reducible IO sources***. I'm working on a project that
in which I analyze lots of data (from huge files on disk), and I'm
frequently doing "reduce" operations on it. I began by writing a custom
sequence, that after I transformed in a custom reducible collection. But
I've just noticed that I could encapsulate it on a kind of reducible IO
source.
I'm writing this message to share the code I wrote relating to reducible IO
sources, if anybody sees any use for it. The idea is to make the ***NIO
Buffers reducible***, so that you can open huge files as **memory mapped
files**, which are instances of `ByteBuffer`. You can then obtain instances
of other buffers, say, `LongBuffer` (which is what I'm using on my current
project), which are also reducible:
(defmacro buffer-reduce [b f val]
`(let [b# (.duplicate ~b)]
(loop [remaining# (.remaining b#)
result# ~val]
(if (zero? remaining#)
result#
(recur (dec remaining#) (~f result# (.get ~b)))))))
(extend-protocol clojure.core.protocols/CollReduce
java.nio.ByteBuffer
(coll-reduce [b f] (coll-reduce b f (f)))
(coll-reduce [b f val] (buffer-reduce b f val))
java.nio.LongBuffer
(coll-reduce [b f] (coll-reduce b f (f)))
(coll-reduce [b f val] (buffer-reduce b f val))
; ... other kinds of buffer ...
)
Hope this might be useful for someone.
Bruno
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
