I also posted this to StackOverflow, so sorry if you saw it there too. If
you want some rep points over there you can answer there too (
http://stackoverflow.com/questions/12427518/clojure-lazy-seq-over-java-iterative-code
).
I'm trying to use create a Clojure seq from some iterative Java library
code that I inherited. Basically what the Java code does is read records
from a file using a parser, sends those records to a processor and returns
an ArrayList of result. In Java this is done by calling parser.readData(),
then parser.getRecord() to get a record then passing that record into
processor.processRecord(). Each call to parser.readData() returns a single
record or null if there are no more records. Pretty common pattern in Java.
So I created this next-record function in Clojure that will get the next
record from a parser.
(defn next-record
"Get the next record from the parser and process it."
[parser processor]
(let [datamap (.readData parser)
row (.getRecord parser datamap)]
(if (nil? row)
nil
(.processRecord processor row 100))))
The idea then is to call this function and accumulate the records into a
Clojure seq (preferably a lazy seq). So here is my first attempt which
works great as long as there aren't too many records:
(defn datamap-seq
"Returns a lazy seq of the records using the given parser and processor"
[parser processor]
(lazy-seq
(when-let [records (next-record parser processor)]
(cons records (datamap-seq parser processor)))))
I can create a parser and processor, and do something like (take 5
(datamap-seq parser processor)) which gives me a lazy seq. And as expected
getting the (first) of that seq only realizes one element, doing count
realizes all of them, etc. Just the behavior I would expect from a lazy seq.
Of course when there are a lot of records I end up with a
StackOverflowException. So my next attempt was to use loop-recur to do the
same thing.
(defn datamap-seq
"Returns a lazy seq of the records using the given parser and processor"
[parser processor]
(lazy-seq
(loop [records (seq '())]
(if-let [record (next-record parser processor)]
(recur (cons record records))
records))))
Now using this the same way and defing it using (def results (datamap-seq
parser processor)) gives me a lazy seq and doesn't realize any elements.
However, as soon as I do anything else like (first results) it forces the
realization of the entire seq.
Can anyone help me understand where I'm going wrong in the second function
using loop-recur that causes it to realize the entire thing?
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
