Aieh, I oversimplified a bit here, and forgot about not being able to
overload with the same arity, without multimethods.
Here is the actual code I was experimenting with, which actually runs...
(defn- read-seq-
[reader extract advance]
(let [segment (extract reader)
reader (advance reader)]
(if segment
(cons segment (lazy-seq (read-seq- reader extract advance))))))
(defmulti read-seq (fn [source & more] (class source)))
(defmethod read-seq clojure.lang.ISeq
([reader]
(read-seq reader (fn [reader] (first reader))))
([reader extract]
(read-seq reader extract (fn [reader] (next reader))))
([reader extract advance]
(read-seq- reader extract advance)))
(defmethod read-seq java.io.BufferedReader
([reader]
(read-seq reader (fn [reader] (. reader readLine))))
([reader extract]
(read-seq reader extract (fn [reader] reader)))
([reader extract advance]
(read-seq- reader extract advance)))
(defmethod read-seq String
[path]
(let [reader (clojure.java.io/reader path)]
(read-seq reader
(fn [reader]
(let [segment (. reader readLine)]
(if segment segment (. reader close)))))))
On Sun, Mar 27, 2011 at 12:17 PM, Stefan Sigurdsson <[email protected]>wrote:
> How do you guys normally manage resources when using lazy sequences?
>
> I was playing around with this question, and I looked into extending
> line-seq to take a file path parameter that is coerced to reader:
>
> (defn line-seq
> ([^java.io.BufferedReader rdr]
> (when-let [line (.readLine rdr)]
> (cons line (lazy-seq (line-seq rdr)))))
> ([^String path]
> (line-seq (clojure.java.io/reader path)))) ; Oops
>
> This sort of works, but not well, because we're relying on finalize to
> close the reader.
>
> (If the extended line-seq is being used by code that opens a lot of files,
> but reads all it needs from each file before moving on, then we'd only have
> one live file handle at any given time, but we could exhaust the number of
> concurrently available file handles before the garbage collector catches
> up.)
>
> Instead the extended line-seq needs to control the closing of the reader it
> opens, but that reader needs to remain open until the last sequence element
> has been consed - which is normally long after the extended line-seq
> returns.
>
> The normal Clojure-style with-open approach doesn't work at all since the
> reader is closed after reading the first line, and we can't read any of the
> remaining lines:
>
> (defn line-seq
> ([^java.io.BufferedReader rdr]
> (when-let [line (.readLine rdr)]
> (cons line (lazy-seq (line-seq rdr))))))
> ([^String filename]
> (with-open [rdr (clojure.java.io/reader filename)] ;
> Oops...
> (line-seq rdr))))
>
> Closing the reader explicitly when the file has been read fixes the
> extended line-seq but would
> break other applications that rely on the original line-seq:
>
> (defn line-seq
> ([^java.io.BufferedReader rdr]
> (if-let [line (.readLine rdr)]
> (cons line (lazy-seq (line-seq rdr)))
> (.close rdr))) ; Ouch
> ([^String filename]
> (line-seq (clojure.java.io/reader filename)))
>
> Adding another overload seems to be the most reasonable solution:
>
> (defn line-seq
> ([^java.io.BufferedReader rdr]
> (line-seq rdr (fn [rdr] (.readLine rdr))))
> ([^java.io.BufferedReader rdr extract]
> (when-let [line (extract rdr)]
> (cons line (lazy-seq (line-seq rdr)))))
> ([^String filename]
> (let [extract (fn [rdr] (if-let [line (.readLine rdr)] line
> (.close rdr))
> (line-seq (clojure.java.io/reader filename) extract)))
>
> But this feels a little cumbersome. Am I missing something? Is there a
> better way?
>
> (I'm just using line-seq for illustration here.)
>
> Cheers,
>
> Stefan
>
> On Fri, Mar 25, 2011 at 10:22 AM, Jules <[email protected]> wrote:
> yes
>
> and that's great where the resource usage is scoped on a per-thread basis,
> but not a per-object basis - but then, I am thinking in OO terms again :-)
>
> Jules
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to [email protected]
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> [email protected]
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
>
>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
