This is lazy: (remove #{(first scores)} scores)
Since find-related-positive-negative-true-false-records does not realize
set-of-scores in order to return its value, then you still have laziness on
scores.
I think, in your original, if you wrap that remove call with doall, you won’t
get a stack overflow.
Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood
On 4/3/17, 9:24 AM, "[email protected] on behalf of
[email protected]" <[email protected] on behalf of
[email protected]> wrote:
I apologize for my ignorance, but why is this lazy?
(conj vector-of-maps-of-vector-key-and-score
(find-related-positive-negative-true-false-records scores (first scores))))
Assuming the functions are not called eagerly simply because I am in a loop.
But what would the eager version of this be? I
On Sunday, April 2, 2017 at 1:42:20 AM UTC-4, Sean Corfield wrote:
> But does that mean one can't use functions to build up sequences in a loop?
> That seems to limit loops a great deal.
You just need to ensure you are building the sequences in an eager manner,
rather than a lazy manner.
Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood
From: [email protected]
Sent: Saturday, April 1, 2017 9:56 PM
To: Clojure
Subject: Re: I can not find any function that might give rise
tothisStackOverflow error
> Because when you recur in your loop, you’re passing in lazy sequences, so
> those
> are essentially building up a giant stack of delayed evaluations – and when
> you hit bottom
> and try to realize those, that’s when your stack overflow hits you.
I was thinking the answer had something to do with the lazyness, but I wasn't
sure because the app seemed to die before the loop finished -- which I assume
is before the functions were realized. But it is possible the logging functions
were slow and didn't show the whole truth. I am grateful to have someone with
your experience clarify this.
But does that mean one can't use functions to build up sequences in a loop?
That seems to limit loops a great deal.
On Saturday, April 1, 2017 at 10:22:04 PM UTC-4, Sean Corfield wrote:
Because when you recur in your loop, you’re passing in lazy sequences, so those
are essentially building up a giant stack of delayed evaluations – and when you
hit bottom and try to realize those, that’s when your stack overflow hits you.
By contrast, your reduce code is eager so the work is done as you work through
the sequence, instead of “stacked up” lazily, to be done later.
Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood
From: [email protected]
Sent: Saturday, April 1, 2017 6:52 PM
To: Clojure
Subject: Re: I can not find any function that might give rise to
thisStackOverflow error
Crazy! I re-wrote the (loop) to use (reduce) instead and now everything works:
(defn loop-over-scores
[set-of-scores]
"2017-03-08 -- called from start.clj"
(reduce
;; 2017-04-01 -- we assume vector-with-path-score looks like this:
;; [[:positive :true 0.88 19 60 10 12 3 1 3 1 2 1] 1]
;; so we do a minor bit of destructuring
(fn [vector-of-maps-of-vector-key-and-score [vector-with-path score]]
(conj vector-of-maps-of-vector-key-and-score
(dire/supervise #'map-of-vector-path-and-true-false-scores
vector-with-path)))
[]
set-of-scores))
The StackOverflow error is gone.
Sad to say, I'm left with a feeling as if I know nothing about Clojure. I don't
know why (loop) gave me a StackOverflow error whereas (reduce) works just fine.
This is black magic to me, which, sad to say, is not a comfortable feeling.
On Saturday, April 1, 2017 at 7:51:08 PM UTC-4, [email protected] wrote:
Well, I am out of ideas. Let's assume I'll re-write this some other way. What
would be better than using (loop)? What would be less likely to cause
StackOverflow, or at least reveal why I'm seeing it.
On Saturday, April 1, 2017 at 6:23:29 PM UTC-4, [email protected] wrote:
I have a function that will run repeatedly, so I use the at-at library to call
it:
https://github.com/overtone/at-at
I don't think this is the problem.
Sad to say, the Error is catching a StackOverflow, which I'm having trouble
finding. I don't see a place where I call a function recursively, so I don't
see where any of this becomes stackoverflow.
I'm using Dire to catch errors, but for now I'm just logging them. I don't see
anything fancy or clever that would give me a stackoverflow, and yet this
anonymous function called by at-at is definitely giving rise to StackOverflow.
(defn- calculate--scores []
(let [my-pool (at/mk-pool)]
(at/every 180000
(fn []
(append-to-file "/var/log/example/-scores.log" "Will attempt to
write scores")
(future
(try
(let [
map-of-all-sums (api/get-accuracy-and-precision)
set-of-all-scores (dire/supervise #'api/path-entries
map-of-all-sums)
path-and--scores (dire/supervise
#'api/loop-over-scores set-of-all-scores)
]
(append-to-file "/var/log/example/-scores.log"
"\n\n\n\n\n")
(append-to-file "/var/log/example/-scores.log" "
path-and--scores: ")
(append-to-file "/var/log/example/-scores.log" (str
"(count set-of-all-scores): " (count set-of-all-scores)))
(append-to-file "/var/log/example/-scores.log"
path-and--scores))
(catch Exception e
(do
(append-to-file "/var/log/example/-scores.log" (str "
EXCEPTION:: " e))
(append-to-file "/var/log/example/-scores.log"
(stack/parse-exception e))
(errors/error e)))
(catch Error o
(println (str " a problem in the anonymous function in
calculate--scores: " o)))
)))
my-pool)))
The problem appears to be here, which will loop several thousand times and then
die. But I don't see why. (This function used to just be a loop, and I added
the (let) so I could see if I could read the end -- but I could not reach the
end).
(defn loop-over-scores
[set-of-scores]
"2017-03-08 -- called from start.clj"
(try
(let [return-value (loop [
how-many-loops 0
scores set-of-scores
vector-of-maps-of-vector-key-and-score []
]
(println (str " in loop-over-scores again "
how-many-loops))
(if (next scores)
(recur
(inc how-many-loops)
(remove #{(first scores)} scores)
(conj vector-of-maps-of-vector-key-and-score
(find-related-positive-negative-true-false-records scores (first scores))))
vector-of-maps-of-vector-key-and-score))
]
(println " return-value in loop-over-scores " return-value " in
loop-over-scores")
return-value)
(catch Exception e (println " exception in loop-over-scores: " e))))
So this line prints out a result several thousand times:
(println " return-value in loop-over-scores " return-value " in
loop-over-scores")
And then I get stackoverflow.
But nothing seems suspicious here:
(defn find-related-positive-negative-true-false-records [set-of-scores
vector-with-path-score]
(println " in find-related-positive-negative-true-false-records the
vector-with-path-score " vector-with-path-score)
(let [
[vector-with-path score] vector-with-path-score
;; 2017-03-08 -- seq-as-path should be something like [30 8 34 20.94 2]
seq-as-path (rest (rest vector-with-path))
;;_ (append-to-file "/var/log/example/-scores.log" "seq-as-path")
;; _ (append-to-file "/var/log/example/-scores.log" seq-as-path)
vector-as-path (apply conj [] seq-as-path)
_ (append-to-file "/var/log/example/-scores.log" "vector-as-path")
_ (append-to-file "/var/log/example/-scores.log" vector-as-path)
]
(dire/supervise #'map-of-vector-path-and-true-false-scores vector-as-path)))
Nor here:
(defn map-of-vector-path-and-true-false-scores
[vector-as-path]
(println " at the start of map-of-vector-path-and-true-false-scores")
(try
(if-not (empty? vector-as-path)
(let [
;; 2017-03-30 -- this fails:
;; user> (apply conj [:positive :false] (30 6 40))
;; ClassCastException java.lang.Long cannot be cast to
clojure.lang.IFn user/eval1719 (NO_SOURCE_FILE:1)
;;
;; but this works:
;; user> (apply conj [:positive :false] '(30 6 40))
;;[:positive :false 30 6 40]
;;
true-positive-path (apply conj [:positive :true] vector-as-path)
false-positive-path (apply conj [:positive :false] vector-as-path)
false-negative-path (apply conj [:negative :false] vector-as-path)
true-positive (get-in @accuracy-and-precision true-positive-path 0)
_ (append-to-file "/var/log/example/paths-with-score.log" (str
true-positive-path " score: " true-positive))
false-positive (get-in @accuracy-and-precision false-positive-path
0)
_ (append-to-file "/var/log/example/paths-with-score.log" (str
false-positive-path " score: " false-positive))
false-negative (get-in @accuracy-and-precision false-negative-path
0)
_ (append-to-file "/var/log/example/paths-with-score.log" (str
false-negative-path " score: " false-negative))
]
{ vector-as-path (dire/supervise #'calculate--score true-positive
false-positive false-negative) }))
(catch Exception e (println " problem in
map-of-vector-path-and-true-false-scores " e))
(catch Error o (println " problem without exception in
map-of-vector-path-and-true-false-scores " o))
))
Nor here:
(defn calculate--score [true-positive false-positive false-negative]
(let [
twice-true-positive (* true-positive 2)
denominator (+ twice-true-positive false-negative false-positive)
]
(if (> denominator 0)
(/ twice-true-positive denominator)
0)))
There is nothing here where a function calls itself over and over, so what
would lead to StackOverflow?
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
