Hi all,
I tripped over something strange yesterday. I work on a tool to create
reports in the REPL from data in an SQL database.
Instead of building SQL statements with complex where clauses, the user
can defined which tables/fields he wants to fetch
and then he can work on these locally.
I implemented local joins and a number of other related operations to
marshal the data.
While doing this, I worked with clojure.set and used the index function
to regroup records matching the same
keys. I then found out that I was not getting the same # of record from
my "local" join compared to the equivalent
SQL statement. The input data was the same however (I compared SQL
statement outputs with my local "fetch" outputs.
I was "losing" records, instead of getting 18000 something records, I
was down to 14000 something.
In fact some keys where missing in the output of index. I did not find a
pattern in the missing entries.
While digging, I started to question the index function sanity from
clojure.set. I cloned it and after a few hours of tossing it around,
decided
to recode it using a mutable Java HashMap.
It then worked... I got the same number of records as in the equivalent
SQL statement output.
The index function in clojure.set goes like this:
(defn index
"Returns a map of the distinct values of ks in the xrel mapped to a
set of the maps in xrel with the corresponding values of ks."
[xrel ks]
(reduce
(fn [m x]
(let [ik (select-keys x ks)]
(assoc m ik (conj (get m ik #{}) x))))
{} xrel))
It looks great and I suspect that the problem is deeper in the core.
Anyone has encountered something similar ? The problem can be replicated
in 1.0 and 1.1.
I did not have the time now to test it again using the master branch.
I will investigate this in the runtime later but maybe someone has
crossed over this problem in the past.
Luc
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
