Hi, The reduce is not a big problem, 'cause after all it'll only process 88 records.
It's the mapper function that cost much more than the java version. One possibility I guess is that the Clojure startup time is slow. So tomorrow I'll try to reduce the mapper task count and see whether it helps. On Friday, April 26, 2013 8:23:04 PM UTC+8, Max Penet wrote: > > Hi, > > In reducer-reduce you iterate twice over the values compared to the java > version, once in map (just to call .get), then in reduce. > > There are other issues probably, but this is one of the obvious ones. > > On Friday, April 26, 2013 12:05:33 PM UTC+2, Ji Zhang wrote: >> >> Hi, >> >> I'm writing map-reduce job with Clojure, yet to find that it seems to be >> much slower than a Jave job. >> >> So I write a simple test case, and upload to gist: >> https://gist.github.com/jizhang/5466149 >> >> At the end of code, there is execution outputs, here are some significant >> stats: >> >> Average time taken by Map tasks: Java 7sec, Clojure 19sec >> CPU time spent (ms): Java 244,000, Clojure 1,145,440 >> >> I'm wondering what slows down the Clojure written map-reduce job. Am I >> using it wrong, or it's just an inappropriate senario. >> >> Any thoughts will be great. Thanks! >> >> Jerry >> > -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to [email protected] Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
