Recently I was studying Stuart Sierra's clojure-hadoop project¹, and there saw a technique that I'd like to discuss. There's a Java class generated whose method definitions get patched based on a provided configuration, and I'd like to understand the scope of this patching and why the technique's effects are acceptable for this project.
The namespace clojure-hadoop.job² uses the macro clojure-hadoop.gen/gen-job-classes³ to, well, generate a few Java classes. Later, in the job/configure-functions functionº, we see use of the alter-var-root function to change the functions associated with the Java methods Mapper#map() (here, mapper-map) and Reducer#reduce() (here, reducer-reduce). Now to my questions. This mutation of the class-method-to-Clojure-function binding appears to be "global." Will it impact /all/ instances of the generated classes within this Clojure process? If this impact is global, is it the case that this software never needs to accommodate instances of these generated classes with different configurations? In other words, the "job"-related classes wind up getting mutated to conform to one particular configuration. That means that a given run of the program can't handle more than one configuration at a time. Is that acceptable here because Hadoop is only going to load these classes for use with a single configuration? The patching looks like an optimization to avoid associating functions with each Mapper or Reducer /instance/, so that it avoids that kind of which-function-should-I-call-now lookup on each invocation. Is that a correct interpretation of the design rationale? Footnotes: ¹ https://github.com/stuartsierra/clojure-hadoop ² https://github.com/stuartsierra/clojure-hadoop/blob/master/src/main/clojure/clojure_hadoop/job.clj ³ https://github.com/stuartsierra/clojure-hadoop/blob/master/src/main/clojure/clojure_hadoop/gen.clj#L5 º https://github.com/stuartsierra/clojure-hadoop/blob/master/src/main/clojure/clojure_hadoop/job.clj#L31 -- Steven E. Harris -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to [email protected] Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/clojure?hl=en
