Hi, I'm currently trying to fix Maven dependencies for Crunch and ran into trouble with the POM for hadoop-core 1.0.3. It looks like the Maven dependencies are different from the actual dependencies at runtime.
As a result, bugs caused by dependency conflicts won't show up until runtime, makeing unit and integration tests in Maven useless for detecting this class of problems. Unfortunately, these bugs tend to be quite nasty. For example, Hadoop ships a very old version of slf4j. If my job includes a newer one this can lead to MethodNotFoundExceptions. Since the dependency to slf4j is undeclared in hadoop-core, there is no way to detect this using Maven dependency reports. Other conflicts may lead to different behavior which is a lot harder to diagnose. What's the best way to get this fixed? Or is there something I'm missing? Below I attached a report of the differences I found. I didn't list the correct ones. Thanks, Matthias On the classpath as per org.apache.hadoop:hadoop-core:jar:1.0.3 --------------------------------------------------------------- ant-1.6.5.jar commons-logging-1.0.3.jar jackson-core-asl-1.0.1.jar jackson-mapper-asl-1.0.1.jar jets3t-0.7.1.jar jsp-2.1-6.1.14.jar jsp-api-2.1-6.1.14.jar junit-3.8.1.jar kfs-0.3.jar servlet-api-2.5-6.1.14.jar Actually on the classpath as per lib directory of hadoop-1.0.3-bin.tar.gz ------------------------------------------------------------------------- asm-3.2.jar aspectjrt-1.6.5.jar aspectjtools-1.6.5.jar commons-daemon-1.0.1.jar commons-io-2.1.jar commons-logging-1.1.1.jar commons-logging-api-1.0.4.jar hadoop-capacity-scheduler-1.0.3.jar hadoop-fairscheduler-1.0.3.jar hadoop-thriftfs-1.0.3.jar jackson-core-asl-1.8.8.jar jackson-mapper-asl-1.8.8.jar jdeb-0.8.jar jersey-core-1.8.jar jersey-json-1.8.jar jersey-server-1.8.jar jets3t-0.6.1.jar jsch-0.1.42.jar junit-4.5.jar kfs-0.2.2.jar log4j-1.2.15.jar mockito-all-1.8.5.jar slf4j-api-1.4.3.jar slf4j-log4j12-1.4.3.jar
