When I designed the new makefiles for the OpenJDK project (http://openjdk.java.net/projects/build-infra/guide.html) The first and foremost problem was, how to efficiently compile Java software from makefiles.
That included: 1 not wanting to spawn a jvm, to check if any java sources had changed. 2 efficient and correct incremental compiles 3 using multiple cores when compiling Java 4 the makefile should be reasonable easy to write 2 and 3 cannot be solved without improvements to javac. These have now been implemented in the smart javac wrapper as part of the build-infra project, they are not yet part of the official OpenJDK. For 4, I created a neat way of passing named parameters to make macros. Thus with the utility library JavaCompilation.gmk (http://hg.openjdk.java.net/build-infra/jdk8/file/345dbc49f7d0/common/makefiles/JavaCompilation.gmk) I can write: $(eval $(call SetupJavaCompiler,GENERATE_NEWBYTECODE, \ JAVAC:=javac,\ FLAGS:=-g) $(eval $(call SetupJavaCompilation,BUILD_JAXP,\ SETUP:=GENERATE_NEWBYTECODE,\ SRC:=jaxp/src,\ BIN:=output/classes,\ SRCZIP:=output/src.zip),\ JAR:=output/classes.jar)) all: output/src.zip output/classes.jar This will setup all the necessary makefile rules to generate src.zip and classes.jar and do proper dependency checking and incremental builds. Now, how about 1: not wanting to spawn a jvm, to check if any java sources had changed. You want to find all relevant sources to compile, easily done using find, for example SRCS:=$(shell find src -name "*.java") SRCS:=do all sorts of filtering on the SRCS _the.batch : $(SRCS) rm _the.batch javac -d bin $(SRCS) touch _the.batch Simple eh? Unfortunately the command line length limits hits us very quickly when compiling java! For example the jdk in the OpenJDK project has ~8800 source files, and this is a small Java project! Its not a problem for make which deals with these number of files fast and efficiently. As you know the number of Java files in a project is a magnitude larger compared to if the project was written in C. If Java source code was more modular, we could of course rely on developers to split up the sources into smaller jars. But it won't help on some platforms like cygwin where the command line length limit is so short that it can barely handle small java project. To get around this particular problem I implemented a workaround marcro called ListPathsSafely that writes the contents of a variable to disk. (http://hg.openjdk.java.net/build-infra/jdk8/file/345dbc49f7d0/common/makefiles/MakeBase.gmk) that lies somewhere in the borderlands between genius and insanity.... I would like to get rid of ListPathsSafely, to do so I would like to suggest adding two new functions to GNU make: file-write and file-writelns. For example: $(file-writelns list.txt,$(SRCS)) would write each word in SRCS on each own line into the file list.txt. $(file-write list.txt,$(SRCS)) would write all the sources on a single line with spaces intact. This is a very simple addition to GNU make, I have supplied implementation and a test at the end of this email. I do not believe it would cause any security holes in make. It would give a lot of developers, not just Java developers, a very useful tool to get around severe command line lengths limits on platforms like cygwin. With these two new functions it would be trivial to write a portable, efficient makefile for Java that can deal with large projects. SRCS:=$(shell find src -name "*.java") SRCS:=do all sorts of filtering on the SRCS _the.batch : $(SRCS) rm _the.batch $(file-writelns _the.batch.tmp,$(SRCS)) javac -d bin @_the.batch.tmp mv _the.batch.tmp _the.batch If you are interested in more details, I gave a presentation on this topic on JavaOne: CON6659 - Building Large Java Projects Faster: Multicore javac and Makefile Integration http://www.myexpospace.com/JavaOne2012/SessionFiles/CON6659_PDF_6659_0001.pdf http://www.myexpospace.com/JavaOne2012/SessionFiles/CON6659_mp4_6659_001.mp4 Fredrik Öhrström ---------------------------------------- In function.c /* $(file-write file-name,content) $(file-writeln file-name,content) Write the content into the newly opened (truncated) file-name. The file-name argument is trimmed from any leading or ending whitespace. file-write writes the content as is, to disk. file-writelns will write '\n' after each and every word in content. */ static char * func_file_write (char *o, char **argv, const char *funcname) { const char *file_name = argv[0]; const char *content = argv[1]; FILE *f; int is_writelns = streq (funcname, "file-writelns"); if (file_name != NULL && content != NULL) { const char *end = file_name + strlen (file_name) - 1; strip_whitespace (&file_name, &end); ((char*)end)[1] = '\0'; if (file_name != NULL) { f = fopen(file_name, "w"); if (f != NULL) { if (is_writelns) { /* Append a new line after each word found in content. */ const char *word_iterator = content; unsigned int len; const char *w = content; while ((w = find_next_token (&word_iterator, &len)) != 0) { fwrite(w, 1, len, f); fwrite("\n", 1, strlen("\n"), f); } } else { /* Write content, as is, to disk. */ int n = strlen(content); fwrite(content, 1, n, f); } fclose(f); } } } return o; } and { STRING_SIZE_TUPLE("file-write"), 1, 2, 1, func_file_write}, { STRING_SIZE_TUPLE("file-writelns"), 1, 2, 1, func_file_write}, and a new test tests/scripts/functions/file-write # -*-perl-*- $description = 'Test the $(file-write ...) function.'; $details = ''; $file_to_write = $workpath.'/'.$dir.'/generated'; #### Test a simple generation run_make_test(' .PHONY: all define NL endef $(file-write '.$file_to_write.',hi$(NL)) all: ; @cat '.$file_to_write.' ','','hi'); #### Test writelns run_make_test(' .PHONY: all $(file-writelns '.$file_to_write.',hi) all: ; @cat '.$file_to_write.' ','','hi'); #### Test multiple lines run_make_test(' .PHONY: all $(file-writelns '.$file_to_write.',alfa beta gamma) all: ; @cat '.$file_to_write.' ','','alfa beta gamma'); #### Test that make handles bad input to file-write without crashing. run_make_test(' .PHONY: all $(file-write) $(file-write,hi) $(file-write,,hi) $(file-writelns) $(file-writelns,hi) $(file-writelns,,hi) all: ; @echo hi ','','hi'); 1; _______________________________________________ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
