On 23 Jun 2003, Paul D. Smith wrote:
> %% Ted Stern <[EMAIL PROTECTED]> writes:
>
> ts> My version of the "make: *** virtual memory exhausted. Stop." bug
> ts> occurs from trying to include 1000's of files. It doesn't matter
> ts> how large the files are --- what is important is the length of the
> ts> files' pathnames. If I allow 128 mega words (1GB) for the qsub
> ts> job, the error occurs with about 5000 80 character filenames. If
> ts> I allow 16MW or 128MB for the job, the error occurs with about
> ts> 1500 included files of about 80-character pathnames each.
>
> ts> In earlier correspondence with Paul Smith, he indicated that he
> ts> thought there may be a memory leak in read.c.
>
> ts> I have verified that the bug has not been fixed in current CVS.
>
> ts> Sure would be nice if somebody with access to purify could run it
> ts> on GNU make ...
>
> I have run the GNU make test suite with Purify on Solaris, and valgrind
> on Linux, and neither have reported any memory leaks with the latest CVS
> code.
>
> If you have a test case that can show this problem pls. let me know.
> Thanks.
I am attaching three files that should give an idea of how to re-create the
problem.
"touchempty.ksh" creates a hierarchy of 1000 directories, each containing 10
*.mk files.
The Makefile tries to include all the 10K makefiles.
The qsub script runs the make job in a batch environment with total memory for
all processes limited by the QSUB "-lM" option. To duplicate this outside of
a batch environment you may have to play with your shell's "limit" command.
The test directory in which I set this up was named "/ptmp/stern/testmake/",
21 leading characters, so to really duplicate this you should give your top
directory an absolute pathname of 21 characters or more.
Ted
--
Ted Stern Applications Group
Cray Inc. office: 206-701-2182
411 First Avenue South, Suite 600 cell: 206-383-1049
Seattle, WA 98104-2860 FAX: 206-701-2500
Frango ut patefaciam -- I break that I may reveal
(The Paleontological Society motto, equally apropos for debugging)
#!/bin/ksh
i=0
while [[ $i -lt 10 ]] ; do
j=0
while [[ $j -lt 10 ]] ; do
k=0
while [[ $k -lt 10 ]] ; do
mkdir -p AAAA_$i/BBBB_$j/CCCC_$k
m=0
while [[ $m -lt 10 ]] ; do
touch AAAA_$i/BBBB_$j/CCCC_$k/DDDD_$m.mk
m=$((m+1))
done
k=$((k+1))
done
j=$((j+1))
done
i=$((i+1))
done
SHELL := /bin/ksh
ALL_INCLUDES := $(shell find `/bin/pwd`/AAAA* -name '*.mk' -print)
include $(ALL_INCLUDES)
.PHONY: all
all:
echo "completed successfully"
#!/bin/ksh
# QSUB -lM 16Mw -lT 86400 -s /bin/ksh
# QSUB -eo -J m -me
#
cd $QSUB_WORKDIR
num_backup testmake.stdout
exec 1> testmake.stdout 2>&1
. $MODULESHOME/init/ksh
module load modules PrgEnv open open34
export PATH=$HOME/local/bin:$PATH
/ptmp/stern/make/make -R -k -d all
_______________________________________________
Bug-make mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/bug-make
