This is my first email to this list, so don't hesitate to correct me when I violate the local etiquette.
I am attempting to use find in a way for which I can find little prior art. I have a huge file structure on a 3 terabyte hard drive. I want to generate a single file that lists all the plane files, directory files, and symlinks in the whole structure. I don't care how long it takes, but I do want it to terminate without error. When I run it, I can monitor memory usage using Gnome-system-monitor and I can see the memory usage slowly rising. I am using Debian Wheezy on an oldish HP desktop tower with Pentium dual core CPU. I have two of these 3terabyte disks and several 500gigabyte disk, all with USB2 interfaces. My project is an attempt at a massive at home data de-duplication of backup data structures that have accumulated over time. Are there difficulties with coreutils on these big disks? My program is a script written in bash. It runs for about an hour and quits without any error message that I can find, at a point where it is deep in a very deep tree traversal, and as I say. with memory usage growing. Some tips on gathering debug data would be much appreciated. The script follows: <begin> #!/bin/bash this=bld-spcl # for use in grand cleanup on disk, gfx2 starting 20140407,pec # # ############################### . /root/bin/arx-declares export arxiv="$PWD" find glbl -warn -depth -mindepth 1 \( \ \( -type f -empty -printf "mdx f e hst etm ${fndnfmt}\n" \) -o \ \( -type f -printf "mdx f f hst etm ${fndnfmt}\n" \) -o \ \( -type d -printf "mdx f d hst etm ${fndnfmt}\n" \) -o \ \( -type l -printf "mdx f l hst etm ${fndnfmt} -> %l\n" \) \) |& \ sed 's|\.\([0-9]\)\{10\} glbl| glbl|' > "$arxiv/find.out_$('date' $TIME_STYLE)" 2>&1 exit ##################################### # mnemonic field names for $fndnfmt: # 1 2 3 4 5 6 7 8 # mdx fqx dfl hst etm nod mod lnx usr grp siz mtm fqnm # 1 2 3 4 5 6 7 8 9 10 11 12 13 ########################################## <eof> Relavant declarations from arx-declares are: export TIME_STYLE='+%Y%m%d_%H%M%S' export fndnfmt="%i %M %n %U %G %s %TY%Tm%Td_%TH%TM%TS %p" The directory, glbl, is the root of the file tree that I am trying to exhaustively search. The complication with sed is to remove the subsecond decimal fractions from the output. It works correctly for a few million times, before the time of the crash. The fixed data are placeholders to data that will be filled in by future processing of this output file, if/when I get this script working. Want more information? Ask, please. -- Paul E Condon pecon...@mesanetworks.net