The issue is not symbolic links per se, but ones that form loops. Note that you can detect such loops by running 'find -L ...' and looking for the error messages. (find by default does not follow any symbolic links, which can be a problem also.)
It is a shortcoming of the current version of list.files(). Bill Dunlap Spotfire, TIBCO Software wdunlap tibco.com > -----Original Message----- > From: jgrn...@gmail.com [mailto:jgrn...@gmail.com] On Behalf Of Jonathan > Greenberg > Sent: Saturday, September 28, 2013 10:51 AM > To: William Dunlap > Cc: r-help > Subject: Re: [R] Error: C stack usage is too close to the limit when using > list.files() > > Thanks all -- ok, so the symbolic link issue is a distinct > possibility, but fundamentally doesn't solve the issue since most > users will have symbolic links on their machines SOMEPLACE, so a full > drive scan will run into these issues -- is list.files calling find, > or is it using a different algorithm? This seems like a shortcoming > in the list.files algorithm -- is there a better solution (short of a > System call, which I'm still not sure will work on Macs without Xcode > -- a colleague of mine did NOT have Xcode, and reported not being able > to run find from the command line) -- perhaps a different package? > > --j > > On Fri, Sep 27, 2013 at 3:08 PM, William Dunlap <wdun...@tibco.com> wrote: > > Toss a couple of extra files in there and you will see the output grow > > exponentially. > > > > % touch dir/IMPORTANT_1 dir/subdir/IMPORTANT_2 > > > > and in R those two new files cause 82 more strings to appear in list.file's > > output: > > > >> nchar(list.files("dir", recursive=TRUE)) > > [1] 11 18 33 40 55 62 77 84 99 106 121 128 143 150 165 172 187 > > 194 209 > > [20] 216 231 238 253 260 275 282 297 304 319 326 341 348 363 370 385 392 > > 407 414 > > [39] 429 436 451 458 473 480 495 502 517 524 539 546 561 568 583 590 605 > > 612 627 > > [58] 634 649 656 671 678 693 700 715 722 737 744 759 766 781 788 803 810 > > 825 832 > > [77] 847 854 869 876 891 898 901 > > > > 'find', by default, does not following symbolic links. > > > > % find dir > > dir > > dir/subdir > > dir/subdir/IMPORTANT_2 > > dir/subdir/linkToUpperDir > > dir/IMPORTANT_1 > > > > The -L option makes it follow them, but it won't follow loops: > > > > % find -L dir > > dir > > dir/subdir > > dir/subdir/IMPORTANT_2 > > find: File system loop detected; `dir/subdir/linkToUpperDir' is part of the > > same file > system loop as `dir'. > > dir/IMPORTANT_1 > > > > Bill Dunlap > > Spotfire, TIBCO Software > > wdunlap tibco.com > > > > > >> -----Original Message----- > >> From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On > Behalf > >> Of William Dunlap > >> Sent: Friday, September 27, 2013 12:56 PM > >> To: Jonathan Greenberg; r-help > >> Subject: Re: [R] Error: C stack usage is too close to the limit when using > >> list.files() > >> > >> Do you have some symbolic links that make loops in your file system? > >> list.files() has problems with such loops and find does not. E.g., on a > >> Linux box: > >> > >> % cd /tmp > >> % mkdir dir dir/subdir > >> % cd dir/subdir > >> % ln -s ../../dir linkToUpperDir > >> % cd /tmp > >> % R --quiet > >> > list.files("dir", recursive=TRUE, full=TRUE) > >> [1] > >> > "dir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToU > >> > pperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkT > >> > oUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/li > >> > nkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdi > >> > r/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/su > >> > bdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir > >> > /subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpper > >> > Dir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUp > >> > perDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkTo > >> > UpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/lin > >> kToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir" > >> > system("find dir") > >> dir > >> dir/subdir > >> dir/subdir/linkToUpperDir > >> > >> Bill Dunlap > >> Spotfire, TIBCO Software > >> wdunlap tibco.com > >> > >> > >> > -----Original Message----- > >> > From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] > >> > On > Behalf > >> > Of Jonathan Greenberg > >> > Sent: Friday, September 27, 2013 12:13 PM > >> > To: r-help > >> > Subject: [R] Error: C stack usage is too close to the limit when using > >> > list.files() > >> > > >> > R-helpers: > >> > > >> > I'm running a file search on my entire drive (Mac OS X) using: > >> > > >> > files_found <- > list.files(dir="/",pattern=somepattern,recursive=TRUE,full.names=TRUE) > >> > where somepattern is a search pattern (which I have confirmed via a > >> > unix "find / -name somepattern" only returns ~ 3 results). > >> > > >> > I keep getting an error: > >> > > >> > Error: C stack usage is too close to the limit > >> > > >> > when running this command. Any ideas on 1) how to fix this or 2) if > >> > there is an alternative to using list.files() to accomplish this > >> > search without resorting to an external package? > >> > > >> > Cheers! > >> > > >> > --jonathan > >> > > >> > > >> > -- > >> > Jonathan A. Greenberg, PhD > >> > Assistant Professor > >> > Global Environmental Analysis and Remote Sensing (GEARS) Laboratory > >> > Department of Geography and Geographic Information Science > >> > University of Illinois at Urbana-Champaign > >> > 259 Computing Applications Building, MC-150 > >> > 605 East Springfield Avenue > >> > Champaign, IL 61820-6371 > >> > Phone: 217-300-1924 > >> > http://www.geog.illinois.edu/~jgrn/ > >> > AIM: jgrn307, MSN: jgrn...@hotmail.com, Gchat: jgrn307, Skype: jgrn3007 > >> > > >> > ______________________________________________ > >> > R-help@r-project.org mailing list > >> > https://stat.ethz.ch/mailman/listinfo/r-help > >> > PLEASE do read the posting guide > >> > http://www.R-project.org/posting-guide.html > >> > and provide commented, minimal, self-contained, reproducible code. > >> > >> ______________________________________________ > >> R-help@r-project.org mailing list > >> https://stat.ethz.ch/mailman/listinfo/r-help > >> PLEASE do read the posting guide > >> http://www.R-project.org/posting-guide.html > >> and provide commented, minimal, self-contained, reproducible code. > > > > -- > Jonathan A. Greenberg, PhD > Assistant Professor > Global Environmental Analysis and Remote Sensing (GEARS) Laboratory > Department of Geography and Geographic Information Science > University of Illinois at Urbana-Champaign > 259 Computing Applications Building, MC-150 > 605 East Springfield Avenue > Champaign, IL 61820-6371 > Phone: 217-300-1924 > http://www.geog.illinois.edu/~jgrn/ > AIM: jgrn307, MSN: jgrn...@hotmail.com, Gchat: jgrn307, Skype: jgrn3007 ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
