Brian Inglis via Cygwin wrote:
On 08/04/2025 19:05, Mark Liam Brown via Cygwin wrote:
Are there tuning variables to improve ls, ls -l, find ., find . -ls
performance for very large dirs?
If we have a SMB dir with 60000+ entries a simple ls -l can take MANY
minutes (22+mins), while cmd.exe dir just floods the terminal with
results immediately.
I just remembered noticing my /var/log/ entries jumped looking at my
last Cygwin upgrade log, so before I clean it up into subdirs, let's
try some stuff!
If you just want a directory name dump, use \ls -U|--sort=none or \ls
-f which is like -aU; command prefix \ overrides the usual aliases
like -CF --color=auto:
$ time \ls -1U /var/log/ | \wc -lwmcL
54049 54049 1885787 1885787 38
real 0m0.336s
user 0m0.046s
sys 0m0.295s
$ time \ls -1f /var/log/ | \wc -lwmcL
54051 54051 1885792 1885792 38
real 0m0.347s
user 0m0.140s
sys 0m0.232s
even with -l, it's only a few seconds:
$ time \ls -lU /var/log/ | \wc -lwmcL
54050 486455 4966716 4966716 102
real 0m15.891s
user 0m1.421s
sys 0m14.560s
$ time \ls -lf /var/log/ | \wc -lwmcL
54052 486473 4966835 4966835 102
real 0m15.858s
user 0m1.405s
sys 0m14.374s
One reason for the above difference is possibly that Cygwin supports the
'dirent.d_type' field for quite some time and tools like 'ls' and 'find'
use it to avoid stat() calls.
Fast - stat() calls avoided:
ls -R
find
find -type f -name 'some*file'
Fast if only few entries require stat():
find -type f -name 'some*file' -mtime +42 # only -mtime requires stat()
find -mtime +42 -type f -name 'some*file' # find does 'cheap' checks
first
Slow - requires stat() for each entry:
ls -Rt
ls -RS
ls -Rl
find -mtime +42
find -ls
--
Regards,
Christian
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation: https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
