On Thu, Jul 29, 2021 at 3:22 AM Simon Josefsson <si...@josefsson.org> wrote: > > Jim Meyering <j...@meyering.net> writes: > > > On Wed, Jul 28, 2021 at 1:08 AM Simon Josefsson via Gnulib discussion > > list <bug-gnulib@gnu.org> wrote: > >> Hi. I replaced GNU InetUtils' old custom fts implementation with the > >> one from gnulib, but self-tests started failing. Looking at the code, > >> it seems gnulib's fts implementation has diverged compared to glibc, and > >> has some optimizations that (I think) change the API (wrt stat and > >> chdir). Also, gnulib's fts module is always enabled, even on modern > >> glibc systems. InetUtils's usage of fts works fine with modern glibc, > >> but it didn't work with gnulib's version (it needed a FTS_NOCHDIR). The > >> gnulib manual for the fts replacement module isn't terribly clear about > >> this. Is there a reason for this behaviour? > >> > >> I would prefer if there were two fts modules in gnulib: > >> > >> 1) One module 'fts' based on glibc's code, that is only enabled in on > >> systems that doesn't have fts, or where fts is known to be buggy. > >> > >> 2) One 'fts-faster' that is the current code, which can be used when you > >> always wants to pull in the optimized implementation. > >> > >> Then InetUtils would use system fts on glibc platforms, and not always > >> have to pull in a replacement copy. > >> > >> What do you think? > >> > >> I could live with a new module 'fts-optional' that only pulls in the > >> current 'fts' module when the system is lacking it. That doesn't fix > >> the API confusion, but is probably sufficient for InetUtils. > > > > There are fundamental flaws in the ABI of glibc's fts that make it > > unsuitable for use in any tool I care about. > > Ouch -- is there disagreement from the glibc people on fixing glibc fts? > Maybe the reaction will be different now.
It's intrinsic in the current ABI. There would have to be a new interface. I chatted with Uli about this many years ago, and he would have been happy to receive a patch adding the new interfaces, but gnulib's fts solved all of my problems, so I never made time to add this to glibc. > Are the problems inherent with the glibc ABI, or can they be fixed? If > it isn't possible to fix, maybe the entire API should be declared > deprecated and eventually removed. > > The glibc manual doesn't seem to document fts?! > > > Those flaws make it easy to hit silly limits or to provoke inordinate > > resource usage/DoS. > > It would be nice to document these problems in more detail in the gnulib > manual. > > Is there any known behavioural difference between glibc fts and gnulib > fts? Documenting that too would be useful. Yes, see each item on the list quoted below. I think I can dig up some documentation on gnulib's fts. > InetUtils' required a FTS_NOCHDIR flag in order to continue behave as > before (a simple command like 'ls foo' where foo is a directory failed). > I don't see any self-tests in gnulib without that flag, so maybe this > suggests there is some API/ABI difference. > > > Is it ok for InetUtil's fts to be unable to do these things? (each of > > which afflicts glibc fts, from what I recall) > > - process files in a tree more than 2^16 levels deep > > - detect certain cycles efficiently > > - delete (in reasonable time) a hierarchy with too many entries in a > > single directory. > > InetUtils only uses fts for "DIR" in ftpd, when it emulate /bin/ls > internally (based on some old BSD implementation of /bin/ls that uses > fts). The first two applies, but not the third, I think, but it sounds > like corner-cases. They are corner cases, indeed. Correcting that last item: s/delete/traverse/: - traverse (in reasonable time) a hierarchy with too many entries in a single directory.
