On Sun, 2019-10-20 at 05:21 -0400, Joshua Kinard wrote: > On 10/20/2019 04:32, Michał Górny wrote: > > On Sun, 2019-10-20 at 04:25 -0400, Joshua Kinard wrote: > > > Why is having a max ~24k files in a directory a bad idea? Modern > > > filesystems are more than capable of handling that. > > > > > > - ext4: unlimited files in a directory > > > - xfs: virtually unlimited (hard limit of 2^64-1 total files per volume) > > > - ntfs: 4,294,967,295 > > > > > > And 24k is a bit more than 1/3rd of all distfiles that we currently have. > > > > For the same reason having ~60k files in a directory was a problem. > > There is really no point in changing anything if you change BIG_NUMBER > > to SMALLER_BIG_NUMBER. > > That doesn't answer my question. Why is it a problem? What criteria are > you using to decide that 24k is a "smaller big number"? Is there some issue > highlighted by the mirror admins where having 24k files in a single > directory offers no significant relief versus the current 60k files?
IIRC Robin set the goal as: | the number of files in a single directory should not exceed 1000, [1] I don't recall how that number was chosen but it's probably pretty arbitrary. In any case, I can notice the difference between working with a listing of 1k files and 24k files, on the hardware running masterdist. > > > Under which scenario do you wind up with 24k files in a single directory? > > > I > > > consider the tex package an outlier in this case (one package should not > > > be > > > the sole dictator of policy). > > > > Three versions of TeXLive living simultaneously. If one package falls > > completely out of bounds, no problem is solved by the change, so what's > > the point of making it? > > The problem in this case is with texlive, not our current, or future, > distfiles methodology. Is it? Are you suggesting we should ban upstream from using multiple distfiles with similar prefix? What about other potential packages that may suffer from the same problem in the future? Go packages have a good potential, given that majority of them starts with 'github.com'. > Has anyone looked at how other distros deal with texlive? Other distros don't mirror original distfiles. > Has anyone complained or filed a bug to texlive developers > upstream about their excessive amount of distfiles and the burden it places > on distro maintainers? You believe it to be a problem. Don't expect others to bother upstream with your preferences. [1] https://www.gentoo.org/glep/glep-0075.html#algorithm-for-splitting-distfiles > -- Best regards, Michał Górny
signature.asc
Description: This is a digitally signed message part
