On 2024-10-08, Andy Smith wrote: > When you have hundreds of millions of files in rsnapshot it really > starts to hurt because every backup run involves: > > - Deleting the oldest tree of files;
rsnapshot can rename it apart and delete it after backup is done. Thus involving only the backup system > - Walking the entire tree of the most recent backup once to cp -l it and > then; rsnapshot only renames directories when rotating backups then does rsync with hard links to the newest > This rsnapshot I have is really quite slow with only two 7200rpm HDDs. > It spends way longer walking its data store than actually backing up any > data. I could definitely make it speedier by switching to something > else. But I like rsnapshot for this particular case. On 7200rpm HDDs I was using xfs over RAID1 and the slowest/blocking part was the deletion > Although it probably matters most how many files you have only in the > most recent backup iteration rather than the entire rsnapshot store. For > me that is approx 5.8 million. I don't remember but I should have been around your volume. rsync uses metadata so it also depends on the filesystem. Some are quicker. I think metadata is quite like the index used by other backup systems.
