Re: [Discuss] Deduplication

2024-09-06 Thread Kent Borg
On 9/5/24 18:53, Dale R. Worley wrote: It looks like duperemove works only on some filesystems because it makes all the files reflink-ed, they still have separate inodes but share the data extents as long as you don't modify any of them. But rdfind can create hardlinks or symlinks, which are sup

Re: [Discuss] Deduplication

2024-09-05 Thread Dale R. Worley
Kinda hilarious to see: discuss-requ...@driftwood.blu.org writes: > Today's Topics: > >1. Re: Grub, EFI, Partitioning? (David Rosenstrauch) >2. Deduplication (Kent Borg) >3. Re: Deduplication (Rich Pieri) >4. Re: Deduplication (Rich Pieri) >5. Re: Deduplication (Dan Ritter) It

Re: [Discuss] Deduplication

2024-09-05 Thread Dan Ritter
Kent Borg wrote: > So today I ran "duperemove" on a couple volumes, and it scared up some > non-trivial space. I decided to run it on a third volume. > > Nope! It works by telling the kernel to make files that match to share the > same extents, but that only works for some file systems. > > - XF

Re: [Discuss] Deduplication

2024-09-05 Thread Rich Pieri
On Thu, 5 Sep 2024 09:32:09 -0400 Rich Pieri wrote: > Aside: The ext# family don't have CoW capability so dupremove can't > work on them. Aside 2: Compression is typically a much better value than dedup at the small end. -- \m/ (--) \m/ ___ Discuss m

Re: [Discuss] Deduplication

2024-09-05 Thread Rich Pieri
I think deduplication is kind of overrated and impractical. As was pointed out several times in the EFI thread: big, fast drives are cheap. So what if there are two or three copies of a file on a backup set? The dedup overhead is more costly than the storage. Where deduplication starts becoming pr

[Discuss] Deduplication

2024-09-04 Thread Kent Borg
For many years now I've been good about keeping off line backups on (encrypted) external disks. I have been backing up my daily computer(s) over several generations of said computers. Which means I manage to put large amounts of data on big disks the modern way: by collecting and storing duplic