On 1/16/24 01:18, Felix Miata wrote:
Felix Miata composed on 2024-01-16 01:05 (UTC-0500):
gene heskett composed on 2024-01-15 18:37 (UTC-0500):
Ah,but I finally glombed onto the bug tan memory bar in htop as it was
runniing, someplace in the data chain is a huge memory leak, my crash is
caused by the OOM daemon killing things. And it only occurs when I run
rsync. Only takes it 10 minute to eat 32G of memory, then 500k into
swap, and the OOM daemon start killing the system until there's nothing
left to run.
What does free report before starting rsync? Do you have all your swap on a
partition? Do you have any swapspace?
I would log out of XFCE, login on a vtty to open top, then login on another to
try
to run rsync. If that fails OOM too, since the target is ostensibly starting
from
scratch, use MC, and divide the job into the source's directories if necessary.
MC
gets rather bogged down if you try to do a bazillion individual files in a
single
copy operation.
Trying to think outside the box, something else to think about, from the man
page:
[quote]
--archive, -a
This is equivalent to -rlptgoD. It is a quick way of saying you want recursion
and want to preserve almost everything. Be aware that it does not include
preserving ACLs (-A), xattrs (-X), atimes (-U), crtimes (-N), nor the finding
and
preserving of hardlinks (-H).
[/quote]
If rsync really is bugged, maybe a change of options would avoid the bug. Try
instead of -av, -rlptgoDAXUNH. Could it be that verbosity is the OOM crippler,
and
not necessarily from rsync itself, but possibly from the xterm in which rsync is
running? Does your source contain any hard links? Do you use ACLs or xattrs?
unreported here because it didn't seem to have any effect, I've tried to
test that theory by clearing the back-trace buffer at 30 second
intervals. no obviously detectable effect, untested is setting that back
to a 1000 line default.
And since I've driven around 170 miles in poor visibility bad weather
today, no more tests will be done tonight, I'm not the 16 years old I
was when I learned to drive 70 mph on even worse roads 75 years ago. So
I'll sign off shortly.
Cheers, Gene Heskett.
--
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
- Louis D. Brandeis
