** Description changed: - I've experienced issues restoring backups when /tmp is tmpfs partition. - I've tried this on older (0.6.15) and latest duplicity (0.6.19), python - 2.4.2 (CentOS 5 default), python 2.7.3 (built from source) and it's - always using /tmp for some parts of the process while I specifically - define TMPDIR which is taken in effect somewhat. + SRU justification : - during earlier stages of restore, it doesn't use /tmp and suddenly - starts using /tmp. lsof reports the file in /tmp is deleted and I can't - actually see what's it's doing there. Here's debug (9) output of the - restore process when this happens: + Duplicity does not honor the TMPDIR, TEMP, TMP or --tempdir redirection + as expected. - Deleting /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-HUu1tG-22 - Forgetting temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-HUu1tG-22 - Processed volume 4 of 145 - Registering (mktemp) temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-yLMJml-23 - Deleting /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-yLMJml-23 - Forgetting temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-yLMJml-23 - Processed volume 5 of 145 - Registering (mktemp) temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-RmEejg-24 - Deleting /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-RmEejg-24 - Forgetting temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-RmEejg-24 - Processed volume 6 of 145 - Registering (mktemp) temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-8mpG2M-25 - Deleting /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-8mpG2M-25 - Forgetting temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-8mpG2M-25 - Processed volume 7 of 145 - Registering (mktemp) temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-0l0tGX-26 - Deleting /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-0l0tGX-26 - Forgetting temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-0l0tGX-26 - Processed volume 8 of 145 - Registering (mktemp) temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-tmyNCY-27 - Deleting /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-tmyNCY-27 - Forgetting temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-tmyNCY-27 - Processed volume 9 of 145 - Registering (mktemp) temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-dYo9xJ-28 - Deleting /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-dYo9xJ-28 - Forgetting temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-dYo9xJ-28 - Processed volume 10 of 145 - Registering (mktemp) temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-n68Y0Z-29 - Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-BLRdqu-19 - Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-TYZNJs-13 - Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-z7pTc1-11 - Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-n68Y0Z-29 - Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-YndPPI-15 - Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-fkl21c-5 - Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-HEGXbH-7 - Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-oTmsi3-3 - Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-Ky2n3b-18 - Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-UgfduZ-10 - Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-YH3N28-17 - Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-ou7sde-16 - Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-AAQhlE-14 - Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-nvni05-6 - Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mkstemp-H5WCdY-1 - Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-eUCf1V-12 - Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-OlulTA-8 - Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-O9HkIO-9 - Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-sZj6Lb-4 - Traceback (most recent call last): - File "/usr/local/bin/duplicity", line 1391, in <module> - with_tempdir(main) - File "/usr/local/bin/duplicity", line 1384, in with_tempdir - fn() - File "/usr/local/bin/duplicity", line 1318, in main - restore(col_stats) - File "/usr/local/bin/duplicity", line 623, in restore - restore_get_patched_rop_iter(col_stats)): - File "/usr/local/lib/python2.7/site-packages/duplicity/patchdir.py", line 522, in Write_ROPaths - for ropath in rop_iter: - File "/usr/local/lib/python2.7/site-packages/duplicity/patchdir.py", line 495, in integrate_patch_iters - final_ropath = patch_seq2ropath( normalize_ps( patch_seq ) ) - File "/usr/local/lib/python2.7/site-packages/duplicity/patchdir.py", line 475, in patch_seq2ropath - misc.copyfileobj( current_file, tempfp ) - File "/usr/local/lib/python2.7/site-packages/duplicity/misc.py", line 170, in copyfileobj - outfp.write(buf) - IOError: [Errno 28] No space left on device + Impact : - I first thought this was something in python itself defaulting to /tmp, - but it also happened on newest python too so I'm fairly sure duplicity - fails here. I'm only starting learning python so can't debug more/write - patches so I depend on you guys. + If the available size in /tmp is too small, restore may fail to complete - lsof output looks like this: - duplicity 13755 root 3r REG 0,21 104835547 25612 /duplybackup/self/duplicity-full.20120509T115438Z.vol63.difftar.gz (x:/backup/node/vm) - duplicity 13755 root 4r REG 8,1 60151379 20433546 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-ZwHDoQ-3 - duplicity 13755 root 5r REG 8,1 78092697 20433547 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-otjk9B-4 - duplicity 13755 root 6r REG 8,1 51332868 20433548 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-Li5Lea-5 - duplicity 13755 root 7r REG 8,1 56067864 20433549 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-b1RRyo-6 - duplicity 13755 root 8r REG 8,1 80891540 20433550 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-6F880l-7 - duplicity 13755 root 9w REG 8,1 104833635 20433551 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-WAoXTy-8 - duplicity 13755 root 10w REG 8,1 71777882 20433552 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-ipj0ni-9 - duplicity 13755 root 11r REG 8,1 48588498 20433553 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-SDiDK_-10 - duplicity 13755 root 12r REG 8,1 38507998 20433554 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-RDECXu-11 - duplicity 13755 root 13r REG 8,1 41653195 20433555 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-isRCM4-12 - duplicity 13755 root 14r REG 8,1 45080364 20433556 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-UlTLTG-13 - duplicity 13755 root 15w REG 8,1 95725089 20433557 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-GrIAel-14 - duplicity 13755 root 16r REG 8,1 87851979 20433558 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-zTQNfz-15 - duplicity 13755 root 17w REG 8,1 104835141 20433559 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-mvtG4c-16 - duplicity 13755 root 18w REG 8,1 67628505 20433560 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-i0XhUm-17 - duplicity 13755 root 19w REG 8,1 63160613 20433561 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-hnjypn-18 - duplicity 13755 root 20w REG 8,1 29449321 20433562 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-zNErBv-19 - duplicity 13755 root 21w REG 7,1 4665376768 48959 /tmp/tmpflxctko (deleted) - duplicity 13755 root 22w REG 8,1 46923776 20433545 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-zHKiCL-29 + Fix : + + Backport upstream fix applied in 0.6.21 (Merge proposal from the + upstream task of this bug) + + Test Case : + Note: Incremental backup is necessary so both the full & difftar are required in /tmp which will require more than 50Mb + + 1) Mount a 50Mb file system under /tmp + ~# df -h /tmp + Filesystem Size Used Avail Use% Mounted on + /dev/sda1 48M 794K 45M 2% /tmp + + 2) Create a 60Mb file to be backed up under /srv called data + dd if=/proc/kcore of=/srv/data bs=1M count=60 + + 3) Do a full backup of /srv into /backup + + duplicity full --name test --encrypt-key A6C785C2 --sign-key A6C785C2 + --volsize 25 /srv file:///backup/duply + + 4) Modify the 60Mb file so it can be picked up by the incremental backup + dd if=/proc/kcore of=/srv/data bs=1M count=10 conv=notrunc oflag=append + + 5) Do an incremental backup + duplicity incr --name 'duply_test' --encrypt-key A6C785C2 --sign-key A6C785C2 --volsize 25 /srv file:///backup/duply + + 6) Restore the flag using TMPDIR : + TEMPDIR=/mytemp duplicity --name 'duply_test' --encrypt-key A6C785C2 --sign-key A6C785C2 --verbosity '4' --volsize 25 -t now file:///backup/duply /restore + + With the patch, the command will succeed. + + Regression : + + None expected, this code is used in the version currently available in + Raring + + Description of the problem : + + When /tmp is too small, duplicity is sometimes unable to do a restore. + Using TMPDIR variable or --tempdir doe not work.
-- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1005901 Title: cannot change temp dir To manage notifications about this bug go to: https://bugs.launchpad.net/duplicity/+bug/1005901/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
