[email protected] wrote, On 21.11.2013 16:58: > Zitat von Hans Thueminger <[email protected]>: > >> [email protected] wrote, On 21.11.2013 10:07: >>> [...] >>> With up to Windows 2008 R2 the supported volume size was 16TB with >>> Windows 2012 it is 64TB. Note that their are other constraints with >>> VSS when used with volumes containing many files or having heavy load >>> while doing snapshots. That said you should always be able to backup >>> without VSS, but open files get you in trouble in this case. >> I didn't find a way to make a backup without VSS, neither with the >> in the operating system included Windows-Server-Backup program nor >> with Bacula. But while writing this sentence I had an idea: What is >> happening, if backing up a mount point instead of a drive letter? >> Now I've just mounted the 120TB Filesystems as a mount point in C: >> (which is a 300GB filesystem) and look at here: >> >> 21-Nov 12:49 bacula-sd JobId 292: Spooling data ... >> 21-Nov 12:49 fs2-fd JobId 292: Generate VSS snapshots. Driver="Win64 >> VSS", Drive(s)="C" >> >> and the status says: >> >> JobId 292 Job fs2-PHOTO.2013-11-21_12.46.00_11 is running. >> VSS Full Backup Job started: 21-Nov-13 12:47 >> Files=31,223 Bytes=308,963,855,104 Bytes/sec=89,192,798 Errors=0 >> Files Examined=31,223 >> Processing file: >> C:/PHOTO/Projects/PHOTO-Projects/09_ALS_Kaernten/GailtalLatschur/Dif-Gailtal-Latschur/s577_s592_p02.tif >> >> Projects is the mountpoint for the 120TB filesystem! >> >> By now trying to backup G:/PHOTO-Projects/ (which is the same 120TB >> filesyste as above), I always received the following error: >> >> 17-Sep 15:54 fs2-fd JobId 36: Generate VSS snapshots. Driver="Win64 >> VSS", Drive(s)="G" >> 17-Sep 15:55 fs2-fd JobId 36: Fatal error: CreateSGenerate VSS >> snapshots failed. ERR=The operation completed successfully. >> >> It seems, that this is a way to trick bacula and windows :-) Of >> course with this workaround we still have the problems with open >> files, but the actual problem which I want to discuss with you is >> the error I receive after creating the second or third or sometimes >> subsequent spoolfile: > Uhm, no. The idea was to set "enable vss = no", but as said this only > works some sort of if the volume in question does not have open files. Ok, now it's clear why it makes no difference to set Enable VSS=yes or no: There are always open files on these filesystems (they are already in use)...
> >>>> Now I'm glad to make backups and restores of files of our 4x40TB >>>> Filesystems with Bacula if they are not too big (< 5TB). That works >>>> fine. If they are too big (> 15TB) I always get an error after >>>> creating the second or third or sometimes subsequent spoolfile >>>> (Error: lib/bsock.c...). Never for the first spoolfile! I've tried >>>> several spool sizes (from 500GB to 16TB) and different network >>>> settings. As attachment (bacula_mailing_list_some_error_logs.txt) >>>> you can find some logs, when the error occurred. What I have also >>>> tried: >>>> - using different Networkinterfaces (at the moment an ethernet cable >>>> is directly connected (no switch) between the fileserver and the >>>> backupserver and this connection is used for the backups (checked >>>> with netstat)) >>>> - heartbeat: enabling on the SD (60 seconds) and >>>> net.ipv4.tcp_keepalive_time also set to 60 >>>> - AllowCompression Yes/No >>>> - many, many hours for trials >>> So you really have files with 15TB in size? What would be worth a try >>> is the following: >> Sorry, that was badly written of me. It's not the size of one file, >> it's the size of all files. So what I wanted to write was: "if the >> amount of files to be backuped is not too large (< 5TB) it works. If >> the amount of the files to be backuped ist larger than 15TB it fails >> always!" >> > This might point to another problem case. We had a similar problem on > our main filer with backup until around 2TB succeeded, anything above > had a ~50% to fail with network errors. We switched the NIC and use > some Intel PlugIn card and the problem went away. I thought I can exclude such hardware problems, because I have already tried different network interfaces. But good to know, that such an error can occur from a NIC too! > >>> - Increase the "maximum file size" for the tape drive. The default is >>> 1G and it limits how big the "blocks" on tape are between EOF markers. >>> Maybe the counter per file is an int and you therefore have trouble >>> with 15TB files? >> I guess the spool file is written as one file to the tape which >> would mean, that for every spool file only one EOF marker would be >> written? Can you confirm that, or are I'm wrong? What would you >> suggest to set for "maximum file size"? > No, the spool file is only to decouple the streaming to tape from > network and client speed and delays. There is not much buffering when > writing to tape so the spool area need to be able to constantly > deliever data faster tahn the tape can consume. It is still written > with the "maximum block size" per transfer and a EOF marker every > "maximum file size" to tape. > >>> - You should increase the default block size written to tape with >>> "maximum block size" set to for example 2M. Warning: You could not >>> read already written tapes with non matching block sizes. >> Ok and thank you for the warning, that's rather good to know! >> >>> - The spool area doesn't need to be that big, but really fast to >>> saturate the tape drive and keep them streaming. Recommended is >>> something like fast SSD or similar. >> The tapes we are using have a native capacity of 4TB. I thought that >> should be the lowest size of the spool size to prevent start and >> stop operations of the drives. With the hardware compression >> sometimes more than 9TB are written on a tape, so I decided the >> ideal spool size is about 16TB?! So where is my error in reasoning? >> > There is no problem in cutting the job in pieces of some GB with the > tape being idle in between. The problem arise if you can deliever > data, but not fast enough. So the tape drive try to reach its max. > speed try to settle down if data are to slow and stop and rewind if > nothing helps. To prevent this you need data chunks delieverable with > (much) more than the tape can handle, so for this chunk you never has > a "underun" of data. IMHO a smaller faster spool is always better than > a bigger slower spool. Thank you for this explantation. Than we will reconstruct the backupserver and remove some of the HDs and put som SSDs in the Server... Regards Hans ------------------------------------------------------------------------------ Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk _______________________________________________ Bacula-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/bacula-users
