Hi,
In a previous post, I indicated that I was working on a plugin for Bareos
Storage Daemon. During my tests, I noted that bsdEventWriteRecordTranslation is
not emitted during a Virtual Full Backup. Have you got a patch to resolv this
problem ?
For information, I published a first alpha release of the plugin on github :
https://github.com/Sherlock221B/bareos/blob/master/src/plugins/stored/dedup-sd.c
I did some tests (backup of 30Go and 400Go) and the first results are
interesting.
The plugin allows to deduplicate the data using block of fixed size (64k) or of
variable size (avg 8k).
Varaiable size need more cpu and memory, but gives a better result.
The plugin also allows to deduplicate each volume (only the volumes whose name
begins with a prefix defines) or a group of volumes (all the data stored on a
set of volumes are deduplicated and stored in the same DB).
For best results, it is preferable that a volume is used only for a job. In my
case, the name of a volume is the name of the job to which I add the prefix for
deduplication.
You will find in attach a small example of the settings I use.
Best regards.
--
You received this message because you are subscribed to the Google Groups
"bareos-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
For more options, visit https://groups.google.com/d/optout.
########
# Pool #
########
Pool {
Name = DedupFull
Pool Type = Backup
Recycle = yes # Bareos can automatically recycle Volumes
AutoPrune = yes # Prune expired volumes
Volume Retention = 365 days # How long should the Full Backups be
kept? (#06)
Maximum Volume Bytes = 50G # Limit Volume size to something
reasonable
Maximum Volumes = 100 # Limit number of Volumes in Pool
Label Format = "DedupFull-$Job" # Volumes will be labeled
"DedupFull-<job-name>"
}
Pool {
Name = DedupDifferential
Pool Type = Backup
Recycle = yes # Bareos can automatically
recycle Volumes
AutoPrune = yes # Prune expired volumes
Volume Retention = 90 days # How long should the
Differential Backups be kept? (#09)
Maximum Volume Bytes = 10G # Limit Volume size to something
reasonable
Maximum Volumes = 100 # Limit number of Volumes in Pool
Label Format = "DedupDifferential-$Job" # Volumes will be labeled
"Differential-<job-name>"
}
Pool {
Name = DedupIncremental
Pool Type = Backup
Recycle = yes # Bareos can automatically recycle
Volumes
AutoPrune = yes # Prune expired volumes
Volume Retention = 30 days # How long should the Incremental
Backups be kept? (#12)
Maximum Volume Bytes = 1G # Limit Volume size to something
reasonable
Maximum Volumes = 100 # Limit number of Volumes in Pool
Label Format = "DedupIncremental-$Job" # Volumes will be labeled
"Incremental-<job-name>"
}
################################
# JobDefs to dedup each volume #
################################
JobDefs {
Name = "DedupJob"
Type = Backup
Level = Incremental
Client = debian-fd
FileSet = "debian" # debian fileset
Storage = DedupFile
Messages = Standard
Pool = DedupIncremental
Priority = 10
Write Bootstrap = "/opt/bareos/var/lib/%c.bsr"
Full Backup Pool = DedupFull # write Full Backups into
"DedupFull" Pool
Differential Backup Pool = DedupDifferential # write Diff Backups into
"DedupDifferential" Pool
Incremental Backup Pool = DedupIncremental # write Incr Backups into
"DedupIncremental" Pool
SD Plugin Options =
"dedup:path=/storage/dedup/:prefix=DedupFull-|DedupDifferential-|DedupIncremental-"
}
#####################################################################
# Files stored on the server after a full and an incremental backup #
#####################################################################
root@bareosdev01:/opt/bareos# ls -R /storage/dedup/
/storage/dedup/:
DedupFull-backup-debian-fd DedupIncremental-backup-debian-fd
/storage/dedup/DedupFull-backup-debian-fd:
DedupFull-backup-debian-fd-0000001.bdb DedupFull-backup-debian-fd-0000001.ddb
DedupFull-backup-debian-fd-0000001.maf DedupFull-backup-debian-fd.cdb
/storage/dedup/DedupIncremental-backup-debian-fd:
DedupIncremental-backup-debian-fd-0000001.bdb
DedupIncremental-backup-debian-fd-0000001.maf
DedupIncremental-backup-debian-fd-0000001.ddb
DedupIncremental-backup-debian-fd.cdb
## Other example ##
#######################################
# JobDefs to dedup a group of volumes #
#######################################
JobDefs {
Name = "DedupJob"
Type = Backup
Level = Incremental
Client = debian-fd
FileSet = "debian" # debian fileset
Storage = DedupFile
Messages = Standard
Pool = DedupIncremental
Priority = 10
Write Bootstrap = "/opt/bareos/var/lib/%c.bsr"
Full Backup Pool = DedupFull # write Full Backups into
"DedupFull" Pool
Differential Backup Pool = DedupDifferential # write Diff Backups into
"DedupDifferential" Pool
Incremental Backup Pool = DedupIncremental # write Incr Backups into
"DedupIncremental" Pool
SD Plugin Options =
"dedup:path=/storage/dedup/:prefix=DedupFull-|DedupDifferential-|DedupIncremental-:group_prefix=DedupFull-|DedupDifferential-|DedupIncremental-"
}
#####################################################################
# Files stored on the server after a full and an incremental backup #
#####################################################################
root@bareosdev01:/opt/bareos# ls -R /storage/dedup/
/storage/dedup/:
backup-debian-fd
/storage/dedup/backup-debian-fd:
backup-debian-fd-0000001.bdb backup-debian-fd-0000001.ddb
backup-debian-fd-0000001.maf backup-debian-fd-0000002.maf backup-debian-fd.cdb
