[
https://issues.apache.org/jira/browse/SOLR-13608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17268005#comment-17268005
]
Jason Gerlowski commented on SOLR-13608:
----------------------------------------
I'm going to take on this issue as a part of the
[SIP-12|https://cwiki.apache.org/confluence/display/SOLR/SIP-12%3A+Incremental+Backup+and+Restore]
work. This changes some of the specifics that Dat wrote up in the description
above to align better with the agreed-upon SIP-12 design. I'll correct these
shortly.
> Incremental backup for Solr
> ---------------------------
>
> Key: SOLR-13608
> URL: https://issues.apache.org/jira/browse/SOLR-13608
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Cao Manh Dat
> Assignee: Jason Gerlowski
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently every call to backup API requires backup the whole index with
> different backupName. This is very costly and nearly useless in case of large
> frequent change indexes.
> Since index files of Lucene are written one only, they also contains the
> informantion about checksum of files. Then we can rely on these to support
> incremental backup -- only upload files that do not present in the repository.
> The design for this issue will be like this
> * Adding another parameter named {{incremental}} to backup API.
> * Adding new methods to {{BackupRepository}}, like compute checksum,
> deletefiles..
> * {{SnapShooter}} will skip uploading files from local if file in repository
> matches in checksum and length.
> * Segments_N will be copied last to guarantee that even the backup process
> get interrupted in the middle, the old backup will still can be used.
> * We only keep the last {{IndexCommit}} therefore after uploading Segments_N
> successfully, any file does not needed for the last {{IndexCommit}} will be
> deleted. We will try to improve this situation in another issue.
> * Any files in ZK will be re-uploaded
> ** The ZK files coressponds first backup will be stored in same location as
> today (to maintain backward compatibility)
> ** On subsequent backups ZK files will be stored in folder {{gen-ith}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
