[
https://issues.apache.org/jira/browse/SOLR-13608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077476#comment-17077476
]
Ishan Chattopadhyaya commented on SOLR-13608:
---------------------------------------------
What remains to be done for this, [~caomanhdat]? Is it ready for us to start
reviewing?
> Incremental backup for Solr
> ---------------------------
>
> Key: SOLR-13608
> URL: https://issues.apache.org/jira/browse/SOLR-13608
> Project: Solr
> Issue Type: Improvement
> Reporter: Cao Manh Dat
> Assignee: Cao Manh Dat
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently every call to backup API requires backup the whole index with
> different backupName. This is very costly and nearly useless in case of large
> frequent change indexes.
> Since index files of Lucene are written one only, they also contains the
> informantion about checksum of files. Then we can rely on these to support
> incremental backup -- only upload files that do not present in the repository.
> The design for this issue will be like this
> * Adding another parameter named {{incremental}} to backup API.
> * Adding new methods to {{BackupRepository}}, like compute checksum,
> deletefiles..
> * {{SnapShooter}} will skip uploading files from local if file in repository
> matches in checksum and length.
> * Segments_N will be copied last to guarantee that even the backup process
> get interrupted in the middle, the old backup will still can be used.
> * We only keep the last {{IndexCommit}} therefore after uploading Segments_N
> successfully, any file does not needed for the last {{IndexCommit}} will be
> deleted. We will try to improve this situation in another issue.
> * Any files in ZK will be re-uploaded
> ** The ZK files coressponds first backup will be stored in same location as
> today (to maintain backward compatibility)
> ** On subsequent backups ZK files will be stored in folder {{gen-ith}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
