Hey Aroop, the general process for our backup is: - Connect all machines to an EFS drive (AWS's NFS service) - Call the collections API to backup into EFS - ZIP the directory once the backup is completed - Copy the ZIP into an s3 bucket
I'll probably have to see which part of the process is the slowest. On another note, can you simply remove the task from the ZK path to continue the execution of tasks? Regards, Ash On Tue, Aug 11, 2020 at 11:40 AM Aroop Ganguly <aroopgang...@icloud.com.invalid> wrote: > 12 hours is extreme, we take backups of 10TB worth of indexes in 15 mins > using the collection backup api. > How are you taking the backup? > > Do you actually see any backup progress or u are just seeing the task in > the overseer queue linger ? > I have seen restore tasks hanging in the queue forever despite process > completing in Solr 77 so wouldn’t be surprised this happens with backup as > well. And also observed that unless that unless that task is removed from > the overseer-collection-queue the next ones do not proceed. > > Also adding replicas while backup seems like overkill, why don’t you just > have the appropriate replication factor in the first place and have > autoAddReplicas=true for indemnity? > > > On Aug 10, 2020, at 6:32 PM, Ashwin Ramesh <ash...@canva.com.INVALID> > wrote: > > > > Hi everybody, > > > > We are using solr 7.6 (SolrCloud). We notices that when the backup is > > running, we cannot add any replicas to the collection. By the looks of > it, > > the job to add the replica is put into the Overseer queue, but it is not > > being processed. Is this expected? And are there any workarounds? > > > > Our backups take about 12 hours. Maybe we should try optimize that too. > > > > Regards, > > > > Ash > > > > -- > > ** > > ** <https://www.canva.com/>Empowering the world to design > > Share accurate > > information on COVID-19 and spread messages of support to your community. > > > > Here are some resources > > < > https://about.canva.com/coronavirus-awareness-collection/?utm_medium=pr&utm_source=news&utm_campaign=covid19_templates> > > > that can help. > > <https://twitter.com/canva> <https://facebook.com/canva> > > <https://au.linkedin.com/company/canva> <https://twitter.com/canva> > > <https://facebook.com/canva> <https://au.linkedin.com/company/canva> > > <https://instagram.com/canva> > > > > > > > > > > > > > > > > > > > > > > -- ** ** <https://www.canva.com/>Empowering the world to design Share accurate information on COVID-19 and spread messages of support to your community. Here are some resources <https://about.canva.com/coronavirus-awareness-collection/?utm_medium=pr&utm_source=news&utm_campaign=covid19_templates> that can help. <https://twitter.com/canva> <https://facebook.com/canva> <https://au.linkedin.com/company/canva> <https://twitter.com/canva> <https://facebook.com/canva> <https://au.linkedin.com/company/canva> <https://instagram.com/canva>
