Erick, thanks, that is of course something I left out of the original question. Our Solr is 7.1, so that should not present a problem (crossing fingers).
However, on my dev box I'm trying out the steps, and here I have some segments created with version 6 of Solr. After having copied data from my non-cloud solr into my single-shard-single-replica collection and verified that Solr Cloud works with this collection, I then submit the splitshard command http://172.17.0.4:8984 /solr/admin/collections?action=SPLITSHARD&collection=procurement&shard=shard1 However, this gives me the error: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error from server at http://172.17.0.4:8984/solr: java.lang.IllegalArgumentException: Cannot merge a segment that has been created with major version 6 into this index which has been created by major version 7"} I have tried running both optimize and IndexUpgrader on the index before shard splitting, but the same error still occurs. Any ideas as to why this happens? Below is an output from running IndexUpgrader, which I cannot decipher. It both states that "All segments upgraded to version 7.1.0" and ''all running merges have aborted" ¯\_(ツ)_/¯ Thanks a lot, Bjarke ====================== java -cp /opt/solr/server/solr-webapp/webapp/WEB-INF/lib/lucene-backward-codecs-7.1.0.jar:/opt/solr/server/solr-webapp/webapp/WEB-INF/lib/lucene-core-7.1.0.jar org.apache.lucene.index.IndexUpgrader -delete-prior-commits -verbose /var/solr/cloud/procurement_shard1_replica_n1/data/index IFD 0 [2018-08-08T13:00:18.244Z; main]: init: current segments file is "segments_4vs"; deletionPolicy=org.apache.lucene.index.KeepOnlyLastCommitDeletionPolicy@721e0f4f IFD 0 [2018-08-08T13:00:18.266Z; main]: init: load commit "segments_4vs" IFD 0 [2018-08-08T13:00:18.270Z; main]: now checkpoint "_bhg(7.1.0):C108396" [1 segments ; isCommit = false] IFD 0 [2018-08-08T13:00:18.270Z; main]: 0 msec to checkpoint IW 0 [2018-08-08T13:00:18.270Z; main]: init: create=false IW 0 [2018-08-08T13:00:18.273Z; main]: dir=MMapDirectory@/var/solr/cloud/procurement_shard1_replica_n1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6debcae2 index=_bhg(7.1.0):C108396 version=7.1.0 analyzer=null ramBufferSizeMB=16.0 maxBufferedDocs=-1 mergedSegmentWarmer=null delPolicy=org.apache.lucene.index.KeepOnlyLastCommitDeletionPolicy commit=null openMode=CREATE_OR_APPEND similarity=org.apache.lucene.search.similarities.BM25Similarity mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=-1, maxMergeCount=-1, ioThrottle=true codec=Lucene70 infoStream=org.apache.lucene.util.PrintStreamInfoStream mergePolicy=UpgradeIndexMergePolicy([TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=5120.0, floorSegmentMB=2.0, forceMergeDeletesPctAllowed=10.0, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1) indexerThreadPool=org.apache.lucene.index.DocumentsWriterPerThreadPool@5ba23b66 readerPooling=true perThreadHardLimitMB=1945 useCompoundFile=true commitOnClose=true indexSort=null writer=org.apache.lucene.index.IndexWriter@2ff4f00f IW 0 [2018-08-08T13:00:18.273Z; main]: MMapDirectory.UNMAP_SUPPORTED=true IndexUpgrader 0 [2018-08-08T13:00:18.274Z; main]: Upgrading all pre-7.1.0 segments of index directory 'MMapDirectory@/var/solr/cloud/procurement_shard1_replica_n1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6debcae2' to version 7.1.0... IW 0 [2018-08-08T13:00:18.274Z; main]: forceMerge: index now _bhg(7.1.0):C108396 IW 0 [2018-08-08T13:00:18.274Z; main]: now flush at forceMerge IW 0 [2018-08-08T13:00:18.274Z; main]: start flush: applyAllDeletes=true IW 0 [2018-08-08T13:00:18.274Z; main]: index before flush _bhg(7.1.0):C108396 DW 0 [2018-08-08T13:00:18.274Z; main]: startFullFlush DW 0 [2018-08-08T13:00:18.275Z; main]: main finishFullFlush success=true IW 0 [2018-08-08T13:00:18.275Z; main]: now apply all deletes for all segments buffered updates bytesUsed=0 reader pool bytesUsed=0 BD 0 [2018-08-08T13:00:18.275Z; main]: waitApply: no deletes to apply UPGMP 0 [2018-08-08T13:00:18.276Z; main]: findForcedMerges: segmentsToUpgrade={} MS 0 [2018-08-08T13:00:18.282Z; main]: initDynamicDefaults spins=true maxThreadCount=1 maxMergeCount=6 MS 0 [2018-08-08T13:00:18.282Z; main]: now merge MS 0 [2018-08-08T13:00:18.282Z; main]: index: _bhg(7.1.0):C108396 MS 0 [2018-08-08T13:00:18.282Z; main]: no more merges pending; now return IndexUpgrader 0 [2018-08-08T13:00:18.282Z; main]: All segments upgraded to version 7.1.0 IndexUpgrader 0 [2018-08-08T13:00:18.283Z; main]: Enforcing commit to rewrite all index metadata... IW 0 [2018-08-08T13:00:18.283Z; main]: commit: start IW 0 [2018-08-08T13:00:18.283Z; main]: commit: enter lock IW 0 [2018-08-08T13:00:18.283Z; main]: commit: now prepare IW 0 [2018-08-08T13:00:18.283Z; main]: prepareCommit: flush IW 0 [2018-08-08T13:00:18.283Z; main]: index before flush _bhg(7.1.0):C108396 DW 0 [2018-08-08T13:00:18.283Z; main]: startFullFlush IW 0 [2018-08-08T13:00:18.283Z; main]: now apply all deletes for all segments buffered updates bytesUsed=0 reader pool bytesUsed=0 BD 0 [2018-08-08T13:00:18.283Z; main]: waitApply: no deletes to apply DW 0 [2018-08-08T13:00:18.284Z; main]: main finishFullFlush success=true IW 0 [2018-08-08T13:00:18.284Z; main]: startCommit(): start IW 0 [2018-08-08T13:00:18.284Z; main]: startCommit index=_bhg(7.1.0):C108396 changeCount=2 IW 0 [2018-08-08T13:00:18.293Z; main]: startCommit: wrote pending segments file "pending_segments_4vt" IW 0 [2018-08-08T13:00:18.295Z; main]: done all syncs: [_bhg_Lucene50_0.tip, _bhg.fdx, _bhg.fnm, _bhg.nvm, _bhg.fdt, _bhg.si, _bhg_Lucene50_0.pos, _bhg.nvd, _bhg_Lucene50_0.doc, _bhg_Lucene50_0.tim] IW 0 [2018-08-08T13:00:18.295Z; main]: commit: pendingCommit != null IW 0 [2018-08-08T13:00:18.298Z; main]: commit: done writing segments file "segments_4vt" IFD 0 [2018-08-08T13:00:18.298Z; main]: now checkpoint "_bhg(7.1.0):C108396" [1 segments ; isCommit = true] IFD 0 [2018-08-08T13:00:18.298Z; main]: deleteCommits: now decRef commit "segments_4vs" IFD 0 [2018-08-08T13:00:18.298Z; main]: delete [segments_4vs] IFD 0 [2018-08-08T13:00:18.299Z; main]: 0 msec to checkpoint IW 0 [2018-08-08T13:00:18.319Z; main]: commit: took 16.0 msec IW 0 [2018-08-08T13:00:18.319Z; main]: commit: done IndexUpgrader 0 [2018-08-08T13:00:18.319Z; main]: Committed upgraded metadata to index. IW 0 [2018-08-08T13:00:18.319Z; main]: now flush at close IW 0 [2018-08-08T13:00:18.319Z; main]: start flush: applyAllDeletes=true IW 0 [2018-08-08T13:00:18.319Z; main]: index before flush _bhg(7.1.0):C108396 DW 0 [2018-08-08T13:00:18.319Z; main]: startFullFlush DW 0 [2018-08-08T13:00:18.320Z; main]: main finishFullFlush success=true IW 0 [2018-08-08T13:00:18.320Z; main]: now apply all deletes for all segments buffered updates bytesUsed=0 reader pool bytesUsed=0 BD 0 [2018-08-08T13:00:18.320Z; main]: waitApply: no deletes to apply MS 0 [2018-08-08T13:00:18.320Z; main]: updateMergeThreads ioThrottle=true targetMBPerSec=10240.0 MB/sec MS 0 [2018-08-08T13:00:18.320Z; main]: now merge MS 0 [2018-08-08T13:00:18.321Z; main]: index: _bhg(7.1.0):C108396 MS 0 [2018-08-08T13:00:18.321Z; main]: no more merges pending; now return IW 0 [2018-08-08T13:00:18.321Z; main]: waitForMerges IW 0 [2018-08-08T13:00:18.321Z; main]: waitForMerges done IW 0 [2018-08-08T13:00:18.321Z; main]: commit: start IW 0 [2018-08-08T13:00:18.321Z; main]: commit: enter lock IW 0 [2018-08-08T13:00:18.321Z; main]: commit: now prepare IW 0 [2018-08-08T13:00:18.321Z; main]: prepareCommit: flush IW 0 [2018-08-08T13:00:18.321Z; main]: index before flush _bhg(7.1.0):C108396 DW 0 [2018-08-08T13:00:18.321Z; main]: startFullFlush IW 0 [2018-08-08T13:00:18.321Z; main]: now apply all deletes for all segments buffered updates bytesUsed=0 reader pool bytesUsed=0 BD 0 [2018-08-08T13:00:18.322Z; main]: waitApply: no deletes to apply DW 0 [2018-08-08T13:00:18.322Z; main]: main finishFullFlush success=true IW 0 [2018-08-08T13:00:18.322Z; main]: startCommit(): start IW 0 [2018-08-08T13:00:18.322Z; main]: skip startCommit(): no changes pending IW 0 [2018-08-08T13:00:18.322Z; main]: commit: pendingCommit == null; skip IW 0 [2018-08-08T13:00:18.322Z; main]: commit: took 0.9 msec IW 0 [2018-08-08T13:00:18.322Z; main]: commit: done IW 0 [2018-08-08T13:00:18.322Z; main]: rollback IW 0 [2018-08-08T13:00:18.322Z; main]: all running merges have aborted IW 0 [2018-08-08T13:00:18.323Z; main]: rollback: done finish merges DW 0 [2018-08-08T13:00:18.323Z; main]: abort DW 0 [2018-08-08T13:00:18.323Z; main]: done abort success=true IW 0 [2018-08-08T13:00:18.323Z; main]: rollback: infos=_bhg(7.1.0):C108396 IFD 0 [2018-08-08T13:00:18.323Z; main]: now checkpoint "_bhg(7.1.0):C108396" [1 segments ; isCommit = false] IFD 0 [2018-08-08T13:00:18.323Z; main]: 0 msec to checkpoint 2018-08-07 16:38 GMT+02:00 Erick Erickson <erickerick...@gmail.com>: > Bjarke: > > One thing, what version of Solr are you moving _from_ and _to_? > Solr/Lucene only guarantee one major backward revision so you can copy > an index created with Solr 6 to another Solr 6 or Solr 7, but you > couldn't copy an index created with Solr 5 to Solr 7... > > Also note that shard splitting is a very expensive operation, so be > patient.... > > Best, > Erick > > On Tue, Aug 7, 2018 at 6:17 AM, Rahul Singh > <rahul.xavier.si...@gmail.com> wrote: > > Bjarke, > > > > I am imagining that at some point you may need to shard that data if it > grows. Or do you imagine this data to remain stagnant? > > > > Generally you want to add solrcloud to do two things : 1. Increase > availability with replicas 2. Increase available data via shards 3. > Increase fault tolerance with leader and replicas being spread around the > cluster. > > > > You would be bypassing general High availability / distributed computing > processes by trying to not reindex. > > > > Rahul > > On Aug 7, 2018, 7:06 AM -0400, Bjarke Buur Mortensen < > morten...@eluence.com>, wrote: > >> Hi List, > >> > >> is there a cookbook recipe for moving an existing solr core to a solr > cloud > >> collection. > >> > >> We currently have a single machine with a large core (~150gb), and we > would > >> like to move to solr cloud. > >> > >> I haven't been able to find anything that reuses an existing index, so > any > >> pointers much appreciated. > >> > >> Thanks, > >> Bjarke >
