Solr 8.0.0 + IndexUpgrader
Hi, I tried to upgrade my test index from Solr 7.7.1 to Solr 8.0.0. The file segments_4h7 already contains the string Lucene70. I upgraded before with this command: java -cp lucene-core-7.7.1.jar;lucene-backward-codecs-7.7.1.jar org.apache.lucene.index.IndexUpgrader C:\solr\server\solr\syneris\data\index\ Everything went successful, when I start solr via solr.cmd start, no errors are logged. Now, when I try to upgrade to Solr 8 I also tried to upgraded the index with the following command: java -cp lucene-core-8.0.0.jar;lucene-backward-codecs-8.0.0.jar org.apache.lucene.index.IndexUpgrader C:\solr\server\solr\syneris\data\index\ But I always get an exception: Exception in thread "main" org.apache.lucene.index.IndexFormatTooOldException: Format version is not supported (resource BufferedChecksumIndexInput(MMapIndexInput(path="C:\solr\server\solr\syneris\data\index\segments_4h7"))): This index was initially created with Lucene 6.x while the current version is 8.0.0 and Lucene only supports reading the current and previous major versions.. This version of Lucene only supports indexes created with release 7.0 and later. at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:318) at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:289) at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:432) at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:429) at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:680) at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:632) at org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:434) at org.apache.lucene.index.DirectoryReader.listCommits(DirectoryReader.java:260) at org.apache.lucene.index.IndexUpgrader.upgrade(IndexUpgrader.java:158) at org.apache.lucene.index.IndexUpgrader.main(IndexUpgrader.java:78) Any ideas, without performing a full reindex? Mit freundlichen Grüßen Herbert Hackelsberger Kundensupport/Qualitätssicherung __ TECHNODAT Technische Datenverarbeitung GmbH Jakob-Haringer-Straße 6 5020 Salzburg / Austria T | +43 (0)662 2282-141 F | +43 (0)662 2282-9 E | h...@technodat.at<mailto:h...@technodat.at> W | www.technodat.at Rechtsform: GmbH; Firmensitz: Salzburg Firmenbuchgericht: Landesgericht Salzburg FN 64072z; DVR: 0481831; UID-Nr. ATU33826508
AW: Solr 8.0.0 + IndexUpgrader
Thanks for the fast response! I used the IndexUpgrader to upgrade to 7.7.1 from 6.x and afterwards from 7.7.1 to 8.0.0 java -cp lucene-core-7.7.1.jar;lucene-backward-codecs-7.7.1.jar org.apache.lucene.index.IndexUpgrader C:\solr\server\solr\syneris\data\index\ java -cp lucene-core-8.0.0.jar;lucene-backward-codecs-8.0.0.jar org.apache.lucene.index.IndexUpgrader C:\solr\server\solr\syneris\data\index\ So, am I correct: - When using the IndexUpgrader, it will make the Index usable in the actual version, without all new features. - Using the Index Upgrader in the future again on the next major version will again result in this error situation. Best Regards -Ursprüngliche Nachricht- Von: Erick Erickson Gesendet: Montag, 1. April 2019 17:33 An: solr-user@lucene.apache.org Betreff: Re: Solr 8.0.0 + IndexUpgrader As of Lucene 6, a marker was written into each segment, and when segments are merged the lowest marker is preserved. If any marker for version of Lucene X-2 is found, you will see the error you see. This has been a source of considerable confusion. The guarantee of “one major revision backwards compatability” has always actually meant that in a case like yours, say from Lucene 5x -> 7x, you wouldn’t get a failure, but you _would_ get subtle errors. From Robert Muir: “...Because it is a lossy index and does not retain all of the user's data, its not possible to safely migrate some things automagically…" IndexUpgraderTool does not actually change this restriction. All it does is insure that all segments were written by the current version. It cannot recreate data that’s not there in the first place. Your only choice at this point is to fully re-index. Best, Erick > On Apr 1, 2019, at 8:19 AM, Herbert Hackelsberger wrote: > > Hi, > > I tried to upgrade my test index from Solr 7.7.1 to Solr 8.0.0. > The file segments_4h7 already contains the string Lucene70. > I upgraded before with this command: > > java -cp lucene-core-7.7.1.jar;lucene-backward-codecs-7.7.1.jar > org.apache.lucene.index.IndexUpgrader > C:\solr\server\solr\syneris\data\index\ > > Everything went successful, when I start solr via solr.cmd start, no errors > are logged. > Now, when I try to upgrade to Solr 8 I also tried to upgraded the index with > the following command: > > java -cp lucene-core-8.0.0.jar;lucene-backward-codecs-8.0.0.jar > org.apache.lucene.index.IndexUpgrader > C:\solr\server\solr\syneris\data\index\ > > But I always get an exception: > > Exception in thread "main" > org.apache.lucene.index.IndexFormatTooOldException: Format version is not > supported (resource > BufferedChecksumIndexInput(MMapIndexInput(path="C:\solr\server\solr\syneris\data\index\segments_4h7"))): > This index was initially created with Lucene 6.x while the current version > is 8.0.0 and Lucene only supports reading the current and previous major > versions.. This version of Lucene only supports indexes created with release > 7.0 and later. >at > org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:318) >at > org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:289) >at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:432) >at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:429) >at > org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:680) >at > org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:632) >at > org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:434) >at > org.apache.lucene.index.DirectoryReader.listCommits(DirectoryReader.java:260) >at > org.apache.lucene.index.IndexUpgrader.upgrade(IndexUpgrader.java:158) >at > org.apache.lucene.index.IndexUpgrader.main(IndexUpgrader.java:78) > > Any ideas, without performing a full reindex? > > > Mit freundlichen Grüßen > > Herbert Hackelsberger > Kundensupport/Qualitätssicherung > __ > TECHNODAT Technische Datenverarbeitung GmbH Jakob-Haringer-Straße 6 > 5020 Salzburg / Austria > > T | +43 (0)662 2282-141 > F | +43 (0)662 2282-9 > E | h...@technodat.at<mailto:h...@technodat.at> > W | www.technodat.at > > Rechtsform: GmbH; Firmensitz: Salzburg > Firmenbuchgericht: Landesgericht Salzburg FN 64072z; DVR: 0481831; > UID-Nr. ATU33826508 >
Index Upgrader Documention - Problem under Microsoft Windows
Hi all, I just wanted to inform you, when I stumbled across the ref guide at https://lucene.apache.org/solr/guide/7_7/indexupgrader-tool.html I had problems to run the command on my Windows System Instead of java -cp lucene-core-7.7.0.jar:lucene-backward-codecs-7.7.0.jar org.apache.lucene.index.IndexUpgrader [-delete-prior-commits] [-verbose] /path/to/index I had to write java -cp lucene-core-7.7.0.jar;lucene-backward-codecs-7.7.0.jar org.apache.lucene.index.IndexUpgrader [-delete-prior-commits] [-verbose] /path/to/index (instead of : use a ; to separate classpath entries) I found this information after research on the official documentation: https://docs.oracle.com/javase/8/docs/technotes/tools/windows/classpath.html#BEHJBHCD I don't know if that's specifically to Windows systems, but others might be happy, if an information regarding difference to MS Windows would be posted in the ref guide.
AW: AW: Solr 8.0.0 + IndexUpgrader
Many Thanks! -Ursprüngliche Nachricht- Von: Shawn Heisey Gesendet: Montag, 1. April 2019 18:03 An: solr-user@lucene.apache.org Betreff: Re: AW: Solr 8.0.0 + IndexUpgrader On 4/1/2019 9:47 AM, Herbert Hackelsberger wrote: > So, am I correct: > - When using the IndexUpgrader, it will make the Index usable in the actual > version, without all new features. > - Using the Index Upgrader in the future again on the next major version will > again result in this error situation. That is correct. If the "new features" are not related to the index format, then you will have full access to them even with an older index. The Lucene IndexUpgrader function does a forceMerge on the index, down to one segment. Solr calls that operation "optimize". There's really no need to use IndexUpgrader. Solr will directly use an index from one major version back with no trouble. If you ask Solr to optimize the index down to one segment, that is an identical operation to IndexUpgrader, with the difference that you can still access the index while it is happening. Thanks, Shawn
Windows SSL.Keystore and Windows TrustStore requires an empty PKCS#12 Key Store
Hi, I managed to get Windows-MY (SSL Personal Store) and Windows-ROOT (Root CA Store) with Solr 8.0.0 to work. How? I enabled the following in solr.in.cmd: set SOLR_SSL_CHECK_PEER_NAME=true set SOLR_SSL_ENABLED=true set SOLR_SSL_KEY_STORE=NONE set SOLR_SSL_KEY_STORE_PASSWORD= set SOLR_SSL_TRUST_STORE=NONE set SOLR_SSL_TRUST_STORE_PASSWORD= set SOLR_SSL_NEED_CLIENT_AUTH=true set SOLR_SSL_WANT_CLIENT_AUTH=false set SOLR_SSL_KEY_STORE_TYPE=Windows-MY set SOLR_SSL_TRUST_STORE_TYPE=Windows-ROOT A also edited solr.cmd in the following way: set "SOLR_SSL_OPTS= -Djavax.net.ssl.keyStoreProvider=SunMSCAPI -Djavax.net.ssl.trustStoreProvider=SunMSCAPI" But there is one problem: The Microsoft Key Store is not a file based Keystore. What happens: SOLR logs a missing KEYSTORE File "NONE" The official documentation at https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html tells me: * javax.net.ssl.keyStore system property. Note that the value NONE may be specified. This setting is appropriate if the keystore is not file-based (for example, it resides in a hardware token) The same is valid for trustStore. So my workaround here is to place an empty PKCS#12 keystore File called "NONE" in the \server directory, where start.jar resides. Solr 4.4 was happy with just an empty 0 byte NONE file. It seems to me, that currently only file based key stores are working without manual workarounds. A proper solution would be very nice for other so it can be easily configured. When I specify null, Solr requires the keystore file to be called null. And if not password specified at all, you won't get it to work. The Solr Reference Guide also lacks information here. The solution would be in the code to specify null when loading the keystore file, and password also null. I found that while searching: https://stackoverflow.com/questions/13697934/windows-keystores-and-certificates/29534497 Other software also seems to have problems with this: https://github.com/gradle/gradle/issues/6584 It would be great to see better integration of the Windows keystore I the future, as it was very difficulty to analyze find out, when you start from zero.
