date:20200504

[jira] [Created] (LUCENE-9358) BKDTree: remove unnecessary tree rotation for the one dimensional case

2020-05-04 Thread Ignacio Vera (Jira)

Ignacio Vera created LUCENE-9358:


 Summary: BKDTree: remove unnecessary tree rotation for the one 
dimensional case 
 Key: LUCENE-9358
 URL: https://issues.apache.org/jira/browse/LUCENE-9358
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Ignacio Vera


This is a spin-off of LUCENE-9807. The reason we need to rotate the one 
dimensional tree is that the expected representation when we pack the index is 
different to the tree generated by the one dimensional logic. It would be easy 
to harmonise how we generate this tree representation to be the same in the one 
dimensional case and the multi-dimensional case and therefore change the 
index-packing logic to work on that representation.

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Resolved] (SOLR-7111) Better handling of relative links for SimplePostTool

2020-05-04 Thread Jira



 [ 
https://issues.apache.org/jira/browse/SOLR-7111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl resolved SOLR-7111.
---
Resolution: Won't Fix

This is not a full-blown crawler, let's not try to make it one :)

> Better handling of relative links for SimplePostTool
> 
>
> Key: SOLR-7111
> URL: https://issues.apache.org/jira/browse/SOLR-7111
> Project: Solr
>  Issue Type: Bug
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
>
> The very simplistic crawler in SimplePostTool could handle links such as 
> {{href="./foo"}}, {{href="../foo"}} and {{href="#foo"}} better. Also, in some 
> cases there will be double {{//}} when concatenating base URL and relative 
> links.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] iverase opened a new pull request #1481: LUCENE-9358: remove unnecessary tree rotation for the one dimensional case

2020-05-04 Thread GitBox



iverase opened a new pull request #1481:
URL: https://github.com/apache/lucene-solr/pull/1481


   This commit changes the way the multi-dimensional tree builder generates the 
intermediate tree representation to be equal to the one dimensional case. 
Therefore, the index packing logic can be changed to work on the representation 
and avoid unnecessary tree and leaves rotation.
   
   A new interface is introduced to avoid copying intermediate List arrays.
   
   split values and split dimensions are handled in different arrays which 
means we are increasing the number of max points the tree builders can handle.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] jpountz opened a new pull request #1482: LUCENE-7822: CodecUtil#checkFooter should throw a CorruptIndexException as the main exception.

2020-05-04 Thread GitBox



jpountz opened a new pull request #1482:
URL: https://github.com/apache/lucene-solr/pull/1482


   See https://issues.apache.org/jira/browse/LUCENE-7822.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7822) IllegalArgumentException thrown instead of a CorruptIndexException

2020-05-04 Thread Adrien Grand (Jira)



[ 
https://issues.apache.org/jira/browse/LUCENE-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17098750#comment-17098750
 ] 

Adrien Grand commented on LUCENE-7822:
--

I opened [https://github.com/apache/lucene-solr/pull/1482/files?w=1] to discuss 
what it could look like. I have a slight preference for changing checkFooter 
instead of calling checksumEntireFile up-front since the former guarantees that 
we verify the checksum of the exact bytes that we just read, but I could be 
convinced otherwise.

> IllegalArgumentException thrown instead of a CorruptIndexException
> --
>
> Key: LUCENE-7822
> URL: https://issues.apache.org/jira/browse/LUCENE-7822
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.5.1
>Reporter: Martin Amirault
>Priority: Minor
> Attachments: LUCENE-7822.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Similarly to LUCENE-7592 , When an {{*.si}} file is corrupted on very 
> specific part an IllegalArgumentException is thrown instead of a 
> CorruptIndexException.
> StackTrace (Lucene 6.5.1):
> {code}
> java.lang.IllegalArgumentException: Illegal minor version: 12517381
>   at 
> __randomizedtesting.SeedInfo.seed([1FEB5987CFA44BE:B8755B5574F9F3BF]:0)
>   at org.apache.lucene.util.Version.(Version.java:385)
>   at org.apache.lucene.util.Version.(Version.java:371)
>   at org.apache.lucene.util.Version.fromBits(Version.java:353)
>   at 
> org.apache.lucene.codecs.lucene62.Lucene62SegmentInfoFormat.read(Lucene62SegmentInfoFormat.java:97)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:357)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:448)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:445)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:692)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:644)
>   at 
> org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:450)
>   at 
> org.apache.lucene.index.DirectoryReader.listCommits(DirectoryReader.java:260)
> {code}
> Simple fix would be to add IllegalArgumentException to the catch list at 
> {{org/apache/lucene/index/SegmentInfos.java:289}}
> Other variations for the stacktraces:
> {code}
> java.lang.IllegalArgumentException: invalid codec filename '_�.cfs', must 
> match: _[a-z0-9]+(_.*)?\..*
>   at 
> __randomizedtesting.SeedInfo.seed([8B3FDE317B8D634A:A8EE07E5EB4B0B13]:0)
>   at 
> org.apache.lucene.index.SegmentInfo.checkFileNames(SegmentInfo.java:270)
>   at org.apache.lucene.index.SegmentInfo.addFiles(SegmentInfo.java:252)
>   at org.apache.lucene.index.SegmentInfo.setFiles(SegmentInfo.java:246)
>   at 
> org.apache.lucene.codecs.lucene62.Lucene62SegmentInfoFormat.read(Lucene62SegmentInfoFormat.java:248)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:357)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:448)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:445)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:692)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:644)
>   at 
> org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:450)
>   at 
> org.apache.lucene.index.DirectoryReader.listCommits(DirectoryReader.java:260)
> {code}
> {code}
> java.lang.IllegalArgumentException: An SPI class of type 
> org.apache.lucene.codecs.Codec with name 'LucenI62' does not exist.  You need 
> to add the corresponding JAR file supporting this SPI to your classpath.  The 
> current classpath supports the following names: [Lucene62, Lucene50, 
> Lucene53, Lucene54, Lucene60]
>   at 
> __randomizedtesting.SeedInfo.seed([925DE160F7260F99:B026EB9373CB6368]:0)
>   at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:116)
>   at org.apache.lucene.codecs.Codec.forName(Codec.java:116)
>   at org.apache.lucene.index.SegmentInfos.readCodec(SegmentInfos.java:424)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:356)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:448)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:445)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:692)
>

[jira] [Updated] (LUCENE-9328) SortingGroupHead to reuse DocValues

2020-05-04 Thread Mikhail Khludnev (Jira)



 [ 
https://issues.apache.org/jira/browse/LUCENE-9328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated LUCENE-9328:
-
Attachment: LUCENE-9328.patch
Status: Patch Available  (was: Patch Available)

> SortingGroupHead to reuse DocValues
> ---
>
> Key: LUCENE-9328
> URL: https://issues.apache.org/jira/browse/LUCENE-9328
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/grouping
>Reporter: Mikhail Khludnev
>Assignee: Mikhail Khludnev
>Priority: Minor
> Attachments: LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch, 
> LUCENE-9328.patch, LUCENE-9328.patch
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> That's why 
> https://issues.apache.org/jira/browse/LUCENE-7701?focusedCommentId=17084365&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17084365



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Resolved] (SOLR-8579) TLP website identity updates

2020-05-04 Thread Jira



 [ 
https://issues.apache.org/jira/browse/SOLR-8579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl resolved SOLR-8579.
---
Resolution: Fixed

> TLP website identity updates
> 
>
> Key: SOLR-8579
> URL: https://issues.apache.org/jira/browse/SOLR-8579
> Project: Solr
>  Issue Type: Task
>  Components: website
>Reporter: Jan Høydahl
>Priority: Major
>  Labels: newdev
> Attachments: download-button.patch
>
>
> The TLP site http://lucene.apache.org/ could need some logo updates, and I 
> feel that the TLP site should better reflect that it is a *Project* and not a 
> *Product* site. As it is now it looks almost identical to the Lucene-Java 
> site. Would be nice to start from scratch, alternatively fix the old one.
> h2. Option A: New responsive TLP site
> Create a super clean new TLP site with responsive design. Content could be 
> limited to describing the three sub projects with logo, short presentation, 
> download button and link to product sites. Perhaps also promote the community 
> in form of some auto-updated stats (active committers, ML activity, link to 
> last board report?). No slideshow, no endless news, no duplicate menus...
> h2. Option B: Refresh the existing site
> *Should*
> * The top branding contains a Lucene+ASF logo. Make a new top with the brand 
> new ASF feather logo, Lucene logo and Solr logo
> * Replace old orange Solr logo in slideshow with the new red one
> * Color scheme is the Lucene pale green, same as for Lucene-core. Choose 
> another color scheme for the TLP!
> * Remove the discontinued OpenRelevance project from top menu and intro 
> bullet list. Keep a link "OpenRelevance (discontinued)" in right-menu?
> *Optional*
> * Color of the Solr Download button should be changed to Solr-RED™
> * Likewise, color of the Lucene Download button could take Lucene-GREEN™ ?
> * Main title says *Welcome to Apache Lucene*. Perhaps it should say *Welcome 
> to Apache Lucene/Solr*?
> * Update the slide show images and texts to better describe the project as of 
> 2016...



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Created] (LUCENE-9359) SegmentInfos.readCommit should verify checksums in case of error

2020-05-04 Thread Adrien Grand (Jira)

Adrien Grand created LUCENE-9359:


 Summary: SegmentInfos.readCommit should verify checksums in case 
of error
 Key: LUCENE-9359
 URL: https://issues.apache.org/jira/browse/LUCENE-9359
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand


SegmentInfos.readCommit only calls checkFooter if reading the commit succeeded. 
We should also call it in case of errors in order to be able to distinguish 
hardware errors from Lucene bugs.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] romseygeek commented on a change in pull request #1462: LUCENE-9328: open group.sort docvalues once per segment.

2020-05-04 Thread GitBox



romseygeek commented on a change in pull request #1462:
URL: https://github.com/apache/lucene-solr/pull/1462#discussion_r419289810



##
File path: 
lucene/grouping/src/test/org/apache/lucene/search/grouping/DocValuesPoolingReaderTest.java
##
@@ -0,0 +1,150 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.search.grouping;
+
+import java.io.IOException;
+
+import org.apache.lucene.analysis.MockAnalyzer;
+import org.apache.lucene.document.BinaryDocValuesField;
+import org.apache.lucene.document.Document;
+import org.apache.lucene.document.NumericDocValuesField;
+import org.apache.lucene.document.SortedDocValuesField;
+import org.apache.lucene.document.SortedNumericDocValuesField;
+import org.apache.lucene.document.SortedSetDocValuesField;
+import org.apache.lucene.index.BinaryDocValues;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.LeafReader;
+import org.apache.lucene.index.LeafReaderContext;
+import org.apache.lucene.index.NumericDocValues;
+import org.apache.lucene.index.RandomIndexWriter;
+import org.apache.lucene.index.SortedDocValues;
+import org.apache.lucene.index.SortedNumericDocValues;
+import org.apache.lucene.index.SortedSetDocValues;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.util.BytesRef;
+import org.apache.lucene.util.LuceneTestCase;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+
+public class DocValuesPoolingReaderTest extends LuceneTestCase {
+  
+  private static RandomIndexWriter w;
+  private static Directory dir;
+  private static DirectoryReader reader;
+
+  @BeforeClass
+  public static void index() throws IOException {
+dir = newDirectory();
+w = new RandomIndexWriter(
+random(),
+dir,
+newIndexWriterConfig(new 
MockAnalyzer(random())).setMergePolicy(newLogMergePolicy()));
+Document doc = new Document();
+doc.add(new BinaryDocValuesField("bin", new BytesRef("binary")));
+doc.add(new BinaryDocValuesField("bin2", new BytesRef("binary2")));
+
+doc.add(new NumericDocValuesField("num", 1L));
+doc.add(new NumericDocValuesField("num2", 2L));
+
+doc.add(new SortedNumericDocValuesField("sortnum", 3L));
+doc.add(new SortedNumericDocValuesField("sortnum2", 4L));
+
+doc.add(new SortedDocValuesField("sort",  new BytesRef("sorted")));
+doc.add(new SortedDocValuesField("sort2",  new BytesRef("sorted2")));
+
+doc.add(new SortedSetDocValuesField("sortset", new BytesRef("sortedset")));
+doc.add(new SortedSetDocValuesField("sortset2", new 
BytesRef("sortedset2")));
+
+w.addDocument(doc);
+w.commit();
+reader = w.getReader();
+w.close();
+  }
+  
+  public void testDVCache() throws IOException {
+assertFalse(reader.leaves().isEmpty());
+for (LeafReaderContext leaf : reader.leaves()) {
+  final DocValuesPoolingReader caching = new 
DocValuesPoolingReader(leaf.reader());
+  
+  assertSame(assertBinaryDV(caching, "bin", "binary"), 
+  caching.getBinaryDocValues("bin"));
+  assertSame(assertBinaryDV(caching, "bin2", "binary2"), 
+  caching.getBinaryDocValues("bin2"));
+  
+  assertSame(assertNumericDV(caching, "num", 1L), 
+  caching.getNumericDocValues("num"));
+  assertSame(assertNumericDV(caching, "num2", 2L), 
+  caching.getNumericDocValues("num2"));
+  
+  assertSame(assertSortedNumericDV(caching, "sortnum", 3L), 
+  caching.getSortedNumericDocValues("sortnum"));
+  assertSame(assertSortedNumericDV(caching, "sortnum2", 4L), 
+  caching.getSortedNumericDocValues("sortnum2"));
+  
+  assertSame(assertSortedDV(caching, "sort", "sorted"), 
+  caching.getSortedDocValues("sort"));
+  assertSame(assertSortedDV(caching, "sort2", "sorted2"), 
+  caching.getSortedDocValues("sort2"));

Review comment:
   I think this still doesn't test iteration through a single doc's values 
on two instances of the shared DV?  We need to pull the iterator twice, advance 
both to the same doc, and then iterate through the values on both - as I read 
it, currently if you iterate through the values on one, then you can'

[jira] [Created] (SOLR-14458) Solr Replica locked in recovering state after a Zookeeper disconnection

2020-05-04 Thread Endika Posadas (Jira)

Endika Posadas created SOLR-14458:
-

Summary: Solr Replica locked in recovering state after a Zookeeper
disconnection
Key: SOLR-14458
URL: https://issues.apache.org/jira/browse/SOLR-14458
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: SolrCloud
Affects Versions: 8.4.1
Environment: A Solr cluster with 2 replicas that each has 2 shards
split across 2 Windows VMS.
They use a 3 replica zookeeper across 3 vms.
Reporter: Endika Posadas
Attachments: replica7.log, solr-thread-dump.log, solr.log

In a solr cluster, a Solr instance containing two shards has lost connection
with zookeeper. Upon reconnecting, it has checked the status with the leader
and start a recovery. However, it's stuck in recovering status without making
further progress (has been like that for days now).

Upon checking a thread dump, `recoveryExecutor-7-thread-3-processing-n` is
trying to acquire the lock to createa new Index Writer: `at
org.apache.solr.update.DefaultSolrCoreState.lock(DefaultSolrCoreState.java:179)`
(

after lock(iwLock.writeLock()){color:#cc7832};{color}). However, the
ReentrantLock it's waiting for is never released. Moreover, no thread can be
found holding the lock, leaving restarting Solr as the only solution.

There is no Error in the logs that can help with the issue. I have attached
solr.log and a grep with node 7 lines, as well as a thread dump.

My hypothesis is that
org.apache.solr.update.DefaultSolrCoreState#closeIndexWriter(org.apache.solr.core.SolrCore,
boolean) was called once but for some reason openIndexWriter was skipped.

81 matches

Mail list logo