[GitHub] [lucene] stefanvodita commented on pull request #12506: Clean up ByteBlockPool

2023-09-30 Thread via GitHub


stefanvodita commented on PR #12506:
URL: https://github.com/apache/lucene/pull/12506#issuecomment-1741739207

   I noticed the failing checks on this PR, but I haven't been able to 
reproduce them. They appear related to the nested javadoc tags I had 
introduced. I've removed them now. Hopefully that satisfies the checks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene] pzygielo opened a new pull request, #12611: Avoid NPEx if the end of the stream has been reached without reading any characters

2023-09-30 Thread via GitHub


pzygielo opened a new pull request, #12611:
URL: https://github.com/apache/lucene/pull/12611

   e.g. by user responding with ^D
   ```
   Press (n)ext page, (q)uit or enter number to jump to a page.
   Exception in thread "main" java.lang.NullPointerException: Cannot invoke 
"String.length()" because "line" is null
at 
org.apache.lucene.demo.SearchFiles.doPagingSearch(SearchFiles.java:244)
at org.apache.lucene.demo.SearchFiles.main(SearchFiles.java:152)
   ```
   
   ```
   Press (p)revious page, (n)ext page, (q)uit or enter number to jump to a page.
   n
   Only results 1 - 50 of 104 total matching documents collected.
   Collect more (y/n) ?
   Exception in thread "main" java.lang.NullPointerException: Cannot invoke 
"String.length()" because "line" is null
at 
org.apache.lucene.demo.SearchFiles.doPagingSearch(SearchFiles.java:198)
at org.apache.lucene.demo.SearchFiles.main(SearchFiles.java:152)
   ```
   
   ### Description
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene] stefanvodita commented on a diff in pull request #12548: Ability to compute vector similarity scores with DoubleValuesSource

2023-09-30 Thread via GitHub


stefanvodita commented on code in PR #12548:
URL: https://github.com/apache/lucene/pull/12548#discussion_r1341952008


##
lucene/core/src/test/org/apache/lucene/search/TestVectorSimilarityValuesSource.java:
##
@@ -0,0 +1,385 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.lucene.search;
+
+import org.apache.lucene.analysis.Analyzer;
+import org.apache.lucene.document.Document;
+import org.apache.lucene.document.Field;
+import org.apache.lucene.document.KnnByteVectorField;
+import org.apache.lucene.document.KnnFloatVectorField;
+import org.apache.lucene.document.SortedDocValuesField;
+import org.apache.lucene.document.StringField;
+import org.apache.lucene.index.IndexReader;
+import org.apache.lucene.index.IndexWriterConfig;
+import org.apache.lucene.index.VectorSimilarityFunction;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.tests.analysis.MockAnalyzer;
+import org.apache.lucene.tests.index.RandomIndexWriter;
+import org.apache.lucene.tests.util.LuceneTestCase;
+import org.apache.lucene.util.BytesRef;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+
+public class TestVectorSimilarityValuesSource extends LuceneTestCase {
+  private static Directory dir;
+  private static Analyzer analyzer;
+  private static IndexReader reader;
+  private static IndexSearcher searcher;
+
+  @BeforeClass
+  public static void beforeClass() throws Exception {
+dir = newDirectory();
+analyzer = new MockAnalyzer(random());
+IndexWriterConfig iwConfig = newIndexWriterConfig(analyzer);
+iwConfig.setMergePolicy(newLogMergePolicy());
+RandomIndexWriter iw = new RandomIndexWriter(random(), dir, iwConfig);
+
+Document document = new Document();
+document.add(new StringField("id", "1", Field.Store.NO));
+document.add(new SortedDocValuesField("id", new BytesRef("1")));
+document.add(new KnnFloatVectorField("knnFloatField1", new float[] {1.f, 
2.f, 3.f}));
+document.add(
+new KnnFloatVectorField(
+"knnFloatField2",
+new float[] {2.2f, -3.2f, -3.1f},
+VectorSimilarityFunction.DOT_PRODUCT));
+document.add(
+new KnnFloatVectorField(
+"knnFloatField3", new float[] {4.5f, 10.3f, -7.f}, 
VectorSimilarityFunction.COSINE));
+document.add(
+new KnnFloatVectorField(
+"knnFloatField4",
+new float[] {-1.3f, 1.0f, 1.0f},
+VectorSimilarityFunction.MAXIMUM_INNER_PRODUCT));
+document.add(new KnnFloatVectorField("knnFloatField5", new float[] {-6.7f, 
-1.0f, -0.9f}));
+document.add(new KnnByteVectorField("knnByteField1", new byte[] {106, 80, 
127}));
+document.add(
+new KnnByteVectorField(
+"knnByteField2", new byte[] {4, 2, 3}, 
VectorSimilarityFunction.DOT_PRODUCT));
+document.add(
+new KnnByteVectorField(
+"knnByteField3", new byte[] {-121, -64, -1}, 
VectorSimilarityFunction.COSINE));
+document.add(
+new KnnByteVectorField(
+"knnByteField4",
+new byte[] {-127, 127, 127},
+VectorSimilarityFunction.MAXIMUM_INNER_PRODUCT));
+iw.addDocument(document);
+
+Document document2 = new Document();
+document2.add(new StringField("id", "2", Field.Store.NO));
+document2.add(new SortedDocValuesField("id", new BytesRef("2")));
+document2.add(new KnnFloatVectorField("knnFloatField1", new float[] {1.f, 
2.f, 3.f}));
+document2.add(
+new KnnFloatVectorField(
+"knnFloatField2",
+new float[] {-5.2f, 8.7f, 3.1f},
+VectorSimilarityFunction.DOT_PRODUCT));
+document2.add(
+new KnnFloatVectorField(
+"knnFloatField3", new float[] {0.2f, -3.2f, 3.1f}, 
VectorSimilarityFunction.COSINE));
+document2.add(new KnnFloatVectorField("knnFloatField5", new float[] {2.f, 
13.2f, 9.1f}));
+document2.add(new KnnByteVectorField("knnByteField1", new byte[] {1, -2, 
-30}));
+document2.add(
+new KnnByteVectorField(
+"knnByteField2", new byte[] {40, 21, 3}, 
VectorSimilarityFunction.DOT_PRODUCT));
+document2.add(
+new KnnByteVectorField(
+"knnByteField3", new by

[GitHub] [lucene] stefanvodita commented on issue #12601: Reproducible TestDrillSideways failure

2023-09-30 Thread via GitHub


stefanvodita commented on issue #12601:
URL: https://github.com/apache/lucene/issues/12601#issuecomment-1741757747

   Reverting #921 fixes the test, so I think this is the same issue that 
@Yuti-G investigated in #12418.
   I ran the test in verbose mode (`./gradlew test --tests 
TestDrillSideways.testRandom -Dtests.seed=CEBE5325E431003F -Dtests.verbose=true 
--max-workers=1`). Here's the info for the bit that's failing:
   
   ```
 1> TEST: iter=4 baseQuery=b numDrillDown=1 useSortedSetDV=true
 1>   dim3=[[e2 a1 9b e2 a2 b8 e2 a0 a1 e2 a1 97 e2 a1 a1 e2 a0 b4 e2 a2 91 
e2 a1 98 e2 a3 9e e2 a1 97 e2 a0 ab e2 a2 80 e2 a1 a0 e2 a0 86 e2 a0 9a]]
 1>   compute expected
 1> exp: id=2 is a near-miss on dim=3
 1>   verify all facets
 1>   verify totHits=0
 1> dim0 topN=2 (vs 2 unique values)
 1>   actual
 1>   expected (unsorted)
 1> dim1 topN=3 (vs 4 unique values)
 1>   actual
 1>   expected (sorted)
 1>   topN=3 expectedTopN=0
 1> dim2 topN=6 (vs 8 unique values)
 1>   actual
 1>   expected (sorted)
 1>   topN=6 expectedTopN=0
 1> dim3 topN=16 (vs 16 unique values)
 1>   actual
 1> 0: [c7 9c c7 a7 c7 be c6 ae]: 1
 1> 1: [ea a5 a4 ea a5 ac ea a5 a8 ea a5 af ea a5 ac ea a5 b1]: 1
 1>   expected (unsorted)
 1> 0: [c7 9c c7 a7 c7 be c6 ae]: 1
 1> 1: [f0 90 92 84]: 1
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org