uschindler commented on issue #12307:
URL: https://github.com/apache/lucene/issues/12307#issuecomment-1803320694
Hi,
this has nothing to do with the minimum Java version in your Java built, it
only has to do with the runtime Java version. If you use Java 19 or later, the
classloader need
gf2121 commented on PR #12784:
URL: https://github.com/apache/lucene/pull/12784#issuecomment-1803319176
I try this approach with `wikimedium10m` on the M2 mac, the sort took sum
decreased ~60%.
Details
https://bytedance.larkoffice.com/sheets/XfVCsZL5phx9letbDEQcaw0snnf";
dat
rmuir opened a new pull request, #12787:
URL: https://github.com/apache/lucene/pull/12787
This saves me a lot of time and prevents making bad changes that help some
cpus and hurt others.
Case in point: #12743
You run a command such as:
```
make PATCH_BRANCH=rmuir:some-spe
rmuir commented on PR #12743:
URL: https://github.com/apache/lucene/pull/12743#issuecomment-1803265618
speeds up as many machines as it slows down.
cascadelake: `['0', 'GenuineIntel', 'Intel(R) Xeon(R) Platinum 8275CL CPU @
3.00GHz', '1', 'GenuineIntel', 'Intel(R) Xeon(R) Platinum 827
rmuir closed pull request #12743: speedup arm int functions?
URL: https://github.com/apache/lucene/pull/12743
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-
gf2121 commented on PR #12784:
URL: https://github.com/apache/lucene/pull/12784#issuecomment-1803223146
As "reorder" gets faster, I'm considering lowering the fallback threshold
and letting radix sort do more of the work.
### Benchmark result:
MAC Intel
```
BASELINE: sor
zhaih commented on code in PR #12767:
URL: https://github.com/apache/lucene/pull/12767#discussion_r1387529653
##
lucene/core/src/test/org/apache/lucene/search/TestRegexpQuery.java:
##
@@ -80,7 +80,10 @@ private long caseInsensitiveRegexQueryNrHits(String regex)
throws IOExcepti
davido commented on issue #12307:
URL: https://github.com/apache/lucene/issues/12307#issuecomment-1803192152
@uschindler
We are using [Bazel](https://bazel.build) build system, and merging the two
JARs like this:
# Merge jars so
# META-INF/services/org.apache.lucene.code
LuXugang closed issue #12401: Skip docs with Docvalues in NumericLeafComparator
URL: https://github.com/apache/lucene/issues/12401
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment
LuXugang merged PR #12405:
URL: https://github.com/apache/lucene/pull/12405
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.ap
gf2121 commented on PR #12784:
URL: https://github.com/apache/lucene/pull/12784#issuecomment-1803144990
Even faster than the original approach on M2:
```
BASELINE: sort 5169965 terms, build histogram took: 489ms, reorder took:
1359ms, total took: 2381ms.
BASELINE: sort 5169965 ter
zacharymorn commented on PR #240:
URL: https://github.com/apache/lucene/pull/240#issuecomment-1803126121
> > We are still a ways away (from seeing Lucene fully utilize available
hardware concurrency available at search time to reduce query latencies)
>
> For example: query concurrency
zacharymorn commented on code in PR #240:
URL: https://github.com/apache/lucene/pull/240#discussion_r1387453543
##
lucene/core/src/test/org/apache/lucene/document/BaseSpatialTestCase.java:
##
@@ -695,8 +695,8 @@ protected void verifyRandomDistanceQueries(IndexReader
reader, Obj
zacharymorn commented on code in PR #240:
URL: https://github.com/apache/lucene/pull/240#discussion_r1387453179
##
lucene/core/src/java/org/apache/lucene/search/TopScoreDocCollector.java:
##
@@ -44,7 +43,7 @@ public void setScorer(Scorable scorer) throws IOException {
}
Re
zacharymorn commented on code in PR #240:
URL: https://github.com/apache/lucene/pull/240#discussion_r1387450681
##
lucene/core/src/java/org/apache/lucene/search/TopFieldCollector.java:
##
@@ -429,106 +432,29 @@ public static TopFieldCollector create(Sort sort, int
numHits, int
zacharymorn commented on code in PR #240:
URL: https://github.com/apache/lucene/pull/240#discussion_r1387449164
##
lucene/core/src/java/org/apache/lucene/search/TopFieldCollectorManager.java:
##
@@ -0,0 +1,198 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
zacharymorn commented on code in PR #240:
URL: https://github.com/apache/lucene/pull/240#discussion_r1387443316
##
lucene/core/src/java/org/apache/lucene/search/TopFieldCollector.java:
##
@@ -174,7 +173,7 @@ private static boolean canEarlyTerminateOnPrefix(Sort
searchSort, Sort
zacharymorn commented on code in PR #240:
URL: https://github.com/apache/lucene/pull/240#discussion_r1387438590
##
lucene/core/src/java/org/apache/lucene/search/TopDocs.java:
##
@@ -232,8 +232,8 @@ public static TopDocs merge(
/**
* Returns a new TopFieldDocs, containing
zacharymorn commented on code in PR #240:
URL: https://github.com/apache/lucene/pull/240#discussion_r1387436327
##
lucene/benchmark/src/java/org/apache/lucene/benchmark/byTask/tasks/SearchWithCollectorTask.java:
##
@@ -45,20 +43,6 @@ public boolean withCollector() {
return
gsmiller merged PR #12454:
URL: https://github.com/apache/lucene/pull/12454
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.ap
dungba88 commented on code in PR #12786:
URL: https://github.com/apache/lucene/pull/12786#discussion_r1387298308
##
lucene/core/src/java/org/apache/lucene/util/fst/NodeHash.java:
##
@@ -289,21 +273,38 @@ public long getNodeAddress(long hashSlot) {
}
/**
- * Set t
dungba88 commented on PR #12786:
URL: https://github.com/apache/lucene/pull/12786#issuecomment-1802854503
> Could not copy file
'/home/runner/work/lucene/lucene/lucene/JRE_VERSION_MIGRATION.md' to
'/home/runner/work/lucene/lucene/lucene/documentation/build/site/JRE_VERSION_MIGRATION.html'.
uschindler commented on PR #12785:
URL: https://github.com/apache/lucene/pull/12785#issuecomment-1802841410
The bug in JDK is here: https://bugs.openjdk.org/browse/JDK-8319756
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub an
dungba88 opened a new pull request, #12786:
URL: https://github.com/apache/lucene/pull/12786
### Description
See the previous PR: https://github.com/apache/lucene/pull/12778
There was a bug in the PR, the copiedNodeAddress is the last address
(inclusively) of the node, thus the
uschindler commented on PR #12707:
URL: https://github.com/apache/lucene/pull/12707#issuecomment-1802765955
This code did not work well as the `isAlive` status may be stale in other
threads. I reworked this one here: #12785
--
This is an automated message from the Apache Git Service.
To r
uschindler opened a new pull request, #12785:
URL: https://github.com/apache/lucene/pull/12785
Unfortunately the solution in #12707 was not working well with concurrency.
The is alive status of `MemorySegment.Scope` may be stale. In that case the
`IllegalStateException` was catched, but the
dungba88 commented on PR #12778:
URL: https://github.com/apache/lucene/pull/12778#issuecomment-1802753207
Thank you for reproducing this! I found the bug, it's quite silly. The node
address is the last address, so I should have do this
```
copiedNodes.append(fallbackTable.cop
benwtrent commented on issue #12615:
URL: https://github.com/apache/lucene/issues/12615#issuecomment-1802705393
@kevindrosendahl if I am reading the code correctly, it does the following:
- Write int8 quantized vectors along side the vector ordinals in the graph
(`.vex` or whatever h
gsmiller commented on PR #12454:
URL: https://github.com/apache/lucene/pull/12454#issuecomment-1802519348
> @gsmiller, I think this PR is ready. Is there anything else you'd like to
see changed?
Gah! I'm sorry I missed this. I'll have a look here shortly. Apologies again.
--
This i
benwtrent merged PR #12780:
URL: https://github.com/apache/lucene/pull/12780
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.a
gf2121 commented on PR #12775:
URL: https://github.com/apache/lucene/pull/12775#issuecomment-1802480151
I came up with https://github.com/apache/lucene/pull/12784 as another idea
to speed up `BytesRefHash#sort`, which has been shown to have performance
improvements running on Intel chips.
gf2121 opened a new pull request, #12784:
URL: https://github.com/apache/lucene/pull/12784
Following https://github.com/apache/lucene/pull/12775, this PR tries another
approach to speed up `BytesRefHash#sort`:
The idea is that since we have extra ints in this map, we can cache the
bucket
nitirajrathore commented on issue #12627:
URL: https://github.com/apache/lucene/issues/12627#issuecomment-1802424885
@benwtrent : I have added draft PR. The code is not at all optimized right
now for performance and I am hoping to fix some obvious stuff and will post
perf results here.
nitirajrathore opened a new pull request, #12783:
URL: https://github.com/apache/lucene/pull/12783
…onnected. No test cases, unoptimized, draft only version.
### Description
Details in this comment :
https://github.com/apache/lucene/issues/12627#issuecomment-1801982741
I wil
benwtrent commented on code in PR #12780:
URL: https://github.com/apache/lucene/pull/12780#discussion_r1386996238
##
lucene/test-framework/src/java/org/apache/lucene/tests/index/BaseKnnVectorsFormatTestCase.java:
##
@@ -81,7 +81,8 @@ public void init() {
protected void addRan
benwtrent commented on code in PR #12780:
URL: https://github.com/apache/lucene/pull/12780#discussion_r1386996018
##
lucene/test-framework/src/java/org/apache/lucene/tests/index/BaseKnnVectorsFormatTestCase.java:
##
@@ -81,7 +81,8 @@ public void init() {
protected void addRan
kevindrosendahl commented on code in PR #12780:
URL: https://github.com/apache/lucene/pull/12780#discussion_r1386992189
##
lucene/test-framework/src/java/org/apache/lucene/tests/index/BaseKnnVectorsFormatTestCase.java:
##
@@ -81,7 +81,8 @@ public void init() {
protected void
kevindrosendahl commented on code in PR #12780:
URL: https://github.com/apache/lucene/pull/12780#discussion_r1386992189
##
lucene/test-framework/src/java/org/apache/lucene/tests/index/BaseKnnVectorsFormatTestCase.java:
##
@@ -81,7 +81,8 @@ public void init() {
protected void
kevindrosendahl commented on code in PR #12780:
URL: https://github.com/apache/lucene/pull/12780#discussion_r1386980454
##
lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99ScalarQuantizedVectorsWriter.java:
##
@@ -196,6 +203,13 @@ void
writeSortedQuantizedVectors(
kevindrosendahl commented on code in PR #12780:
URL: https://github.com/apache/lucene/pull/12780#discussion_r1386980331
##
lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99ScalarQuantizedVectorsWriter.java:
##
@@ -143,6 +143,13 @@ private void
writeQuantizedVector
mikemccand commented on PR #12778:
URL: https://github.com/apache/lucene/pull/12778#issuecomment-1802219722
(And does not reproduce once I revert).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
mikemccand commented on PR #12778:
URL: https://github.com/apache/lucene/pull/12778#issuecomment-1802214263
And it does reproduce for me. I'll revert this change for now!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and us
mikemccand commented on PR #12778:
URL: https://github.com/apache/lucene/pull/12778#issuecomment-1802213131
Hmm running `./gradlew check` after pulling this change into my dev box I
hit:
```
org.apache.lucene.util.fst.TestFSTs > test suite's output saved to
/s1/l/trunk/lucene/core
mikemccand merged PR #12778:
URL: https://github.com/apache/lucene/pull/12778
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.
benwtrent commented on issue #12627:
URL: https://github.com/apache/lucene/issues/12627#issuecomment-1802098274
`68% increase` in index time is untenable, I would be a hard no on a change
that slows things down this much. Maybe we can find something better.
@nitirajrathore i know
htt
nitirajrathore commented on issue #12627:
URL: https://github.com/apache/lucene/issues/12627#issuecomment-1801982741
I was able to conduct some perf-test as well. I would like to propose
following changes
With example of adding node `a` to 'b, c, d'
| node | neighbours|
|---|---
mikemccand merged PR #12781:
URL: https://github.com/apache/lucene/pull/12781
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.
dungba88 commented on code in PR #12778:
URL: https://github.com/apache/lucene/pull/12778#discussion_r1386637002
##
lucene/core/src/java/org/apache/lucene/util/fst/NodeHash.java:
##
@@ -289,21 +273,34 @@ public long getNodeAddress(long hashSlot) {
}
/**
- * Set t
easyice opened a new pull request, #12782:
URL: https://github.com/apache/lucene/pull/12782
As discussed in issue https://github.com/apache/lucene/issues/12717
the read performance of group-varint is 14-30%% faster than vint, the
`Mode` 16-248 is the number of ints will be read.
benwtrent commented on code in PR #12780:
URL: https://github.com/apache/lucene/pull/12780#discussion_r1386577696
##
lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99ScalarQuantizedVectorsWriter.java:
##
@@ -143,6 +143,13 @@ private void
writeQuantizedVectors(Quan
dungba88 commented on PR #12715:
URL: https://github.com/apache/lucene/pull/12715#issuecomment-1801792656
@mikemccand can you help to review this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
gf2121 commented on PR #12775:
URL: https://github.com/apache/lucene/pull/12775#issuecomment-1801756914
> it's fine if this yields a speedup on some platforms but not other
platforms
+1
> it doesn't seem to slow down flushing either?
Yes!
--
This is an automated messa
jpountz commented on PR #12775:
URL: https://github.com/apache/lucene/pull/12775#issuecomment-1801748121
FWIW I think it's fine if this yields a speedup on some platforms but not
other platforms. From your results, it doesn't seem to slow down flushing
either?
--
This is an automated mes
mikemccand commented on code in PR #12778:
URL: https://github.com/apache/lucene/pull/12778#discussion_r1386504215
##
lucene/core/src/java/org/apache/lucene/util/fst/NodeHash.java:
##
@@ -289,21 +273,34 @@ public long getNodeAddress(long hashSlot) {
}
/**
- * Set
slow-J opened a new pull request, #12781:
URL: https://github.com/apache/lucene/pull/12781
Clean-up from adding the Lucene99PostingsFormat in
https://github.com/apache/lucene/pull/12741
These test cases were moved to Lucene99 test directory and I forgot to copy
the unmodified version
mikemccand commented on PR #240:
URL: https://github.com/apache/lucene/pull/240#issuecomment-1801638506
> We are still a ways away
For example: query concurrency is still tied to segment geometry, which is
insane. An "optimized" (`forceMerge to 1 segment`) index loses all of its
con
mikemccand commented on code in PR #240:
URL: https://github.com/apache/lucene/pull/240#discussion_r1386407230
##
lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java:
##
@@ -602,34 +579,10 @@ private TopFieldDocs searchAfter(
final Sort rewrittenSort = sort.rew
dantuzi commented on PR #12433:
URL: https://github.com/apache/lucene/pull/12433#issuecomment-1801394824
Thanks @mikemccand for your feedback.
I had to address some comments from @alessandrobenedetti, that's why this PR
is still WIP.
At the moment I have other priorities at work but I'
dweiss closed issue #12654: TestIndexWriterOnVMError.testUnknownError times out
(potential IndexWriter deadlock with tragic exceptions)
URL: https://github.com/apache/lucene/issues/12654
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to G
dweiss merged PR #12751:
URL: https://github.com/apache/lucene/pull/12751
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apac
60 matches
Mail list logo
