boris-pichugin commented on PR #12827:
URL: https://github.com/apache/lucene/pull/12827#issuecomment-1820444693
I've got the same problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the spec
gf2121 commented on PR #12800:
URL: https://github.com/apache/lucene/pull/12800#issuecomment-1820461507
I did some more work to find out the balance between memory / performance in
various data distribution. The way i'm thinking now is that we keep the
timsorter here, but make the run lengt
gf2121 commented on code in PR #12800:
URL: https://github.com/apache/lucene/pull/12800#discussion_r1400210595
##
lucene/core/src/java/org/apache/lucene/util/BaseLSBRadixSorter.java:
##
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
gf2121 commented on PR #12800:
URL: https://github.com/apache/lucene/pull/12800#issuecomment-1820564110
I also run the index script to see flush time with this new approach, result
in ~15% faster for random data and no regression on asc/desc :)
Benchmark Detail
**Baseline**
jpountz commented on PR #12622:
URL: https://github.com/apache/lucene/pull/12622#issuecomment-1820579253
@s1monw Could you take another look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the s
s1monw commented on code in PR #12622:
URL: https://github.com/apache/lucene/pull/12622#discussion_r1400320980
##
lucene/core/src/java/org/apache/lucene/index/IndexWriter.java:
##
@@ -3475,6 +3475,8 @@ public void addIndexesReaderMerge(MergePolicy.OneMerge
merge) throws IOExcep
mikemccand commented on PR #12800:
URL: https://github.com/apache/lucene/pull/12800#issuecomment-1820626033
> I also run the index script to see flush time with this new approach,
result in ~15% faster for random data and no regression on asc/desc :)
Hmm it looks like random got a bit
mikemccand merged PR #12827:
URL: https://github.com/apache/lucene/pull/12827
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.
s1monw opened a new pull request, #12829:
URL: https://github.com/apache/lucene/pull/12829
Today index sorting with likely break document blocks added with
`IndexWriter#addDocuments(...)` and friends since the index sorter has no
indication of what documents are part of a block. This change
javanna commented on code in PR #12799:
URL: https://github.com/apache/lucene/pull/12799#discussion_r1400378967
##
lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99HnswVectorsFormat.java:
##
@@ -160,12 +160,12 @@ public Lucene99HnswVectorsFormat(int maxConn, int
b
s1monw commented on PR #12711:
URL: https://github.com/apache/lucene/pull/12711#issuecomment-1820666090
@mikemccand @msokolov @jpountz @martijnvg see #12829 for reference and
further discussions
--
This is an automated message from the Apache Git Service.
To respond to the message, plea
javanna commented on PR #12798:
URL: https://github.com/apache/lucene/pull/12798#issuecomment-1820666255
Is this in fact superseded by #12799 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
s1monw closed pull request #12711: Prevent users from using document block APIs
when sort is configured
URL: https://github.com/apache/lucene/pull/12711
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
gf2121 commented on PR #12800:
URL: https://github.com/apache/lucene/pull/12800#issuecomment-1820687175
Thanks for feedback @mikemccand !
> Hmm it looks like random got a bit slower in candidate? Flush time ~550
ish ms in baseline and maybe ~650 ish ms in candidate?
Ohhh! I rec
shubhamvishu commented on code in PR #12799:
URL: https://github.com/apache/lucene/pull/12799#discussion_r1400409812
##
lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99HnswVectorsFormat.java:
##
@@ -160,12 +160,12 @@ public Lucene99HnswVectorsFormat(int maxConn, i
shubhamvishu commented on code in PR #12799:
URL: https://github.com/apache/lucene/pull/12799#discussion_r1400409812
##
lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99HnswVectorsFormat.java:
##
@@ -160,12 +160,12 @@ public Lucene99HnswVectorsFormat(int maxConn, i
mikemccand commented on PR #12750:
URL: https://github.com/apache/lucene/pull/12750#issuecomment-1820713053
This looks like the root cause?:
```
java.lang.AssertionError: inconsistent endOffset 1 pos=0 posLen=1
token=/a/b expected:<2> but was:<4>
```
Indeed I think
shubhamvishu commented on PR #12798:
URL: https://github.com/apache/lucene/pull/12798#issuecomment-1820748202
> Is this in fact superseded by https://github.com/apache/lucene/pull/12799 ?
Yes, there would be a conflict `HnswConcurrentMergeBuilder` class due to
#12799 which we could re
mikemccand commented on code in PR #12797:
URL: https://github.com/apache/lucene/pull/12797#discussion_r1400425827
##
lucene/core/src/java/org/apache/lucene/index/CheckIndex.java:
##
@@ -442,19 +442,19 @@ public void close() throws IOException {
IOUtils.close(writeLock);
mikemccand merged PR #12814:
URL: https://github.com/apache/lucene/pull/12814
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.
javanna commented on code in PR #12799:
URL: https://github.com/apache/lucene/pull/12799#discussion_r1400536223
##
lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99HnswVectorsFormat.java:
##
@@ -160,12 +160,12 @@ public Lucene99HnswVectorsFormat(int maxConn, int
b
javanna commented on code in PR #12799:
URL: https://github.com/apache/lucene/pull/12799#discussion_r1400536223
##
lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99HnswVectorsFormat.java:
##
@@ -160,12 +160,12 @@ public Lucene99HnswVectorsFormat(int maxConn, int
b
javanna commented on PR #240:
URL: https://github.com/apache/lucene/pull/240#issuecomment-1820848611
That is fine with me @zacharymorn . Indeed I have observed as well that
there will be new usages introduced while we work on removing current usages,
and deprecating early can help with that
dungba88 commented on code in PR #12624:
URL: https://github.com/apache/lucene/pull/12624#discussion_r1395333947
##
lucene/core/src/java/org/apache/lucene/util/fst/GrowableByteArrayDataOutput.java:
##
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) und
mikemccand merged PR #12802:
URL: https://github.com/apache/lucene/pull/12802
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.
jpountz commented on issue #12826:
URL: https://github.com/apache/lucene/issues/12826#issuecomment-1820899056
For reference, another place where I'm interested in seeing whether we can
use group-varint is `SortingPostingsEnum#addPositions` /
`SortingPostingsEnum#nextPosition`. This is curre
mikemccand commented on code in PR #12624:
URL: https://github.com/apache/lucene/pull/12624#discussion_r1400539919
##
lucene/core/src/java/org/apache/lucene/util/fst/ByteBuffersFSTReader.java:
##
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
shubhamvishu commented on code in PR #12799:
URL: https://github.com/apache/lucene/pull/12799#discussion_r1400609546
##
lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99HnswVectorsFormat.java:
##
@@ -160,12 +160,12 @@ public Lucene99HnswVectorsFormat(int maxConn, i
dungba88 commented on code in PR #12624:
URL: https://github.com/apache/lucene/pull/12624#discussion_r1400633005
##
lucene/core/src/java/org/apache/lucene/util/fst/GrowableByteArrayDataOutput.java:
##
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) unde
dungba88 commented on code in PR #12624:
URL: https://github.com/apache/lucene/pull/12624#discussion_r1400634116
##
lucene/core/src/java/org/apache/lucene/util/fst/FST.java:
##
@@ -435,6 +433,13 @@ public FST(FSTMetadata metadata, DataInput in,
Outputs outputs, FSTStore f
dungba88 commented on code in PR #12624:
URL: https://github.com/apache/lucene/pull/12624#discussion_r1400638946
##
lucene/core/src/java/org/apache/lucene/util/fst/FSTCompiler.java:
##
@@ -153,6 +176,34 @@ private FSTCompiler(
}
}
+ // Get the respective FSTReader of
dungba88 commented on code in PR #12624:
URL: https://github.com/apache/lucene/pull/12624#discussion_r1400641544
##
lucene/core/src/java/org/apache/lucene/util/fst/FSTCompiler.java:
##
@@ -248,15 +305,17 @@ public Builder
directAddressingMaxOversizingFactor(float factor) {
dungba88 commented on code in PR #12624:
URL: https://github.com/apache/lucene/pull/12624#discussion_r1400642484
##
lucene/core/src/java/org/apache/lucene/util/fst/FSTCompiler.java:
##
@@ -277,9 +336,9 @@ public long getMappedStateCount() {
return dedupHash == null ? 0 : no
dungba88 commented on code in PR #12624:
URL: https://github.com/apache/lucene/pull/12624#discussion_r1400663661
##
lucene/core/src/java/org/apache/lucene/util/fst/FSTCompiler.java:
##
@@ -827,22 +910,24 @@ void setEmptyOutput(T v) {
}
void finish(long newStartNode) {
-
dungba88 commented on code in PR #12624:
URL: https://github.com/apache/lucene/pull/12624#discussion_r1400638946
##
lucene/core/src/java/org/apache/lucene/util/fst/FSTCompiler.java:
##
@@ -153,6 +176,34 @@ private FSTCompiler(
}
}
+ // Get the respective FSTReader of
dungba88 commented on code in PR #12624:
URL: https://github.com/apache/lucene/pull/12624#discussion_r1400638946
##
lucene/core/src/java/org/apache/lucene/util/fst/FSTCompiler.java:
##
@@ -153,6 +176,34 @@ private FSTCompiler(
}
}
+ // Get the respective FSTReader of
dungba88 commented on code in PR #12624:
URL: https://github.com/apache/lucene/pull/12624#discussion_r1400687529
##
lucene/core/src/java/org/apache/lucene/util/fst/ByteBuffersFSTReader.java:
##
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one o
dungba88 commented on code in PR #12624:
URL: https://github.com/apache/lucene/pull/12624#discussion_r1400633005
##
lucene/core/src/java/org/apache/lucene/util/fst/GrowableByteArrayDataOutput.java:
##
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) unde
javanna commented on PR #12799:
URL: https://github.com/apache/lucene/pull/12799#issuecomment-1821249014
Thanks @shubhamvishu !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific commen
javanna merged PR #12799:
URL: https://github.com/apache/lucene/pull/12799
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apa
msokolov commented on issue #12725:
URL: https://github.com/apache/lucene/issues/12725#issuecomment-1821297551
@SreehariG73 we generally don't assign issues here, but if you have a
contribution to make, it would certainly be welcome
--
This is an automated message from the Apache Git Serv
easyice commented on issue #12826:
URL: https://github.com/apache/lucene/issues/12826#issuecomment-1821299530
I ran the benchmark with Java 21 and using `MemorySegmentIndexInput`, but it
seems slower than current, The `Score` is very similar to the code i wrote
earlier :) The
[code](https
msokolov commented on PR #12819:
URL: https://github.com/apache/lucene/pull/12819#issuecomment-1821304564
Agree we should try to preserve this; could we pass it along to the
DocAndScoreQuery? I don't know what the profiling query is?
--
This is an automated message from the Apache Git Se
jpountz commented on PR #12819:
URL: https://github.com/apache/lucene/pull/12819#issuecomment-1821309808
I was referring to all the `*Profile*` classes we have under
`lucene/sandbox/src/java/org/apache/lucene/sandbox/search/`. These classes
already have logic to track how many times `nextDo
jdyer1 closed pull request #363: SOLR-12276
URL: https://github.com/apache/lucene-solr/pull/363
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-
itygh commented on PR #363:
URL: https://github.com/apache/lucene-solr/pull/363#issuecomment-1821333496
这是来自QQ邮箱的假期自动回复邮件。您好，我最近正在休假中，无法亲自回复您的邮件。我将在假期结束后，尽快给您回复。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
mikemccand commented on PR #12699:
URL: https://github.com/apache/lucene/pull/12699#issuecomment-1821362519
I'll try to review the latest PR soon -- thanks @gf2121.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
U
jpountz commented on issue #12826:
URL: https://github.com/apache/lucene/issues/12826#issuecomment-1821390184
I updated the benchmark to interleave numbers of bytes for each integer,
plus added a couple minor optimizations to the decoding logic, and I'm now
seeing better performance for the
jpountz commented on issue #12826:
URL: https://github.com/apache/lucene/issues/12826#issuecomment-1821396781
Hmm I had misread your benchmark and just realized that the new impl was
already faster when numBytesPerInt is 0. I'm not sure we should care too much
about other values of `numByte
slow-J commented on code in PR #12797:
URL: https://github.com/apache/lucene/pull/12797#discussion_r1400997709
##
lucene/MIGRATE.md:
##
@@ -101,6 +101,13 @@ The deprecated getter for the `Executor` that was
optionally provided to the `In
has been removed. Users that want to ex
slow-J commented on code in PR #12797:
URL: https://github.com/apache/lucene/pull/12797#discussion_r1400999774
##
lucene/core/src/java/org/apache/lucene/index/CheckIndex.java:
##
@@ -2479,15 +2462,11 @@ public static Status.TermIndexStatus
testPostings(CodecReader reader, Print
slow-J commented on PR #12797:
URL: https://github.com/apache/lucene/pull/12797#issuecomment-1821441651
Thanks for the review @mikemccand, will respond to the rest of the comments
later!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
slow-J commented on code in PR #12797:
URL: https://github.com/apache/lucene/pull/12797#discussion_r1401175756
##
lucene/core/src/java/org/apache/lucene/index/CheckIndex.java:
##
@@ -3661,7 +3640,7 @@ private static void checkDocValues(
*/
public static Status.TermVectorS
slow-J commented on code in PR #12797:
URL: https://github.com/apache/lucene/pull/12797#discussion_r1401195063
##
lucene/core/src/java/org/apache/lucene/index/CheckIndex.java:
##
@@ -4127,15 +4123,33 @@ public static Options parseOptions(String[] args) {
int i = 0;
whi
slow-J commented on code in PR #12797:
URL: https://github.com/apache/lucene/pull/12797#discussion_r1401212917
##
lucene/core/src/java/org/apache/lucene/index/CheckIndex.java:
##
@@ -4191,7 +4207,8 @@ public static Options parseOptions(String[] args) {
+ "If no pa
slow-J commented on code in PR #12797:
URL: https://github.com/apache/lucene/pull/12797#discussion_r1400997736
##
lucene/core/src/java/org/apache/lucene/index/CheckIndex.java:
##
@@ -442,19 +442,19 @@ public void close() throws IOException {
IOUtils.close(writeLock);
}
jpountz commented on code in PR #12797:
URL: https://github.com/apache/lucene/pull/12797#discussion_r1401280939
##
lucene/core/src/java/org/apache/lucene/index/CheckIndex.java:
##
@@ -2382,7 +2365,7 @@ private static Status.TermIndexStatus checkFields(
static void checkImpact
msokolov commented on code in PR #12829:
URL: https://github.com/apache/lucene/pull/12829#discussion_r1401297534
##
lucene/core/src/test/org/apache/lucene/index/TestIndexSorting.java:
##
@@ -3173,4 +3173,184 @@ public void
testSortDocsAndFreqsAndPositionsAndOffsets() throws IOE
dungba88 opened a new pull request, #12830:
URL: https://github.com/apache/lucene/pull/12830
### Description
Spawn out of #12624 . This PR make FSTCompiler.Builder to throw IOException
as required by the other PR and avoid large diffs. These 2 PRs can be merged in
any order.
--
Th
dungba88 commented on code in PR #12624:
URL: https://github.com/apache/lucene/pull/12624#discussion_r1400641544
##
lucene/core/src/java/org/apache/lucene/util/fst/FSTCompiler.java:
##
@@ -248,15 +305,17 @@ public Builder
directAddressingMaxOversizingFactor(float factor) {
stefanvodita commented on issue #12585:
URL: https://github.com/apache/lucene/issues/12585#issuecomment-1821880175
I thought some more about this issue and it really seems like a bug that I
can have a non-positive aggregation value, but I can't return it in top
children.
If we were to ad
dungba88 commented on code in PR #12624:
URL: https://github.com/apache/lucene/pull/12624#discussion_r1400663661
##
lucene/core/src/java/org/apache/lucene/util/fst/FSTCompiler.java:
##
@@ -827,22 +910,24 @@ void setEmptyOutput(T v) {
}
void finish(long newStartNode) {
-
dungba88 commented on code in PR #12624:
URL: https://github.com/apache/lucene/pull/12624#discussion_r1400633005
##
lucene/core/src/java/org/apache/lucene/util/fst/GrowableByteArrayDataOutput.java:
##
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) unde
dungba88 commented on code in PR #12624:
URL: https://github.com/apache/lucene/pull/12624#discussion_r1400663661
##
lucene/core/src/java/org/apache/lucene/util/fst/FSTCompiler.java:
##
@@ -827,22 +910,24 @@ void setEmptyOutput(T v) {
}
void finish(long newStartNode) {
-
dungba88 commented on code in PR #12624:
URL: https://github.com/apache/lucene/pull/12624#discussion_r1400663661
##
lucene/core/src/java/org/apache/lucene/util/fst/FSTCompiler.java:
##
@@ -827,22 +910,24 @@ void setEmptyOutput(T v) {
}
void finish(long newStartNode) {
-
65 matches
Mail list logo
