[GitHub] [lucene] zhaih commented on pull request #11840: GITHUB-11838 Add api to allow concurrent query rewrite

zhaih commented on PR #11840: URL: https://github.com/apache/lucene/pull/11840#issuecomment-1275696638 Hi @jpountz thank for taking a look! > We already have one class that wraps an IndexReader and an Executor: IndexSearcher. Should this new rewrite method take an IndexSearcher instead of

[GitHub] [lucene] donnerpeter commented on issue #11459: Remove uses of wall-clock time in codebase [LUCENE-10423]

donnerpeter commented on issue #11459: URL: https://github.com/apache/lucene/issues/11459#issuecomment-1275869750 I've found a mistake in conversion made during these changes (missing parentheses and wrong zero count). I've fixed it where I need it (https://github.com/apache/lucene/commit/a

[GitHub] [lucene] jpountz commented on pull request #11840: GITHUB-11838 Add api to allow concurrent query rewrite

jpountz commented on PR #11840: URL: https://github.com/apache/lucene/pull/11840#issuecomment-1276029374 I don't think it'd be weird to require an `IndexSearcher`, `Query#rewrite` is essentially a way to improve caching and keep `Query#createWeight` simple. Given that `Query#createWeight` a

[GitHub] [lucene] rmuir opened a new issue, #11842: TestLongBitSet.testHugeCapacity OOM

rmuir opened a new issue, #11842: URL: https://github.com/apache/lucene/issues/11842 ### Description Failed on 9.x nightly build here: https://ci-builds.apache.org/job/Lucene/job/Lucene-NightlyTests-9.x/338/console ``` org.apache.lucene.util.TestLongBitSet > testHugeCapacity

[GitHub] [lucene] benwtrent opened a new pull request, #11843: Remove cancellation check on every vector

benwtrent opened a new pull request, #11843: URL: https://github.com/apache/lucene/pull/11843 PR: https://github.com/apache/lucene/pull/833 helpfully introduced query cancellation checks for KNN vector queries. However, checking for cancellation on every vector read has a negative i

[GitHub] [lucene] mikemccand commented on pull request #11780: GH#11601: Add ability to compute reader states after refresh

mikemccand commented on PR #11780: URL: https://github.com/apache/lucene/pull/11780#issuecomment-1276316505 > 3\. Allow the user to update the ordinal maps in the reader states they already have without requiring them to completely recreate the reader states. I’m not sure how much this acco

[GitHub] [lucene] rmuir commented on issue #11842: TestLongBitSet.testHugeCapacity OOM

rmuir commented on issue #11842: URL: https://github.com/apache/lucene/issues/11842#issuecomment-1276409926 The test allocates a bitset exceeding length of 2^31, so we can expect it to need hundreds of megabytes of heap space. I think rather than mark the test `@Nightly` we should ch

[GitHub] [lucene] rmuir opened a new pull request, #11844: Mark TestLongBitSet.testHugeCapacity @Monster as it requires a lot of memory

rmuir opened a new pull request, #11844: URL: https://github.com/apache/lucene/pull/11844 This test case needs hundreds of MB of heap, and causes OOMs in nightly builds. Let's mark it `@Monster` appropriately. Closes #11842 -- This is an automated message from the Apache Git Servic

[GitHub] [lucene] rmuir opened a new issue, #11845: WrapperDownloader should retry on Layer3/Layer4 network errors

rmuir opened a new issue, #11845: URL: https://github.com/apache/lucene/issues/11845 ### Description WrapperDownloader became more stable in CI after #11766 added Layer7 retries for HTTP 5xx. But I think we should also retry on lower-level network failures (e.g. DNS failure, c

[GitHub] [lucene] rmuir opened a new pull request, #11846: WrapperDownloader: add retries for network blips around connect(), too

rmuir opened a new pull request, #11846: URL: https://github.com/apache/lucene/pull/11846 Add retries for common issues such as connect timeout, etc. This won't solve the problem of read-timeouts happening around the actual transferTo, but it is an easy incremental improvement.

[GitHub] [lucene] dweiss commented on pull request #11846: WrapperDownloader: add retries for network blips around connect(), too

dweiss commented on PR #11846: URL: https://github.com/apache/lucene/pull/11846#issuecomment-1276505202 LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

[GitHub] [lucene] jtibshirani commented on pull request #11843: Remove cancellation check on every vector

jtibshirani commented on PR #11843: URL: https://github.com/apache/lucene/pull/11843#issuecomment-1276520789 To give some context, Elasticsearch exposes a query type that performs a kNN exact scan. It iterates through all the `VectorValues` matching a query, and computes the similarity. Sin

[GitHub] [lucene] zhaih commented on pull request #11840: GITHUB-11838 Add api to allow concurrent query rewrite

zhaih commented on PR #11840: URL: https://github.com/apache/lucene/pull/11840#issuecomment-1277038263 @jpountz You're right, our case is a bit complex since currently we're not even using Lucene's Query (but we're planning to in the future!) so I totally forgot the createWeight also takes

[GitHub] [lucene] jpountz commented on a diff in pull request #11840: GITHUB-11838 Add api to allow concurrent query rewrite

jpountz commented on code in PR #11840: URL: https://github.com/apache/lucene/pull/11840#discussion_r994219709 ## lucene/core/src/java/org/apache/lucene/search/FieldExistsQuery.java: ## @@ -18,16 +18,7 @@ import java.io.IOException; import java.util.Objects; -import org.apac