[GitHub] [lucene] jpountz commented on a diff in pull request #11840: GITHUB-11838 Add api to allow concurrent query rewrite

2022-10-12 Thread GitBox
jpountz commented on code in PR #11840: URL: https://github.com/apache/lucene/pull/11840#discussion_r994219709 ## lucene/core/src/java/org/apache/lucene/search/FieldExistsQuery.java: ## @@ -18,16 +18,7 @@ import java.io.IOException; import java.util.Objects; -import org.apac

[GitHub] [lucene] zhaih commented on pull request #11840: GITHUB-11838 Add api to allow concurrent query rewrite

2022-10-12 Thread GitBox
zhaih commented on PR #11840: URL: https://github.com/apache/lucene/pull/11840#issuecomment-1277038263 @jpountz You're right, our case is a bit complex since currently we're not even using Lucene's Query (but we're planning to in the future!) so I totally forgot the createWeight also takes

[GitHub] [lucene] jtibshirani commented on pull request #11843: Remove cancellation check on every vector

2022-10-12 Thread GitBox
jtibshirani commented on PR #11843: URL: https://github.com/apache/lucene/pull/11843#issuecomment-1276520789 To give some context, Elasticsearch exposes a query type that performs a kNN exact scan. It iterates through all the `VectorValues` matching a query, and computes the similarity. Sin

[GitHub] [lucene] dweiss commented on pull request #11846: WrapperDownloader: add retries for network blips around connect(), too

2022-10-12 Thread GitBox
dweiss commented on PR #11846: URL: https://github.com/apache/lucene/pull/11846#issuecomment-1276505202 LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

[GitHub] [lucene] rmuir opened a new pull request, #11846: WrapperDownloader: add retries for network blips around connect(), too

2022-10-12 Thread GitBox
rmuir opened a new pull request, #11846: URL: https://github.com/apache/lucene/pull/11846 Add retries for common issues such as connect timeout, etc. This won't solve the problem of read-timeouts happening around the actual transferTo, but it is an easy incremental improvement.

[GitHub] [lucene] rmuir opened a new issue, #11845: WrapperDownloader should retry on Layer3/Layer4 network errors

2022-10-12 Thread GitBox
rmuir opened a new issue, #11845: URL: https://github.com/apache/lucene/issues/11845 ### Description WrapperDownloader became more stable in CI after #11766 added Layer7 retries for HTTP 5xx. But I think we should also retry on lower-level network failures (e.g. DNS failure, c

[GitHub] [lucene] rmuir opened a new pull request, #11844: Mark TestLongBitSet.testHugeCapacity @Monster as it requires a lot of memory

2022-10-12 Thread GitBox
rmuir opened a new pull request, #11844: URL: https://github.com/apache/lucene/pull/11844 This test case needs hundreds of MB of heap, and causes OOMs in nightly builds. Let's mark it `@Monster` appropriately. Closes #11842 -- This is an automated message from the Apache Git Servic

[GitHub] [lucene] rmuir commented on issue #11842: TestLongBitSet.testHugeCapacity OOM

2022-10-12 Thread GitBox
rmuir commented on issue #11842: URL: https://github.com/apache/lucene/issues/11842#issuecomment-1276409926 The test allocates a bitset exceeding length of 2^31, so we can expect it to need hundreds of megabytes of heap space. I think rather than mark the test `@Nightly` we should ch

[GitHub] [lucene] mikemccand commented on pull request #11780: GH#11601: Add ability to compute reader states after refresh

2022-10-12 Thread GitBox
mikemccand commented on PR #11780: URL: https://github.com/apache/lucene/pull/11780#issuecomment-1276316505 > 3\. Allow the user to update the ordinal maps in the reader states they already have without requiring them to completely recreate the reader states. I’m not sure how much this acco

[GitHub] [lucene] benwtrent opened a new pull request, #11843: Remove cancellation check on every vector

2022-10-12 Thread GitBox
benwtrent opened a new pull request, #11843: URL: https://github.com/apache/lucene/pull/11843 PR: https://github.com/apache/lucene/pull/833 helpfully introduced query cancellation checks for KNN vector queries. However, checking for cancellation on every vector read has a negative i

[GitHub] [lucene] rmuir opened a new issue, #11842: TestLongBitSet.testHugeCapacity OOM

2022-10-12 Thread GitBox
rmuir opened a new issue, #11842: URL: https://github.com/apache/lucene/issues/11842 ### Description Failed on 9.x nightly build here: https://ci-builds.apache.org/job/Lucene/job/Lucene-NightlyTests-9.x/338/console ``` org.apache.lucene.util.TestLongBitSet > testHugeCapacity

[GitHub] [lucene] jpountz commented on pull request #11840: GITHUB-11838 Add api to allow concurrent query rewrite

2022-10-12 Thread GitBox
jpountz commented on PR #11840: URL: https://github.com/apache/lucene/pull/11840#issuecomment-1276029374 I don't think it'd be weird to require an `IndexSearcher`, `Query#rewrite` is essentially a way to improve caching and keep `Query#createWeight` simple. Given that `Query#createWeight` a

[GitHub] [lucene] donnerpeter commented on issue #11459: Remove uses of wall-clock time in codebase [LUCENE-10423]

2022-10-12 Thread GitBox
donnerpeter commented on issue #11459: URL: https://github.com/apache/lucene/issues/11459#issuecomment-1275869750 I've found a mistake in conversion made during these changes (missing parentheses and wrong zero count). I've fixed it where I need it (https://github.com/apache/lucene/commit/a

[GitHub] [lucene] zhaih commented on pull request #11840: GITHUB-11838 Add api to allow concurrent query rewrite

2022-10-12 Thread GitBox
zhaih commented on PR #11840: URL: https://github.com/apache/lucene/pull/11840#issuecomment-1275696638 Hi @jpountz thank for taking a look! > We already have one class that wraps an IndexReader and an Executor: IndexSearcher. Should this new rewrite method take an IndexSearcher instead of