Re: [I] gradle-wrapper.jar will not be updated when its sha/version changes [lucene]
dweiss commented on issue #14598: URL: https://github.com/apache/lucene/issues/14598#issuecomment-2849301978 https://github.com/apache/lucene/pull/14608 This adds sha256 check on Windows and Linux. The Linux part is a redesigned version of what chatgpt came up with. I believe even chatgpt fried its ai brains when it had to deal with cmd syntax... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [PR] Always check gradle wrapper sha checksum and download if necessary [lucene]
dweiss commented on PR #14608: URL: https://github.com/apache/lucene/pull/14608#issuecomment-2849304533 I've verified this works on Windows, Linux (ubuntu) and Mac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [PR] Update created version major [lucene]
rahulgoswami commented on PR #14607: URL: https://github.com/apache/lucene/pull/14607#issuecomment-2849305608 > As states on the dev list, I don't think we should do this. Can you please elaborate why? I have tried to address the concern you brought up on the mailing list via the implementation in this PR. The primary premise behind the API is that IF all segments of an index are created by the LATEST version, the index in all respects is LATEST. "indexCreatedVersionMajor" should ideally not block a Lucene upgrade in that case. Happy to learn if I am missing anything. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [PR] Always check gradle wrapper sha checksum and download if necessary [lucene]
dweiss commented on code in PR #14608:
URL: https://github.com/apache/lucene/pull/14608#discussion_r2072657950
##
gradlew.bat:
##
@@ -81,10 +81,22 @@ SET DEFAULT_JVM_OPTS=%DEFAULT_JVM_OPTS%
"-Djava.io.tmpdir=%GRADLE_TEMPDIR%"
@rem LUCENE-9266: verify and download the gradle wrapper jar if we don't have
one.
set GRADLE_WRAPPER_JAR=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar
-IF NOT EXIST "%GRADLE_WRAPPER_JAR%" (
-"%JAVA_EXE%" %JAVA_OPTS%
"%APP_HOME%/build-tools/build-infra/src/main/java/org/apache/lucene/gradle/WrapperDownloader.java"
"%GRADLE_WRAPPER_JAR%"
-IF %ERRORLEVEL% EQU 1 goto failWithJvmMessage
-IF %ERRORLEVEL% NEQ 0 goto fail
+set GRADLE_WRAPPER_CHECKSUM=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar.sha256
+
+@rem Read the expected hash from .sha256 file
+for /f "tokens=1" %%A in (%GRADLE_WRAPPER_CHECKSUM%) do (
+set "EXPECTED=%%A"
+)
Review Comment:
the assumption here is that the sha file only contains the
gradle-wrapper.jar checksum and it's sane. If anything doesn't match/ work
here, we fall back to the Java version in WrapperDownloader anyway, where
things are checked in depth.
##
gradlew.bat:
##
@@ -81,10 +81,22 @@ SET DEFAULT_JVM_OPTS=%DEFAULT_JVM_OPTS%
"-Djava.io.tmpdir=%GRADLE_TEMPDIR%"
@rem LUCENE-9266: verify and download the gradle wrapper jar if we don't have
one.
set GRADLE_WRAPPER_JAR=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar
-IF NOT EXIST "%GRADLE_WRAPPER_JAR%" (
-"%JAVA_EXE%" %JAVA_OPTS%
"%APP_HOME%/build-tools/build-infra/src/main/java/org/apache/lucene/gradle/WrapperDownloader.java"
"%GRADLE_WRAPPER_JAR%"
-IF %ERRORLEVEL% EQU 1 goto failWithJvmMessage
-IF %ERRORLEVEL% NEQ 0 goto fail
+set GRADLE_WRAPPER_CHECKSUM=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar.sha256
+
+@rem Read the expected hash from .sha256 file
+for /f "tokens=1" %%A in (%GRADLE_WRAPPER_CHECKSUM%) do (
+set "EXPECTED=%%A"
+)
+@rem Get actual SHA-256 hash using certutil
+for /f "tokens=* delims=" %%H in ('certutil -hashfile "%GRADLE_WRAPPER_JAR%"
SHA256 ^| findstr /R /B /I /X "[0-9a-f]*"') do (
+set "ACTUAL=%%H"
+)
Review Comment:
this computes sha256 using certutil, then tries to pick a line that looks
like an sha256 checksum. Again - if something fails, we rely on the Java
version to tell us what's wrong.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [PR] Add Query for reranking KnnFloatVectorQuery with full-precision vectors [lucene]
dungba88 commented on PR #14009: URL: https://github.com/apache/lucene/pull/14009#issuecomment-2849835324 > I'm curious about https://github.com/apache/lucene/pull/14009#issuecomment-2502665806 -- why is recall better for 1bit and 4bit than 7bit, when reranking? If we compare the recall with the same oversample, then 7-bit is always better or same. The difference becomes smaller at higher oversample. E.g, at oversample=1 , 7 bit has 20% higher recall than 1-bit but at oversample=1 they are mostly the same. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [I] gradle-wrapper.jar will not be updated when its sha/version changes [lucene]
dweiss closed issue #14598: gradle-wrapper.jar will not be updated when its sha/version changes URL: https://github.com/apache/lucene/issues/14598 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [PR] Always check gradle wrapper sha checksum and download if necessary [lucene]
dweiss merged PR #14608: URL: https://github.com/apache/lucene/pull/14608 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [PR] Always check gradle wrapper sha checksum and download if necessary [lucene]
dweiss commented on PR #14608: URL: https://github.com/apache/lucene/pull/14608#issuecomment-2849376294 Interesting. I've got no idea, to be honest. Here is a list of shas for all releases - https://gradle.org/release-checksums/ v8.14 is there and consistent with what we have. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [I] gradle-wrapper.jar will not be updated when its sha/version changes [lucene]
dweiss commented on issue #14598: URL: https://github.com/apache/lucene/issues/14598#issuecomment-2849293690 This is actually a sort-of regression introduced in #13404 (90e07f666422). Previously, we'd always check the checksum of the wrapper file. After this change - to speed up startup time - we only do it if the file isn't there already. I'll file a PR to revert to the old behavior but use sha256sum to verify the checksum on Linux/ Mac. On Windows, I'll revert back to calling the wrapper. Yes, it is slower (adds about a second to execution time) but scripting something similar with cmd is tough. Maybe we can follow-up to cater to the needs of Windows users. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[PR] Overrides rewrite in PointRangeQuery to optimize AllDocs/NoDocs cases [lucene]
ebradshaw opened a new pull request, #14609: URL: https://github.com/apache/lucene/pull/14609 Overrides rewrite in PointRangeQuery range to handle cases where the query either fully contains or fully excludes all documents within the shard. Often, particularly when using time based partitioning, range queries may overlap several indexes. Many of these indexes have timestamp values that are fully contained by the query, in which case the query can be rewritten to a MatchAllDocsQuery or a FieldExistsQuery. On the other hand, many indexes can be fully excluded if they're outside the requested time range, in which case the query can be rewritten to a MatchNoDocsQuery. While a similar optimization exists at the leaf level in the createWeight function, rewriting at the shard level enables other optimizations downstream. Please let me know if this has been ruled out in the past for other reasons or if the implementation misses anything. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [PR] Always check gradle wrapper sha checksum and download if necessary [lucene]
vigyasharma commented on PR #14608: URL: https://github.com/apache/lucene/pull/14608#issuecomment-2849497626 Thanks for fixing this! I ran into the same problem. My old `gradle-wrapper.jar` was not getting updated and I kept hitting: ```bash % ./gradlew clean no main manifest attribute, in /Users/vigyas/repos/lucene/gradle/wrapper/gradle-wrapper.jar ``` My workaround till last night was to delete the `gradle-wrapper.jar`. Verified that this change fixes it by putting back the old jar and having gradle find checksum mismatch and redownload. ```bash % ./gradlew clean Checksum mismatch, will attempt to re-download gradle-wrapper.jar /Users/vigyas/repos/lucene/gradle/wrapper/gradle-wrapper.jar Downloading gradle-wrapper.jar from https://raw.githubusercontent.com/gradle/gradle/v8.14.0/gradle/wrapper/gradle-wrapper.jar Starting a Gradle Daemon (subsequent builds will be faster) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[PR] Update jdk requirements in README to OpenJDK 24 [lucene]
vigyasharma opened a new pull request, #14610: URL: https://github.com/apache/lucene/pull/14610 Lucene now depends on Java 24. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [PR] Support adaptive refresh in Searcher Managers. [lucene]
vigyasharma commented on code in PR #14443:
URL: https://github.com/apache/lucene/pull/14443#discussion_r2072749702
##
lucene/core/src/java/org/apache/lucene/search/SearcherManager.java:
##
@@ -131,17 +133,32 @@ public SearcherManager(DirectoryReader reader,
SearcherFactory searcherFactory)
this.current = getSearcher(searcherFactory, reader, null);
}
+ /** Set supplier for selecting commits to refresh on */
+ public void setRefreshCommitSupplier(RefreshCommitSupplier
refreshCommitSupplier) {
Review Comment:
Makes sense, I'll add it to the ctor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [PR] Update jdk requirements in README to OpenJDK 24 [lucene]
dweiss merged PR #14610: URL: https://github.com/apache/lucene/pull/14610 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [PR] Update jdk requirements in README to OpenJDK 24 [lucene]
dweiss commented on PR #14610: URL: https://github.com/apache/lucene/pull/14610#issuecomment-2849993132 Thank you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [PR] Update created version major [lucene]
rmuir commented on PR #14607: URL: https://github.com/apache/lucene/pull/14607#issuecomment-2849142575 As states on the dev list, I don't think we should do this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
