HUSTERGS opened a new pull request, #14806:
URL: https://github.com/apache/lucene/pull/14806
### Description
This PR propose to utilize `docIdRunEnd` on `ReqExclBulkScorer`, so we can
jump faster on `MUST_NOT` clause
--
This is an automated message from the Apache Git Service.
github-actions[bot] commented on PR #14806:
URL: https://github.com/apache/lucene/pull/14806#issuecomment-2982992815
This PR does not have an entry in lucene/CHANGES.txt. Consider adding one.
If the PR doesn't need a changelog entry, then add the skip-changelog label to
it and you will stop
dweiss commented on PR #14804:
URL: https://github.com/apache/lucene/pull/14804#issuecomment-2983643568
The reason it's never up to date with a custom formatting step:
https://github.com/diffplug/spotless/issues/2516
--
This is an automated message from the Apache Git Service.
To respon
dweiss commented on PR #14804:
URL: https://github.com/apache/lucene/pull/14804#issuecomment-2983964771
It is wicked fast indeed (uses all cores).
```
dweiss@dweiss-beast:~/work/apache/lucene$ time ast-grep scan . -r
/home/dweiss/tmp/no-wildcard-imports.sg.yaml
...
real 0m0.738s
gf2121 commented on code in PR #14806:
URL: https://github.com/apache/lucene/pull/14806#discussion_r2154439718
##
lucene/core/src/java/org/apache/lucene/search/ReqExclBulkScorer.java:
##
@@ -57,7 +57,10 @@ public int score(LeafCollector collector, Bits acceptDocs,
int min, int
gf2121 commented on code in PR #14806:
URL: https://github.com/apache/lucene/pull/14806#discussion_r2154454156
##
lucene/core/src/java/org/apache/lucene/search/ReqExclBulkScorer.java:
##
@@ -57,7 +57,10 @@ public int score(LeafCollector collector, Bits acceptDocs,
int min, int
jpountz merged PR #14784:
URL: https://github.com/apache/lucene/pull/14784
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apa
jpountz commented on PR #14773:
URL: https://github.com/apache/lucene/pull/14773#issuecomment-2983993101
Doing this is a bit less obvious in my mind since terms dictionaris are
allowed to have a random access pattern, when doc-value iterators are required
to be consumed in order?
Som
easyice closed pull request #14773: Add prefetching for terms dict in doc values
URL: https://github.com/apache/lucene/pull/14773
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
easyice commented on PR #14773:
URL: https://github.com/apache/lucene/pull/14773#issuecomment-2984815277
Thanks for the explanation, You're right, I'll close this PR. I feel an API
to prefetch full range of data could be useful, e.g. for files like `.tip`,
which are relatively small but hav
github-actions[bot] commented on PR #14806:
URL: https://github.com/apache/lucene/pull/14806#issuecomment-2983425250
This PR does not have an entry in lucene/CHANGES.txt. Consider adding one.
If the PR doesn't need a changelog entry, then add the skip-changelog label to
it and you will stop
github-actions[bot] commented on PR #14806:
URL: https://github.com/apache/lucene/pull/14806#issuecomment-2983430347
This PR does not have an entry in lucene/CHANGES.txt. Consider adding one.
If the PR doesn't need a changelog entry, then add the skip-changelog label to
it and you will stop
HUSTERGS commented on code in PR #14806:
URL: https://github.com/apache/lucene/pull/14806#discussion_r2154727636
##
lucene/core/src/java/org/apache/lucene/search/ReqExclBulkScorer.java:
##
@@ -57,7 +57,10 @@ public int score(LeafCollector collector, Bits acceptDocs,
int min, in
dweiss commented on PR #14804:
URL: https://github.com/apache/lucene/pull/14804#issuecomment-2983757840
Error prone has a rule for this, actually -
```
diff --git
a/build-tools/build-infra/src/main/groovy/lucene.validation.error-prone.gradle
b/build-tools/build-infra/src/main/groovy/l
benwtrent commented on PR #14792:
URL: https://github.com/apache/lucene/pull/14792#issuecomment-2983852051
@Pulkitg64 I think the format has access to the quantized vectors. I am
saying we shouldn't add a new `getQuantizedVEctors` to the leaf or kNN APIs.
When the `vec` file isn't pr
rmuir commented on PR #14804:
URL: https://github.com/apache/lucene/pull/14804#issuecomment-2983891154
yeah, that's fine.
we can discuss it separately maybe. Elsewhere I use the tool a lot, replace
entire slow linters with it! It is wicked fast...
there's a strange cultural thi
dweiss commented on PR #14804:
URL: https://github.com/apache/lucene/pull/14804#issuecomment-2983916752
I have noticed this as well - in the js world most (fast) things use a rust
backend.
Anyway, I'll fiddle with this a bit, it's not a high priority thing. Maybe
make it optionally a
dweiss commented on PR #14804:
URL: https://github.com/apache/lucene/pull/14804#issuecomment-2982832608
I'll take a look, sure. I'll also try to figure out why the heck spotless
isn't behaving the way it should.
--
This is an automated message from the Apache Git Service.
To respond to th
Pulkitg64 commented on PR #14792:
URL: https://github.com/apache/lucene/pull/14792#issuecomment-2984235394
Oh I understand now what you meant in your comments. This approach is much
more cleaner and doesn't require any new API addition. Will raise a new
revision in some time.
--
This is
dsmiley commented on PR #14740:
URL: https://github.com/apache/lucene/pull/14740#issuecomment-2984277753
The existing editorconfig was redundant with information in this
editorconfig, so I simply removed it. I'll merge this tonight.
--
This is an automated message from the Apache Git Ser
romseygeek opened a new pull request, #14807:
URL: https://github.com/apache/lucene/pull/14807
This was used in the past to correctly order terms in
MergedPrefixCodedTermsIterator,
but that class was removed long ago so the method is no longer needed.
--
This is an automat
github-actions[bot] commented on PR #14807:
URL: https://github.com/apache/lucene/pull/14807#issuecomment-2984306433
This PR does not have an entry in lucene/CHANGES.txt. Consider adding one.
If the PR doesn't need a changelog entry, then add the skip-changelog label to
it and you will stop
dweiss commented on PR #14804:
URL: https://github.com/apache/lucene/pull/14804#issuecomment-2984368443
I've added a task that runs ast-grep on the CI and locally, if you define
```lucene.tool.ast-grep``` property (for example,
```lucene.tool.ast-grep=ast-grep``` or permanently, in local bu
dweiss commented on PR #14804:
URL: https://github.com/apache/lucene/pull/14804#issuecomment-2984370238
I'll try to finalize this patch later today, have to leave now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use th
HUSTERGS commented on code in PR #14806:
URL: https://github.com/apache/lucene/pull/14806#discussion_r2154711602
##
lucene/core/src/java/org/apache/lucene/search/ReqExclBulkScorer.java:
##
@@ -57,7 +57,10 @@ public int score(LeafCollector collector, Bits acceptDocs,
int min, in
Pulkitg64 commented on PR #14792:
URL: https://github.com/apache/lucene/pull/14792#issuecomment-2983706927
Thanks @benwtrent @msokolov
> Again, I think we should do the nice thing, de-quantize the vectors as the
user asks for them.
Sorry, I am surely missing something. But if
rmuir commented on PR #14804:
URL: https://github.com/apache/lucene/pull/14804#issuecomment-2984212372
In my builds, I install such tools via `pip` or `npm` package managers. So
it's no different than downloading jar and using that, just faster. Just food
for thought.
--
This is an autom
rmuir commented on PR #14804:
URL: https://github.com/apache/lucene/pull/14804#issuecomment-2984219215
The python linting, formatting, type checking in dev tools is an example.
None of those are written in the python programming language.
--
This is an automated message from the Apache Gi
rmuir commented on issue #14803:
URL: https://github.com/apache/lucene/issues/14803#issuecomment-2985347971
The advantage of letting a filesystem such as zfs (which was designed to do
exactly this), is that it is integrated in the correct place and operating
system caches work as expected.
nipunbatra8 commented on issue #14148:
URL: https://github.com/apache/lucene/issues/14148#issuecomment-2985361444
Hello, I will be diving into this issue for my summer internship. Looking
forward to discussing more in the near future and contributing to this project!
--
This is an automat
rmuir commented on PR #14804:
URL: https://github.com/apache/lucene/pull/14804#issuecomment-2985443802
Another related idea here is to prevent any IDE from automatically adding
the *'s we don't want.
Eclipse's default limit is 99 imports before it starts doing this... of
course user
rmuir commented on code in PR #14804:
URL: https://github.com/apache/lucene/pull/14804#discussion_r2155360626
##
gradle/validation/ast-grep/sgconfig.yml:
##
@@ -0,0 +1,2 @@
+ruleDirs:
+ - ./rules
Review Comment:
I've found the built-in self-test to be essential for having a
dweiss opened a new pull request, #14808:
URL: https://github.com/apache/lucene/pull/14808
This is a spinoff from #14804. We should allow running ```ast-grep```
(https://ast-grep.github.io/) rules as validation checks. This is a powerful
tool (and fast).
For now, running ast-grep is
github-actions[bot] commented on PR #14808:
URL: https://github.com/apache/lucene/pull/14808#issuecomment-2985485459
This PR does not have an entry in lucene/CHANGES.txt. Consider adding one.
If the PR doesn't need a changelog entry, then add the skip-changelog label to
it and you will stop
dweiss commented on PR #14804:
URL: https://github.com/apache/lucene/pull/14804#issuecomment-2985485780
I've forked the ast-grep thing into a separate PR -
https://github.com/apache/lucene/pull/14808. I think we can merge that one in,
then I can apply a smaller patch for wildcards, using yo
dweiss commented on code in PR #14804:
URL: https://github.com/apache/lucene/pull/14804#discussion_r2155370086
##
gradle/validation/ast-grep/sgconfig.yml:
##
@@ -0,0 +1,2 @@
+ruleDirs:
+ - ./rules
Review Comment:
> I run these with ast-grep test --skip-snapshot-tests before
rmuir commented on PR #14812:
URL: https://github.com/apache/lucene/pull/14812#issuecomment-2986074641
This is just starting small. I really want to next clean up some of the rat
interaction in this file, especially given the changing APIs and
thread-unsafeness that makes the gradle code co
dweiss commented on code in PR #14804:
URL: https://github.com/apache/lucene/pull/14804#discussion_r2155429690
##
gradle/validation/ast-grep/sgconfig.yml:
##
@@ -0,0 +1,2 @@
+ruleDirs:
+ - ./rules
Review Comment:
I've added it, nice.
--
This is an automated message from
rmuir commented on PR #14808:
URL: https://github.com/apache/lucene/pull/14808#issuecomment-2985574748
Thank you @dweiss, I was just slow, was testing it out. I set the property
locally, and added rule and test:
```yaml
---
# yaml-language-server:
$schema=https://raw.githubusercont
rmuir commented on code in PR #14804:
URL: https://github.com/apache/lucene/pull/14804#discussion_r2155451618
##
gradle/validation/ast-grep/rules/no-wildcard-imports.sg.yaml:
##
@@ -0,0 +1,16 @@
+# yaml-language-server:
$schema=https://raw.githubusercontent.com/ast-grep/ast-gre
dweiss commented on PR #14804:
URL: https://github.com/apache/lucene/pull/14804#issuecomment-2985617584
Ok. I think this one is ready.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
github-actions[bot] commented on PR #14810:
URL: https://github.com/apache/lucene/pull/14810#issuecomment-2985745189
This PR does not have an entry in lucene/CHANGES.txt. Consider adding one.
If the PR doesn't need a changelog entry, then add the skip-changelog label to
it and you will stop
rmuir opened a new pull request, #14810:
URL: https://github.com/apache/lucene/pull/14810
This is just an example of a dead-simple check to cross off the list.
organize the error-prone rules into an errorprone.yml file with matching
linter IDs.
--
This is an automated messa
dweiss merged PR #14809:
URL: https://github.com/apache/lucene/pull/14809
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apac
rmuir commented on PR #14804:
URL: https://github.com/apache/lucene/pull/14804#issuecomment-2985386981
Thanks for playing with it. I would be happy to try to write ast-grep rules
to try to replace some of the slower error-prone checks, or whatever else
(nocommit checks etc).
We could
rmuir opened a new pull request, #14811:
URL: https://github.com/apache/lucene/pull/14811
This rule is attempting to ban dangerous usages of ClassLoader which could
be security hazard (IMO a good idea). forbidden APIs is a good tool for
banning, just like we use forbidden-apis to ban JNDI a
github-actions[bot] commented on PR #14811:
URL: https://github.com/apache/lucene/pull/14811#issuecomment-2985810339
This PR does not have an entry in lucene/CHANGES.txt. Consider adding one.
If the PR doesn't need a changelog entry, then add the skip-changelog label to
it and you will stop
rmuir commented on code in PR #14811:
URL: https://github.com/apache/lucene/pull/14811#discussion_r2155576445
##
gradle/validation/forbidden-apis/defaults.all.txt:
##
@@ -76,3 +76,15 @@ java.lang.Math#fma(float,float,float)
java.lang.Math#fma(double,double,double)
java.lang.
github-actions[bot] commented on PR #14811:
URL: https://github.com/apache/lucene/pull/14811#issuecomment-2985834553
This PR does not have an entry in lucene/CHANGES.txt. Consider adding one.
If the PR doesn't need a changelog entry, then add the skip-changelog label to
it and you will stop
rmuir commented on PR #14811:
URL: https://github.com/apache/lucene/pull/14811#issuecomment-2985845882
Source of the check:
https://github.com/google/error-prone/blob/master/core/src/main/java/com/google/errorprone/bugpatterns/BanClassLoader.java
It also checks for explicit `extends U
dweiss merged PR #14808:
URL: https://github.com/apache/lucene/pull/14808
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apac
dweiss commented on PR #14808:
URL: https://github.com/apache/lucene/pull/14808#issuecomment-2985638437
Sorry for not waiting, I wanted to proceed. Please open a new issue with
those rules. This one looks very neat indeed - we can even force it to run
prior to javac. It's a C based parser f
dweiss closed issue #14553: Fail spotless check for wildcard imports
URL: https://github.com/apache/lucene/issues/14553
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsub
dweiss merged PR #14804:
URL: https://github.com/apache/lucene/pull/14804
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apac
dweiss commented on code in PR #14804:
URL: https://github.com/apache/lucene/pull/14804#discussion_r2155467101
##
gradle/validation/ast-grep/rules/no-wildcard-imports.sg.yaml:
##
@@ -0,0 +1,16 @@
+# yaml-language-server:
$schema=https://raw.githubusercontent.com/ast-grep/ast-gr
rmuir commented on code in PR #14804:
URL: https://github.com/apache/lucene/pull/14804#discussion_r2155464257
##
gradle/validation/ast-grep/rules/no-wildcard-imports.sg.yaml:
##
@@ -0,0 +1,16 @@
+# yaml-language-server:
$schema=https://raw.githubusercontent.com/ast-grep/ast-gre
dweiss opened a new pull request, #14809:
URL: https://github.com/apache/lucene/pull/14809
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-ma
dweiss commented on PR #14804:
URL: https://github.com/apache/lucene/pull/14804#issuecomment-2985545202
> We should be able to make it easier for IntelliJ to do the right thing
without too much trouble. One option is to use `gradle-idea-ext`,
I've never used this plugin - have it on
dweiss commented on PR #14808:
URL: https://github.com/apache/lucene/pull/14808#issuecomment-2985547456
I'll allow myself to merge this in without a review. I don't think anybody
will object (but if anybody does, shout out).
--
This is an automated message from the Apache Git Service.
To
rmuir commented on code in PR #14804:
URL: https://github.com/apache/lucene/pull/14804#discussion_r2155375027
##
gradle/validation/ast-grep/rules/no-wildcard-imports.sg.yaml:
##
@@ -0,0 +1,15 @@
+id: wildcard-import-not-allowed
Review Comment:
```suggestion
# yaml-languag
msfroh commented on PR #14804:
URL: https://github.com/apache/lucene/pull/14804#issuecomment-2985537756
> I don't know how intellij behaves here or if we can prevent similar
behavior. All this stuff can be followups, really, I'm just brainstorming.
We should be able to make it easier
dweiss commented on PR #14804:
URL: https://github.com/apache/lucene/pull/14804#issuecomment-2985418128
Well, if this patch goes in, all it takes to add a rule is to create a new
file in gradle/validation/ast-grep - it should be picked up automatically. We
can also make ast-grep a requireme
rmuir opened a new pull request, #14812:
URL: https://github.com/apache/lucene/pull/14812
The validation.source-patterns.gradle is intimidating, there's a lot going
on here. Many of the rules are doing regex matches, which is tricky to maintain
and can't give good error messages, autofixes
github-actions[bot] commented on PR #14812:
URL: https://github.com/apache/lucene/pull/14812#issuecomment-2986028296
This PR does not have an entry in lucene/CHANGES.txt. Consider adding one.
If the PR doesn't need a changelog entry, then add the skip-changelog label to
it and you will stop
rmuir commented on code in PR #14812:
URL: https://github.com/apache/lucene/pull/14812#discussion_r2155783331
##
gradle/validation/ast-grep/rules/java-patterns.yml:
##
@@ -0,0 +1,24 @@
+# Banned Lucene source patterns
+# Historically implemented as regexes which are more difficu
dweiss commented on code in PR #14804:
URL: https://github.com/apache/lucene/pull/14804#discussion_r2155455149
##
gradle/validation/ast-grep/rules/no-wildcard-imports.sg.yaml:
##
@@ -0,0 +1,16 @@
+# yaml-language-server:
$schema=https://raw.githubusercontent.com/ast-grep/ast-gr
dweiss commented on code in PR #14804:
URL: https://github.com/apache/lucene/pull/14804#discussion_r2155456818
##
gradle/validation/ast-grep/rules/no-wildcard-imports.sg.yaml:
##
@@ -0,0 +1,16 @@
+# yaml-language-server:
$schema=https://raw.githubusercontent.com/ast-grep/ast-gr
dweiss commented on code in PR #14804:
URL: https://github.com/apache/lucene/pull/14804#discussion_r2155458113
##
gradle/validation/ast-grep/rules/no-wildcard-imports.sg.yaml:
##
@@ -0,0 +1,16 @@
+# yaml-language-server:
$schema=https://raw.githubusercontent.com/ast-grep/ast-gr
rmuir commented on code in PR #14804:
URL: https://github.com/apache/lucene/pull/14804#discussion_r2155458055
##
gradle/validation/ast-grep/rules/no-wildcard-imports.sg.yaml:
##
@@ -0,0 +1,16 @@
+# yaml-language-server:
$schema=https://raw.githubusercontent.com/ast-grep/ast-gre
gf2121 commented on code in PR #14806:
URL: https://github.com/apache/lucene/pull/14806#discussion_r215686
##
lucene/core/src/java/org/apache/lucene/search/ReqExclBulkScorer.java:
##
@@ -57,7 +57,10 @@ public int score(LeafCollector collector, Bits acceptDocs,
int min, int
70 matches
Mail list logo
