dweiss opened a new issue, #14316:
URL: https://github.com/apache/lucene/issues/14316
### Description
Just bookmarking - these builds are currently failing because they're on
Java 21.
https://github.com/apache/lucene/actions/runs/13600135669
### Version and environment detai
dweiss commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2692034559
Thanks for sharing the kernel performance tracking magic! I'll certainly try
it. What is puzzling to me is why - if it's a problem on the java side
somewhere - there are such large v
dweiss commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2692036366
bq. That seems fine, I didnt realize only one test method was impacted: just
this slicing one? There are other test methods here doing thousands of seeks
and slices similar evil stuf
github-actions[bot] commented on PR #14178:
URL: https://github.com/apache/lucene/pull/14178#issuecomment-2691766624
This PR has not had activity in the past 2 weeks, labeling it as stale. If
the PR is waiting for review, notify the d...@lucene.apache.org list. Thank you
for your contributi
dweiss commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2691115857
I've no idea how to improve this - never had to wade into Linux kernel side
of things. Right now this particular test is unusable, at least on Ubuntu,
which is troubling - these few
rmuir commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2691679489
This is almost certainly a bug (possible in the openjdk). I reproduced the
seed (disabling tmpfs), then strace'd the pid while it was running. You can see
it repeatedly spamming exact
dweiss commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2691393106
Making it run on top of tmpfs is cheating though? :) What I'd do is override
this one problematic method in TestDirectIODirectory, check if it's Linux and
then either skip the test o
rmuir commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2691519568
> What I'd do is override this one problematic method in
TestDirectIODirectory, check if it's Linux and then either skip the test or
have a shorter timeout before it calls the
> te
rmuir commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2691501127
Also to answer your question, I have kernel 6.13.4 and use ext4 filesystem
but I dont expect any tests use that.
And my ext4 has some non-default features enabled such as
rmuir commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2691492019
> That's even more weird. What OS/filesystem are you using, if you don't
mind me asking? I wonder if it's ext4 to blame here. I'll try to narrow it down
during the weekend, I'm curiou
dweiss commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2691463108
I see similar results on Ubuntu 24.04, regardless of the filesystem used
(all the examples below, except for tmpfs, are on a 5gb loopback image).
TestDirectIODirectory.testRand
rmuir commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2691204279
it is no guarantee that /tmp is tmpfs either, it often isn't by default. I
configure that. Especially when running in containers it often isn't (GH
runners?), and when i need a real t
rmuir commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2691170470
i have the slowest computer on the planet (2-core i5 from 2018) and i
haven't hit this issue. Can we solve it in CI with gradle configuration?
I set this: `tests.workDir=/tmp/lu
dweiss commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2691147658
Do you think it's ok to somehow throttle this particular test for
TestDirectIODirectory? I'm sorry but fixing it in a better way - utilizing
those new kernel flags - is probably beyo
rmuir commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2690878935
Since there is now a non-deranged-monkey way to accomplish this same
use-case in the mainline kernel (RWF_DONTCACHE), maybe we should focus our
efforts on that instead?
--
This is
dweiss commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2690767252
"The thing that has always disturbed me about O_DIRECT is that the whole
interface is just stupid, and was probably designed by a deranged monkey on
some serious mind-controlling sub
epotyom commented on PR #14237:
URL: https://github.com/apache/lucene/pull/14237#issuecomment-2690720688
> I'm happy with it, I'm just wondering if you want to put it in 10.2.
Makes sense, I updated the branch.
Also rebased once again, and added a commit
https://github.com/apac
rmuir commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2690714355
the people who maintain the linux kernel tell you not to use O_DIRECT for a
reason
--
This is an automated message from the Apache Git Service.
To respond to the message, please log
dweiss commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2690539906
This must have something to do with the filesystem. I've re-ran the same
seed on a tmpfs-mounted folder:
```
:lucene:misc:test (SUCCESS): 63 test(s), 2 skipped
The slowest te
dweiss commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2690500887
I didn't think it was the Java part. gh runner results seem to confirm it's
something with Linux. Perhaps the java native binding is calling that pread
with too conservative argument
ChrisHegarty commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2690490408
My Linux is Ubuntu 24.04
I reverted the recent changes in Direct I/O from #14106, and the problem
still reproduces. So #14106 is not the cause.
--
This is an automated
dweiss commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2690460991
On ubuntu/arm it's slow as well -
https://github.com/dweiss/lucene/actions/runs/13587509034
--
This is an automated message from the Apache Git Service.
To respond to the messag
dweiss commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2690457786
Yep - mac and windows - no problem at all. Linux - slow like hell.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub an
ChrisHegarty commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2690452885
```diff
---
a/lucene/test-framework/src/java/org/apache/lucene/tests/store/BaseDirectoryTestCase.java
+++
b/lucene/test-framework/src/java/org/apache/lucene/tests/store/B
dweiss commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2690439018
I'll correct the workflow but on mac images this test finishes in under 10
seconds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please l
dweiss commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2690437406
I ran this, for curiosity. Very intriguing.
https://github.com/dweiss/lucene/actions/runs/13587410484
--
This is an automated message from the Apache Git Service.
To respond to t
dweiss commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2690411362
It seems like something is slow on Ubuntu and that particular kernel... I
don't think how the Java side of the code could be buggy here. The Ubuntu
system is on an AMD CPU - it'd be
dweiss commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2690413608
I'll play around with this later, unless you figure it out, Chris.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub an
ChrisHegarty commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2690407344
I can only think that this is a bug that I introduced with recent
refactoring. Looking ...
--
This is an automated message from the Apache Git Service.
To respond to the mess
dweiss commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2690403279
On a beefy mac, it takes 8 seconds...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
dweiss commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2690388381
I don't know if it's the JDK. Seems like it's direct i/o all the way to the
kernel -
dweiss commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2690389408
For the seed above, it takes 4m29s to execute this test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
ChrisHegarty commented on issue #14315:
URL: https://github.com/apache/lucene/issues/14315#issuecomment-2690372376
Any particular JDK, or observable in many ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL abo
uschindler commented on issue #14314:
URL: https://github.com/apache/lucene/issues/14314#issuecomment-2690348250
OK, the Policeman Jenkins was updated and 2 new config scripts for JVM
randomization were added. The default JDK for all jobs was set to 23, the JVM
settings are - as usual - rep
dweiss closed issue #14307: TestSysoutLimits still occasionally failing
URL: https://github.com/apache/lucene/issues/14307
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To un
dweiss opened a new issue, #14315:
URL: https://github.com/apache/lucene/issues/14315
### Description
Something is odd here. This, on main:
```
./gradlew :lucene:misc:test --tests
"org.apache.lucene.misc.store.TestDirectIODirectory" -Ptests.jvms=4
-Ptests.jvmargs= -Ptests.
dweiss commented on issue #14314:
URL: https://github.com/apache/lucene/issues/14314#issuecomment-2690321972
> You may have access to the machine if you legged in, but it could be that
you forgot your password.
Could be, could be. :older_man:
--
This is an automated message fr
uschindler commented on issue #14314:
URL: https://github.com/apache/lucene/issues/14314#issuecomment-2690301266
> Thanks, Uwe. I didn't want to bother you yesterday and none of us has
access to your precious, shiny, made-in-Germany test machine. 🏎️ 🚔 🚔 🚔 :)
You may have access to the
dweiss commented on issue #14314:
URL: https://github.com/apache/lucene/issues/14314#issuecomment-2690205290
Thanks, Uwe. I didn't want to bother you yesterday and none of us has access
to your precious, shiny, made-in-Germany test machine. :racing_car:
:oncoming_police_car: :oncoming_poli
uschindler commented on issue #14314:
URL: https://github.com/apache/lucene/issues/14314#issuecomment-2690175472
Policeman Jenkins also needs to update. I disabled the jobs there yesterday
evening.
--
This is an automated message from the Apache Git Service.
To respond to the message, ple
thecoop commented on PR #14304:
URL: https://github.com/apache/lucene/pull/14304#issuecomment-2690126444
Unfortunately not, I've been unable to get the quantized vector datasets
working on my machine
--
This is an automated message from the Apache Git Service.
To respond to the message, p
dweiss opened a new issue, #14314:
URL: https://github.com/apache/lucene/issues/14314
### Description
Just a record of jenkins configurations that need to be updated after a
min-jdk version bump on main (#14302):
* https://ci-builds.apache.org/job/Lucene/job/Lucene-Artifacts-ma
dweiss closed issue #14314: Update jenkins configurations after min JDK version
bump
URL: https://github.com/apache/lucene/issues/14314
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific c
stefanvodita merged PR #14279:
URL: https://github.com/apache/lucene/pull/14279
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucen
stefanvodita closed issue #13898: Create a bot to check if there is a CHANGES
entry for new PRs
URL: https://github.com/apache/lucene/issues/13898
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
jpountz commented on code in PR #14306:
URL: https://github.com/apache/lucene/pull/14306#discussion_r1974994754
##
lucene/core/src/java/org/apache/lucene/codecs/lucene101/Lucene101PostingsFormat.java:
##
@@ -351,6 +352,14 @@ public final class Lucene101PostingsFormat extends
Po
dweiss merged PR #14309:
URL: https://github.com/apache/lucene/pull/14309
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apac
47 matches
Mail list logo
