ChrisHegarty merged PR #13998:
URL: https://github.com/apache/lucene/pull/13998
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucen
jpountz commented on PR #13998:
URL: https://github.com/apache/lucene/pull/13998#issuecomment-2494266353
Sorry for derailing the PR, let's not implement it on ByteBuffersIndexInput
then. We can look into it in a separate PR if we want.
--
This is an automated message from the Apache Git S
ChrisHegarty commented on PR #13998:
URL: https://github.com/apache/lucene/pull/13998#issuecomment-2494090457
> > yeah, I think that this prob makes sense. Lemme satisfy myself that it
will always be true.
>
> it won't be in core if currently swapped out, no? I don't think a
hardcode
rmuir commented on PR #13998:
URL: https://github.com/apache/lucene/pull/13998#issuecomment-2493975727
> yeah, I think that this prob makes sense. Lemme satisfy myself that it
will always be true.
it won't be in core if currently swapped out, no? I don't think a hardcoded
`true` work
jpountz commented on code in PR #13998:
URL: https://github.com/apache/lucene/pull/13998#discussion_r1850224848
##
lucene/core/src/java21/org/apache/lucene/store/MemorySegmentIndexInput.java:
##
@@ -406,6 +406,15 @@ void advise(long offset, long length,
IOConsumer advice) throw
ChrisHegarty commented on PR #13998:
URL: https://github.com/apache/lucene/pull/13998#issuecomment-2489087426
> This works for me. Maybe implement this API on our in-memory index inputs
to return true, e.g. `ByteBuffersIndexInput`?
yeah, I think that this prob makes sense. Lemme satis
ChrisHegarty commented on code in PR #13998:
URL: https://github.com/apache/lucene/pull/13998#discussion_r1850657822
##
lucene/core/src/java/org/apache/lucene/store/IndexInput.java:
##
@@ -226,4 +227,17 @@ public String toString() {
* @param length the number of bytes to pre
ChrisHegarty commented on code in PR #13998:
URL: https://github.com/apache/lucene/pull/13998#discussion_r1850653616
##
lucene/core/src/java21/org/apache/lucene/store/MemorySegmentIndexInput.java:
##
@@ -406,6 +406,15 @@ void advise(long offset, long length,
IOConsumer advice)
ChrisHegarty commented on PR #13998:
URL: https://github.com/apache/lucene/pull/13998#issuecomment-2485150279
> yes, agreed about `mincore` as a followup. Let's use existing JDK plumbing
as a start as done here.
++
> i'm very much against using `mlock`, there are so many prob
ChrisHegarty commented on PR #13998:
URL: https://github.com/apache/lucene/pull/13998#issuecomment-2482415401
> @ChrisHegarty this will be a very useful thing.
Indeed.
> Can we also figure out how much data is loaded with this API? So lets say
an IndexInput is 30GB and only 10G
rmuir commented on PR #13998:
URL: https://github.com/apache/lucene/pull/13998#issuecomment-2483336503
Also for debugging these issues, you can get this information at non-java
level using `fincore` from util-linux, which is probably on any machine:
```
myindexdir$ fincore --output
rmuir commented on PR #13998:
URL: https://github.com/apache/lucene/pull/13998#issuecomment-2483230262
You would need to call `mincore` or something yourself. I can't remember,
but the native access may already be plumbed.
for non-mmapped i/o you can do similar with syscalls such as `
rmuir commented on PR #13998:
URL: https://github.com/apache/lucene/pull/13998#issuecomment-2483324513
> Yeah, we can look at how to call `mincore`, and it might not be that much
of a lift with the existing plumbing. Maybe something can look at as a follow
up? I'm really trying to get to a
ChrisHegarty commented on code in PR #13998:
URL: https://github.com/apache/lucene/pull/13998#discussion_r1846722542
##
lucene/core/src/java21/org/apache/lucene/store/MemorySegmentIndexInput.java:
##
@@ -406,6 +406,14 @@ void advise(long offset, long length,
IOConsumer advice)
ChrisHegarty commented on PR #13998:
URL: https://github.com/apache/lucene/pull/13998#issuecomment-2483288393
Yeah, we can look at how to call `mincore`, and it might not be that much of
a lift with the existing plumbing. Maybe something can look at as a follow up?
I'm really trying to ge
rmuir commented on code in PR #13998:
URL: https://github.com/apache/lucene/pull/13998#discussion_r1846706623
##
lucene/core/src/java21/org/apache/lucene/store/MemorySegmentIndexInput.java:
##
@@ -406,6 +406,14 @@ void advise(long offset, long length,
IOConsumer advice) throws
navneet1v commented on PR #13998:
URL: https://github.com/apache/lucene/pull/13998#issuecomment-2479683979
@ChrisHegarty this will be a very useful thing. Can we also figure out how
much data is loaded with this API? So lets say an IndexInput is 30GB and only
10GB is loaded/mapped in memory
17 matches
Mail list logo
