You cannot guarantee this when you're running out of RAM. You'd have a
problem then anyway.
Why are you caring that much? Did you yet have performance issues? 1GB
should load really fast, and both auto warming and OS cache should help
a lot as well. With such an index, you usually don't need to fine tune
performance that much.
Did you think about using a SSD? Since you want to persist your index,
you'll need to live with disk IO anyway.
Greetings,
Kuli
Am 11.06.2012 11:20, schrieb Li Li:
I am sorry. I make a mistake. even use RAMDirectory, I can not
guarantee they are not swapped out.
On Mon, Jun 11, 2012 at 4:45 PM, Michael Kuhlmann<k...@solarier.de> wrote:
Set the swapiness to 0 to avoid memory pages being swapped to disk too
early.
http://en.wikipedia.org/wiki/Swappiness
-Kuli
Am 11.06.2012 10:38, schrieb Li Li:
I have roughly read the codes of RAMDirectory. it use a list of 1024
byte arrays and many overheads.
But as far as I know, using MMapDirectory, I can't prevent the page
faults. OS will swap less frequent pages out. Even if I allocate
enough memory for JVM, I can guarantee all the files in the directory
are in memory. am I understanding right? if it is, then some less
frequent queries will be slow. How can I let them always in memory?
On Fri, Jun 8, 2012 at 5:53 PM, Lance Norskog<goks...@gmail.com> wrote:
Yes, use MMapDirectory. It is faster and uses memory more efficiently
than RAMDirectory. This sounds wrong, but it is true. With
RAMDirectory, Java has to work harder doing garbage collection.
On Fri, Jun 8, 2012 at 1:30 AM, Li Li<fancye...@gmail.com> wrote:
hi all
I want to use lucene 3.6 providing searching service. my data is
not very large, raw data is less that 1GB and I want to use load all
indexes into memory. also I need save all indexes into disk
persistently.
I originally want to use RAMDirectory. But when I read its javadoc.
Warning: This class is not intended to work with huge indexes.
Everything beyond several hundred megabytes
will waste resources (GC cycles), because it uses an internal buffer
size of 1024 bytes, producing millions of byte
[1024] arrays. This class is optimized for small memory-resident
indexes. It also has bad concurrency on
multithreaded environments.
It is recommended to materialize large indexes on disk and use
MMapDirectory, which is a high-performance
directory implementation working directly on the file system cache of
the operating system, so copying data to
Java heap space is not useful.
should I use MMapDirectory? it seems another contrib instantiated.
anyone test it with RAMDirectory?
--
Lance Norskog
goks...@gmail.com
