Hi,
We've investigated a memory dump, which was taken after some frequent OOM
incidents.
The main issue we found was a lot of millions of LazyField instances,
taking ~2GB of memory, even though queries request about 10 small fields
only.
We've found that LazyDocument creates a LazyField object for every item in
a multivalued field, even if do not want this field.
For example, documents contain a multivalued field, named "f", with a lot
of values (let's say 100 values per document). Queries set fl=id (request
only document id). The documentCache will grow up in memory :(
In our case, documentCache was configured to 32000. There are 2 cores per
node, so 64000 LazyDocument instances are in memory. This is pretty big
number, and we'll reduce it.
I'm curious whether it's a known issue or not? and why should the
LazyDocument know the amount of values in a multivalued field which is not
requested?
Another thought which I had: Is it reasonable to add something like
"{!cache=false}" which will affect documentCache. For example. If my query
request "id" only, with a big rows parameter, I don't want documentCache to
hold these big LazyDocument objects.
Did anyone else encounter this?
