Hello, in our project we need to execute some big queries against Solr once a day, with maybe more than 1000 results, in order to trigger a batch proccess with the results. In the fl parameter we only are putting the ID field, because we don't need large text fields.
This is our scenary: - Our documents are generally very big, but as I have said we only request for the ID field. - We have the enableLazyFieldLoading parameter set to true in solrconfig.xml, so the DocumentCache should load only the ID field that we are requesting. - Our DocumetCache is set to 8192 objects. - This test have been executed in Solr 4.2.1, 4.6.1 and 4.7, in no Solr Cloud as well as in Solr Cloud mode. The issue we have got is this: - When we request for more than 1000 docs. more or less JVM takes a lot of memory and ends with an OOM. - Seeing in "real time" as the DocumentCache inserts documents, we have seen that memory grows when the documents are bigger (and time needed to load that documents in Cache is larger also), but we don't understand why, because with enableLazyFieldLoading only the ID should be loaded, so, ¿why does memory grow in that way? We know that one solution is increase RAM memory and other is decrease the size of DocumentCache (we have already done this), but we'd like to know why this issue with memory. On the other hand, one good solution for us would be make the queries without cache. ¿Is there any way to say Solr not to cache some specific queries? I don't think so, but maybe I am wrong. Thank very much, David Dávila Atienza AEAT - Departamento de Informática Tributaria Subdirección de Tecnologías de Análisis de la Información e Investigación del Fraude Teléfono: 917681160 Extensión: 30160
