Thanks everyone for the responses. I did some more queries and watched disk activity with iostat. Sure enough, during some of the slow queries the disk was pegged at 100% (or more.)
The requirement for the app I'm building is to be able to retrieve 500 results in ideally one second. The index has 3 million records. Most searches do bring back 500 results, or close to that many. This is my schema: <field name="metaDataUrl" type="string" indexed="true" stored="true" required="true"/> <field name="title" type="text" stored="true" indexed="true"/> <field name="snippet" type="text" indexed="true" stored="true"/> <field name="rest" type="string" stored="true" indexed="false" multiValued="true"/> <field name="date_indexed" type="date" indexed="true" stored="true" default="NOW" multiValued="false"/> <field name="all" type="text" stored="false" indexed="true" multiValued="true"/> These are my copyField values: <copyField source="title" dest="all"/> <copyField source="snippet" dest="all"/> The "rest" field is an array of several fairly small strings. I've set size and initialSize to 100000 for queryResultCache and documentCache. I've got the tomcat configured to use 4 GB of memory. Yonik, can you please explain what you mean by this? > For scalability, we don't read all of the stored fields for the whole > list into memory before starting to send the results, but stream them > back instead. How do you implement this streaming? I suppose I could experiment with returning just the metaDataUrl field. If that's consistently fast then I can look at ways to retrieve title, snippet, and rest more quickly. I'd appreciate any input you, or anyone, can provide on how to optimize this. -- View this message in context: http://lucene.472066.n3.nabble.com/small-QTime-but-slow-results-to-user-tp4027100p4027224.html Sent from the Solr - User mailing list archive at Nabble.com.
