As I've mentioned before, I'm very new to Solr. I'm not a Java guy or an Apache guy. I'm a .Net guy.
We have a rather large schema - some 100 + fields plus a large number of dynamic fields. We've been trying to improve performance and finally got around to implementing fastvectorhighlighting which gave us an immediate improvement on the qtime (nearly 70%) which also improved the overall response time by over 20%. With that, we also bring back an extraordinarly large amount of data in the XML. Some results (20 records) come back with a payload between 3MB and even 17MB. We have a lot of report text that is used for searching and highlighting. We recently implemented field list wildcards on two versions of Solr to test it out. This allowed us to leave the report text off the return and decreased the payload significantly - by nearly 85% in the large cases... SO, we'd expect a performance boost there, however we are seeing greatly increased response times on these builds of Solr even though the qtime is incredibly fast. To put it in perspective - our original Solr core is 4.0, I believe the 4.0.0.2010.12.10.08.54.56 version. On our test boxes, we have one running 4.0.0.2011.11.17 and one running 4.0.0.2012.02.16 version. with the older version (not having the wildcard field list), it returns a payload of approximately 13MB in an average of 1.5 seconds. with the new version (2012.02.16) which is on the same machines as the older version (so network traffic/latency/hardware/etc are all the same), it's returning the reduced payload (approximately 1.5MB in an average of 3.5-4 seconds). I will say that we reloaded the core once and briefly saw the 1.5MB payload come back in 150-200 milliseconds, but within minutes we were back to the 3.5-4 seconds. We also noticed the CPU was being pegged for seconds when running the queries on the new build with the wildcard field list. We have a lower scale box running the 2011.11.17 version and had more success for a while. We were getting the 150-200 ms response time on the reduced payload for probably 30 minutes or so, and then it did the same thing - bumped up to 3-4 seconds in response time. Anyone have any experience with this type of random yet consistent performance degradation or have insight as to what might be causing the issues and how to fix them? We'd love to not only have the performance boost from fast vector highlighting, but also the decreased payload size. Thanks in advance! -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Performance-Improvement-and-degradation-Help-tp3767015p3767015.html Sent from the Solr - User mailing list archive at Nabble.com.
