Having a large number of fields is not the same as having a large number of facets. To facets are something you would display to users as aid for query refinement or navigation. There is no way for a user to use 3700 facets at the same time. So it more of question on how to determine what facets to fetch on search time based on the user's actions or based on certain predefined configurations. I have written an application with 30 some facetable fields on millions of records, I also ran into the issue of calculate all facets as the server resources as limited to number of caches available and CPU cycles available for facet calculations. I then realize why display all these facet regardless user want to see them or not? I have then change to approach to only fetch minimum set of facets by default and make the rest of facets fields open on-demand (using AJAX). I was able to dramatically increase the response time by spreading the facet loading overtime. There are still issues of total facet caches when you have a large number available facets, but you need realistically evaluate what does it means to a user to have large number of facet. I don't think on typical user interface having more than 10 filters showing at the same time will be any more effective than having a small number of filters to begin with and progressive showing more on-demand (hierarchical facets?)
Rahul R wrote: > > Hello, > We are trying to get Solr to work for a really huge parts database. > Details > of the database > - 55 million parts > - Totally 3700 properties (facets). But each record will not have value > for > all properties. > - Most of these facets are defined as dynamic fields within the Solr Index > > We were getting really unacceptable timing while doing faceting/searches > on > an index created with this database. With only one user using the system, > query times are in excess of 1 minute. With more users concurrently using > the system, the response times are further high. > > We thought that by limiting the number of properties that are available > for > faceting, the performance can be improved. To test this, we enabled only 6 > properties for faceting by setting indexed=true (in schema.xml) for only > these properties. All other properties which are defined as dynamic > properties had indexed=false. The observations after this change : > > - Index size reduced by a meagre 5 % only > - Performance did not improve. Infact during PSR run we observed that it > degraded. > > My questions: > - Will reducing the number of facets improve faceting and search > performance ? > - Is there a better way to reduce the number of facets ? > - Will having a large number of properties defined as dynamic fields, > reduce > performance ? > > Thank you. > > Regards > Rahul > > -- View this message in context: http://www.nabble.com/Limiting-facets-for-huge-data---setting-indexed%3Dfalse-in-schema.xml-tp24751763p24761778.html Sent from the Solr - User mailing list archive at Nabble.com.
