Nitin, I meant to reply, but....
I think the thing to watch out for are Lucene segment merges. I think this is another thing I saw in a client engagement where the client had a crazy number of fields. If I recall correctly, it was segment merges that were painfully slow. So try creating a non-trivial index where you have some big Lucene index segments and see how that goes. This was a while back before various Lucene improvements around indexing (different merge policies, non-blocking flushing to disk, etc.) were implemented, so things may be different now. Otis ---- Performance Monitoring for Solr / ElasticSearch / HBase - http://sematext.com/spm >________________________________ > From: "Keswani, Nitin - BLS CTR" <keswani.ni...@bls.gov> >To: "solr-user@lucene.apache.org" <solr-user@lucene.apache.org> >Sent: Monday, May 14, 2012 10:27 AM >Subject: RE: Documents With large number of fields > >Unfortunately I never got any response. However I did a POC with a Document >containing 400 fields and loaded around 1000 docs to my local machine. I >didn’t see any issue but then again the document set was very small. Hopefully >as mentioned below providing enough memory should help alleviate any >performance issues. > >Thanks. > >Regards, > >Nitin Keswani > > >-----Original Message----- >From: Jack Krupansky [mailto:j...@basetechnology.com] >Sent: Sunday, May 13, 2012 10:42 PM >To: solr-user@lucene.apache.org >Subject: Re: Documents With large number of fields > >I didn't see any response. There was a similar issue recently, where someone >had 400 faceted fields with 50-70 facets per query and they were running out >of memory due to accumulation of the FieldCache for these faceted fields, but >that was on a 3 GB system. > >It probably could be done, assuming a fair number of 64-bit sharded machines. > >-- Jack Krupansky > >-----Original Message----- >From: Darren Govoni >Sent: Sunday, May 13, 2012 7:56 PM >To: solr-user@lucene.apache.org >Subject: Re: Documents With large number of fields > >Was there a response to this? > >On Fri, 2012-05-04 at 10:27 -0400, Keswani, Nitin - BLS CTR wrote: >> Hi, >> >> My data model consist of different types of data. Each data type has >> its own characteristics >> >> If I include the unique characteristics of each type of data, my >> single Solr Document could end up containing 300-400 fields. >> >> In order to drill down to this data set I would have to provide >> faceting on most of these fields so that I can drilldown to very small >> set of Documents. >> >> Here are some of the questions : >> >> 1) What's the best approach when dealing with documents with large >> number of fields . >> Should I keep a single document with large number of fields or >> split my >> document into a number of smaller documents where each document >> would consist of some fields >> >> 2) From an operational point of view, what's the drawback of having a >> single document with a very large number of fields. >> Can Solr support documents with large number of fields (say 300 to >> 400). >> >> >> Thanks. >> >> Regards, >> >> Nitin Keswani >> > > > >
