Re: Auto commit and optimize settings

2008-05-18 Thread Vaijanath N. Rao
Hi Otis and Solr-users, I was under the impression that when one call optimize all the indexes created so far get's merged. Hence I went about the question on optimize. The reason I want optimize is that I have autoCommit feature in the solrConfig.xml to commit after every 1000 documents. Onc

Re: Some advice on scalability

2008-05-18 Thread Otis Gospodnetic
Hi, Not that I can think of at the moment. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: Norberto Meijome <[EMAIL PROTECTED]> > To: solr-user@lucene.apache.org > Sent: Sunday, May 18, 2008 10:31:19 AM > Subject: Re: Some advice on scalabi

Re: Auto commit and optimize settings

2008-05-18 Thread Otis Gospodnetic
Hi, There is no such option currently and it is not likely that such feature will be added because index optimization is not really a quick and lightweight operation, so one typically optimized only after the index is fully built and one knows the index will remain unchanged for a while. If yo

Auto commit and optimize settings

2008-05-18 Thread Vaijanath N. Rao
Hi Solr-Users, I have gone through the solrConfig.xml file in the example directory of the solr build (nightly build). I wanted to know is there a way to tell solr to optimize the index after certain number of seconds elapsed or number of records indexed as we do in case of auto-commit. --Th

RE: solr feed problem

2008-05-18 Thread Steven A Rowe
Hi Cam, On 05/18/2008 at 7:59 PM, Cam Bazz wrote: > SEVERE: org.xmlpull.v1.XmlPullParserException: entity > reference names can not start with character '\ufffd' You likely have the sequence "&\ufffd" in a parsed character data section of a document, and the parser, seeing the ampersand, knows t

Re: solr feed problem

2008-05-18 Thread Yonik Seeley
\ufffd isn't really a valid character. http://www.fileformat.info/info/unicode/char/fffd/index.html Your XML document or data probably had some kind of encoding issue along the way somewhere. -Yonik On Sun, May 18, 2008 at 7:59 PM, Cam Bazz <[EMAIL PROTECTED]> wrote: > hello, > > I am trying to f

solr feed problem

2008-05-18 Thread Cam Bazz
hello, I am trying to feed solr with xml files of my own schema, and I am getting: SEVERE: org.xmlpull.v1.XmlPullParserException: entity reference names can not start with character '\ufffd' my xml is utf8 for sure, as well as the text inside. but for some reason I get this exception and then so

Re: Some advice on scalability

2008-05-18 Thread Norberto Meijome
On Thu, 15 May 2008 09:23:03 -0700 "William Pierce" <[EMAIL PROTECTED]> wrote: [...] > > Our app in brief: We get merchant sku files (in either xml/csv) which we > process and index and make available to our site visitors to search. Our > current plan calls for us to support approx 10,000 me

Re: Some advice on scalability

2008-05-18 Thread Norberto Meijome
On Thu, 15 May 2008 12:54:25 -0700 (PDT) Otis Gospodnetic <[EMAIL PROTECTED]> wrote: > 5) Hardware recommendations are hard to do. While people may make > suggestions, the only way to know how *your* hardware works with *your* data > and *your* shards and *your* type of queries is by benchmarki