Re: Lucene-based Distributed Index Leveraging Hadoop

2008-02-08 Thread Srikant Jakilinki
Hi Ning, In continuation with our offline conversation, here is a public expression of interest in your work and a description of our work. Sorry for the length in advance and I hope that the folk will be able to collaborate and/or share experiences and/or give us some pointers... 1) We are

Re: Memory improvements

2008-02-08 Thread Mike Klaas
On 7-Feb-08, at 3:29 PM, Sundar Sankaranarayanan wrote: Hi All, I am running an application in which I am having to index about 300,000 records of a table which has 6 columns. I am committing to the solr server after every 10,000 rows and I observed that the by the end of about 150,

RE: strange updating inconsistency

2008-02-08 Thread patrik
Chris, thanks for the response. The two documents attached were different docs, one of them was working and one wasn't. I found the problem, and it wasn't related to SOLR though for a long while it did to me. Sorry to have taken up your time. patrik -Original Message- From: Chris Hostette

RE: Many updates slow down SOLR performance, no commit/autocommit

2008-02-08 Thread Fuad Efendi
No... I just moved to master/slave, I believe it happened during 'merge' of uncommitted data... And I tuned merge factor and maxBufferedDocs, hope it will help... At least, I don't see any performance problem on Master with 600,000 updates since yesterday... > do you have a stack trace around the

Re: SolrJ and RawResponses

2008-02-08 Thread Ryan McKinley
Grant Ingersoll wrote: btw, is https://issues.apache.org/jira/browse/SOLR-402 the issue you were referring to? yes. -Grant On Feb 8, 2008, at 4:14 PM, Ryan McKinley wrote: I think Erik was looking at this earlier... I think there may be an open JIRA issue for it, but don't recall whic

Re: SolrJ and RawResponses

2008-02-08 Thread Yonik Seeley
On Feb 8, 2008 5:13 PM, Ryan McKinley <[EMAIL PROTECTED]> wrote: > Thinking about this some more... yes, the SolrRequest should say what > parser is used. It is the request that has to deal with the output in > the end. I could see it either way really... passed as a param to request() or as an

Re: SolrJ and RawResponses

2008-02-08 Thread Ryan McKinley
Thinking about this some more... yes, the SolrRequest should say what parser is used. It is the request that has to deal with the output in the end. Any change here need to make sure to address lines 125,126 of CommonsHttpSolrServer _invariantParams.set( CommonParams.WT, _processor.getWrit

Re: SolrJ and RawResponses

2008-02-08 Thread Grant Ingersoll
On Feb 8, 2008, at 4:39 PM, Yonik Seeley wrote: On Feb 8, 2008 4:27 PM, Grant Ingersoll <[EMAIL PROTECTED]> wrote: OK, I will look into it. setProcessor is only defined for the CommonsHttpSolrServer and I am trying to use the abstract SolrServer, but I guess I can cast. Also, the setProcesso

Re: SolrJ and RawResponses

2008-02-08 Thread Ryan McKinley
what about extending CommonsHttpSolrServer and setting the _processor in the constructor? I guess it depends on how things will be used. ryan Grant Ingersoll wrote: OK, I will look into it. setProcessor is only defined for the CommonsHttpSolrServer and I am trying to use the abstract SolrSe

Re: SolrJ and RawResponses

2008-02-08 Thread Grant Ingersoll
btw, is https://issues.apache.org/jira/browse/SOLR-402 the issue you were referring to? -Grant On Feb 8, 2008, at 4:14 PM, Ryan McKinley wrote: I think Erik was looking at this earlier... I think there may be an open JIRA issue for it, but don't recall which one off the top of my head.

Re: SolrJ and RawResponses

2008-02-08 Thread Yonik Seeley
On Feb 8, 2008 4:27 PM, Grant Ingersoll <[EMAIL PROTECTED]> wrote: > OK, I will look into it. setProcessor is only defined for the > CommonsHttpSolrServer and I am trying to use the abstract SolrServer, > but I guess I can cast. Also, the setProcessor isn't great in that it > isn't thread-safe (i

Re: SolrJ and RawResponses

2008-02-08 Thread Grant Ingersoll
OK, I will look into it. setProcessor is only defined for the CommonsHttpSolrServer and I am trying to use the abstract SolrServer, but I guess I can cast. Also, the setProcessor isn't great in that it isn't thread-safe (i.e. if I am reusing said SolrServer for other tasks in other thread

Re: SolrJ and RawResponses

2008-02-08 Thread Ryan McKinley
I think Erik was looking at this earlier... I think there may be an open JIRA issue for it, but don't recall which one off the top of my head. I think you want an ResponseParser that just stuffs the text into the named list. If you have any trouble, i can throw something together real quick.

SolrJ and RawResponses

2008-02-08 Thread Grant Ingersoll
I want to programmatically retrieve the schema and the config from the ShowFileRequestHandler. I have tried doing it (mostly just to see how it would handle it w/o looking at the code) using SolrJ, but get in an exception in the XMLResponseParser in SolrJ. Ideally, I guess SolrJ would stu

Re: Many updates slow down SOLR performance, no commit/autocommit

2008-02-08 Thread Grant Ingersoll
do you have a stack trace around the Lucene clone() stuff? -Grant On Feb 7, 2008, at 9:56 PM, Fuad Efendi wrote: Question: Why constant updates slow down SOLR performance even if I am not executing Commit? I just noticed this... Thead dump shows something "Lucene ... Clone()", and signifi

Re: Get Config / Schema, 1.3-dev Broken?

2008-02-08 Thread Ryan McKinley
add: class="org.apache.solr.handler.admin.AdminHandlers" /> to solr config, then check: /admin/file?file=solrconfig.xml This is using a requestHandler rather then a jsp file... CHANGES.txt exaplins this too ryan Matthew Runo wrote: Hello! Recently, using the latest SVN code, it se

RE: Lucene index verifier

2008-02-08 Thread Lance Norskog
Given the size of our index, using file checksums is more feasible. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Yonik Seeley Sent: Friday, February 08, 2008 5:10 AM To: solr-user@lucene.apache.org Subject: Re: Lucene index verifier If someone wanted

Re: Searching solr without qualified field

2008-02-08 Thread Yerraguntla
Nm, the fileds need to be defined as part of copyfield section just like in the example. silly me :( Yerraguntla wrote: > > Hi, > > Basic question :) > > I have added new xml data after changing the example's schema.xml. When > I am using admin qquery page. The example works given the v

Get Config / Schema, 1.3-dev Broken?

2008-02-08 Thread Matthew Runo
Hello! Recently, using the latest SVN code, it seems that the links to view the schema & config files have been broken. Urls such as /solr/admin/file/?file=solrconfig.xml result in a 404 error. Has anyone else noticed this behavior? I just wanted to point it out if so. Thanks! Matthew

RE: Memory improvements

2008-02-08 Thread Sundar Sankaranarayanan
Hi Lance, Thanks for the reply. But I am using solr 1.3 in my application. Could you send me sample code or something like that which I can check it out too. I hope I am following the api in the right manner. I am currently doing something like this SolrServer server = createNewSo

Searching solr without qualified field

2008-02-08 Thread Yerraguntla
Hi, Basic question :) I have added new xml data after changing the example's schema.xml. When I am using admin qquery page. The example works given the value of "video". But for my data the field is looking for fieldname:value to return the results. What should I do if i want to just search

Re: Search result not coming for normal special characters...

2008-02-08 Thread Erick Erickson
What analyzers are you using? Many analyzers (both index and query time) will remove non-alpha characters. Best Erick On Feb 7, 2008 1:14 PM, nithyavembu <[EMAIL PROTECTED]> wrote: > > Hi All, > > Now i am facing problem in special character search. > I tried with the following special charact

Re: Lucene index verifier

2008-02-08 Thread Yonik Seeley
If someone wanted those additional checks, it seems like the right place to hook it in would be the snapshooter or snapinstaller. -Yonik On Feb 8, 2008 8:04 AM, Grant Ingersoll <[EMAIL PROTECTED]> wrote: > I think Mike M. put up a tool called CheckIndex that is a simple > driver program that chec

Re: Lucene index verifier

2008-02-08 Thread Grant Ingersoll
I think Mike M. put up a tool called CheckIndex that is a simple driver program that checks for corruption. However, my understanding is that he isn't sure it is complete just yet, but it is a start. Have a look in the latest release. Maybe it would be useful to have it run either on star

Highlighting the search results

2008-02-08 Thread Laxmilal Menaria
Hello, I have created a Index of some xml files (xml's was created from mysql database) with solr. In xml's I have one text field. (more than 1000 characters). Its searches fine, but solr result page is too long. I have tried highlight (&hl=on&hl.fl=fname), its shows highlighting results at end of