httpclient.ProtocolException using Solrj
Hi, I'm sending 15K records at once using Solrj (server.addBeans(...)) and have two threads writing to same index. One thread goes fine, but the second thread always fails with, org.apache.solr.client.solrj.SolrServerException: org.apache.commons.httpclient.ProtocolException: Unbuffered entity enclosing request can not be repeated. at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:470) at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:242) at org.apache.solr.client.solrj.request.UpdateRequest.process(UpdateRequest.java:259) at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:48) at org.apache.solr.client.solrj.SolrServer.addBeans(SolrServer.java:57) at com.apple.afterchat.indexer.solr.handler.BeanIndexHandler.indexData(BeanIndexHandler.java:44) at com.apple.afterchat.indexer.Indexer.indexData(Indexer.java:77) at com.apple.afterchat.indexer.Indexer.run(Indexer.java:39) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:637) Caused by: org.apache.commons.httpclient.ProtocolException: Unbuffered entity enclosing request can not be repeated. at org.apache.commons.httpclient.methods.EntityEnclosingMethod.writeRequestBody(EntityEnclosingMethod.java:487) at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:2114) at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1096) at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398) at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:417) Does anyone know what could be the problem? Thanks, -vivek
Re: Oracle Clob column with DIH does not turn to String
Yes, you are correct. But the documentation for DIH says the column names are case insensitive. That should be fixed. Here is what it says: = A shorter data-config In the above example, there are mappings of fields to Solr fields. It is possible to totally avoid the field entries in entities if the names of the fields are same (case does not matter) as those in Solr schema. Noble Paul നോബിള് नोब्ळ् wrote: > > it is very expensive to do a case insensitive lookup. It must first > convert all the keys to lower case and try looking up there. because > it may not be always in uppercase it can be in mixed case as well > > On Sat, Apr 4, 2009 at 12:58 AM, ashokc wrote: >> >> Happy to report that it is working. Looks like we have to use UPPER CASE >> for >> all the column names. When I examined the map 'aRow', it had the column >> names in upper case, where as my config had lower case. No match was >> found >> so nothing happened. Changed my config and it works now. Thanks for your >> help. Perhaps this transformer can be modified to be case-insensitive for >> the column names. If you had written it perhaps it is a quick change for >> you? >> >> Noble Paul നോബിള് नोब्ळ् wrote: >>> >>> I guess u can write a custom transformer which gets a String out of >>> the oracle.sql.CLOB. I am just out of clue, why this may happen. I >>> even wrote a testcase and it seems to work fine >>> --Noble >>> >>> On Fri, Apr 3, 2009 at 10:23 PM, ashokc wrote: I downloaded the nightly build yesterday (2nd April), modified the ClobTransformer.java file with some prints, compiled it all (ant dist). It produced a war file, "apache-solr-1.4-dev.war". That is what I am using. My modification & compilation has not affected the results. I was getting the same behavior with the 'war' that download came with. Thanks Noble. Noble Paul നോബിള് नोब्ळ् wrote: > > and which version of Solr are u using? > > On Fri, Apr 3, 2009 at 10:09 PM, ashokc wrote: >> >> Sure: >> >> data-config Xml >> === >> >> >> > url="jdbc:oracle:thin:@x" user="remedy" password="y"/> >> >> > query="SELECT >> mylog_ato, name_char, dsc FROM log_tbl"> >> >> >> >> >> >> >> >> === >> >> A search result on the field short_desc: >> -- >> >> >> 1.8670129 >> oracle.sql.c...@155e3ab >> 4486 >> Develop Rating functionality for QIN >> 2009-04-03T11:47:32.635Z >> >> >> >> >> >> Noble Paul നോബിള് नोब्ळ् wrote: >>> >>> There is something else wrong with your setup. >>> >>> can you just paste the whole data-config.xml >>> >>> --Noble >>> >>> On Fri, Apr 3, 2009 at 5:39 PM, ashokc wrote: Noble, I put in a few 'System.out.println' statements in the ClobTransformer.java file & remade the war. But I see none of these prints coming up in my 'catalina.out' file. Is that the right file to be looking at? As an aside, is 'catalina.out' the ONLY log file for SOLR? I turned on the logging to 'FINE' for everything. Also, these settings seem to go away when Tomcat is restarted. - ashok Noble Paul നോബിള് नोब्ळ् wrote: > > yeah, ant dist will give you the .war file you may need . just > drop > it > in and you are set to go. or if you can hook up a debugger to a > running Solr that is the easiest > --Noble > > On Fri, Apr 3, 2009 at 9:35 AM, ashokc > wrote: >> >> That would require me to recompile (with ant/maven scripts?) the >> source >> and >> replace the jar for DIH, right? I can try - for the first time. >> - ashok >> >> Noble Paul നോബിള് नोब्ळ् wrote: >>> >>> This looks strange. Apparently the Transformer did not get >>> applied. >>> Is >>> it possible for you to debug ClobTransformer >>> adding(System.out.println >>> into ClobTransformer may help) >>> >>> On Fri, Apr 3, 2009 at 6:04 AM, ashokc >>> wrote: Correcting my earlier post. It lost some lines some how. Hi, I have set up to import some oracle clob columns with DIH. I am using the latest nightly release. My config says, >>> ... >>> /> But it does not
Re: Oracle Clob column with DIH does not turn to String
On Sat, Apr 4, 2009 at 7:15 PM, ashokc wrote: > > Yes, you are correct. But the documentation for DIH says the column names > are > case insensitive. That should be fixed. Here is what it says: > > = > A shorter data-config > > In the above example, there are mappings of fields to Solr fields. It is > possible to totally avoid the field entries in entities if the names of the > fields are same (case does not matter) as those in Solr schema. > > > It says that the field definitions can be avoided if the field names in the entity and the schema.xml are the same. But if present in the data-config, they are supposed to have been provided in the correct case. -- Regards, Shalin Shekhar Mangar.
Search Within
I am not sure if this is a really easy or newbee-ish type question. I would like to implement a search within these results type feature. Has anyone done this and could you please share some tips, pointers and or documentation on how to implement this. Thanks Vern
Multiple Core schemas with single solr.solr.home
I am planning to configure a solr server with multiple cores with different schema for themselves with a single solr.solr.home . Are there any examples in the wiki to the wiki ( the ones that I see have a single schema.xml for a given solr.solr.home under schema directory. ). Thanks for helping pointing to the same.
Re: Multiple Core schemas with single solr.solr.home
On Sat, Apr 4, 2009 at 9:51 PM, Rakesh Sinha wrote: > I am planning to configure a solr server with multiple cores with > different schema for themselves with a single solr.solr.home . Are > there any examples in the wiki to the wiki ( the ones that I see have > a single schema.xml for a given solr.solr.home under schema directory. > ). > > Thanks for helping pointing to the same. > It should be possible though I don't there are any examples. You can specify the same instanceDir for different cores but different dataDir (specifying dataDir in solr.xml is a trunk feature) -- Regards, Shalin Shekhar Mangar.
Re: Search Within
Hi, Vernon. In Blacklight, the way we've been doing this is just to stack queries on top of each other. It's a conceptual shift from the way one might think about "search within", but it accomplishes the same thing. For example: search1 ==> q=horse search2 ==> q=horse AND dog The second search, from the user's point of view, takes the search results from the horse search and further narrows them to those items that also contain dog. But you're really just doing a new search, one that contains both search values. Does that help? Or am I misunderstanding your question? Bess On 4-Apr-09, at 12:10 PM, Vernon Chapman wrote: I am not sure if this is a really easy or newbee-ish type question. I would like to implement a search within these results type feature. Has anyone done this and could you please share some tips, pointers and or documentation on how to implement this. Thanks Vern
Re: Search Within
Bess, I think that might work I'll try it out and see how it works for my case. thanks Bess Sadler wrote: Hi, Vernon. In Blacklight, the way we've been doing this is just to stack queries on top of each other. It's a conceptual shift from the way one might think about "search within", but it accomplishes the same thing. For example: search1 ==> q=horse search2 ==> q=horse AND dog The second search, from the user's point of view, takes the search results from the horse search and further narrows them to those items that also contain dog. But you're really just doing a new search, one that contains both search values. Does that help? Or am I misunderstanding your question? Bess On 4-Apr-09, at 12:10 PM, Vernon Chapman wrote: I am not sure if this is a really easy or newbee-ish type question. I would like to implement a search within these results type feature. Has anyone done this and could you please share some tips, pointers and or documentation on how to implement this. Thanks Vern
Re: Search Within
This post describes the search-within-search implementation. http://sujitpal.blogspot.com/2007/04/lucene-search-within-search-with.html Shashi On Sat, Apr 4, 2009 at 1:21 PM, Vernon Chapman wrote: > Bess, > > I think that might work I'll try it out and see how it works for my case. > > thanks > > > Bess Sadler wrote: > >> Hi, Vernon. >> >> In Blacklight, the way we've been doing this is just to stack queries on >> top of each other. It's a conceptual shift from the way one might think >> about "search within", but it accomplishes the same thing. For example: >> >> search1 ==> q=horse >> >> search2 ==> q=horse AND dog >> >> The second search, from the user's point of view, takes the search results >> from the horse search and further narrows them to those items that also >> contain dog. But you're really just doing a new search, one that contains >> both search values. >> >> Does that help? Or am I misunderstanding your question? >> >> Bess >> >> On 4-Apr-09, at 12:10 PM, Vernon Chapman wrote: >> >> I am not sure if this is a really easy or newbee-ish type question. >>> I would like to implement a search within these results type feature. >>> Has anyone done this and could you please share some tips, pointers and >>> or documentation on how to implement this. >>> >>> Thanks >>> >>> Vern >>> >>> >> >>
Re: Multi-valued fields with DIH
That worked. Thanks again. Noble Paul നോബിള് नोब्ळ् wrote: > > the column names are case sensitive try this > > > > On Sat, Apr 4, 2009 at 3:58 AM, ashokc wrote: >> >> Hi, >> I need to assign multiple values to a field, with each value coming from >> a >> different column of the sql query. >> >> My data config snippet has lines like >> >> >> >> >> where 'project_area' & 'project_version' are output by the sql query to >> the >> datasource. The 'verbose-output' from dataimport.jsp does show that these >> columns have values returned by the query >> >> === >> >> >> - >> >> - >> >> + >> >> x >> >> 0:0:0.142 >> --- row #1- >> MySource/Area/Admin >> MySource/Version/06.02 >> 2008-10-21T07:00:00Z >> . >> >> == >> >> But the resulting index has no data in the field 'projects'. Is it NOT >> possible to create multi-valued fields with DIH? >> >> Thanks >> -- >> View this message in context: >> http://www.nabble.com/Multi-valued-fields-with-DIH-tp22877509p22877509.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> >> > > > > -- > --Noble Paul > > -- View this message in context: http://www.nabble.com/Multi-valued-fields-with-DIH-tp22877509p22886586.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Phrase Query Issue
On Apr 4, 2009, at 1:25 AM, dabboo wrote: Erik, Thanks a lot for your reply. I have made some changes in the solr code and now field clauses are working fine with dismax request. Not only this, wildcard characters are also working with dismax and q query parameter. If you want I can share modified code with you. That'd be good to share. Simply open a Solr JIRA issue with this enhancement request and post your code there. Test cases and documentation always appreciated too, but working code to start with is fine. Erik
Re: ExtractingRequestHandler Question
Hi TIA, I have the same desired requirement. If you look up in the archives, you might find a similar thread between myself and the always super helpful Erik Hatcher. Basically, it can't be done (right now). You can however use the "ExtractOnly" request handler, and just get the extracted text back from solr, and then use xpath to get out the attributes and then add them to your XML you are sending. Not ideal because the file has to be transfered twice. The only other option is to send the file as per the instructions via POST with its attributes as POST fields. Keep in mind that Solr documents are immutable, which means they cannot change. When you update a document with the same primary key, it will simply delete the existing one and add the new one. hth, Jacob On Sat, Apr 4, 2009 at 5:59 AM, Venu Mittal wrote: > Hi, > > I am using ExtractingRequestHandler to index rich text documents. > The way I am doing it is I get some data related to the document from > database and then post an xml (containing only this data ) to solr. Then I > make another call to solr, which sends the actual document to be indexed. > But while doing so I am loosing out all the other data that is related to the > document. > > Is this the right way to do handle it or am I missing out on something. > > TIA > > > > -- +1 510 277-0891 (o) +91 33 7458 (m) web: http://pajamadesign.com Skype: pajamadesign Yahoo: jacobsingh AIM: jacobsingh gTalk: jacobsi...@gmail.com
