Re: Upgrade to Solr 1.4, very slow at start up when loading all cores

2010-10-01 Thread Dennis Gearon
VERY interesting. Looking forward to what happens on the first queries. Dennis Gearon Signature Warning EARTH has a Right To Life, otherwise we all die. Read 'Hot, Flat, and Crowded' Laugh at http://www.yert.com/film.php --- On Fri, 10/1/10, Renee Sun wrote: > From: Renee

Re: Tuning solr

2010-10-01 Thread Gora Mohanty
On Sat, Oct 2, 2010 at 5:21 AM, Stavros Korokithakis wrote: [...] > Is there a guide for tuning solr somewhere? We have about a million > documents (the documents are 8 fields, one of which is the full text of > webpages) and we'd like to give solr a bit more memory and generally > tune it for bet

Re: Upgrade to Solr 1.4, very slow at start up when loading all cores

2010-10-01 Thread Renee Sun
http://lucene.472066.n3.nabble.com/file/n1617135/solrconfig.xml solrconfig.xml Hi Yonik, I have uploaded our solrconfig.xml file for your reference. we also tried 1.4.1, for same index data, it took about 30-55 minutes to load up all 130 cores, it did not help at all. There is no query running

Re: Upgrade to Solr 1.4, very slow at start up when loading all cores

2010-10-01 Thread Renee Sun
Hi Yonik, I attached the solrconfig.xml to you in previous post, and we do have firstSearch and newSearch hook ups. I commented them out, all 130 cores loaded up in 1 minute, same as in solr 1.3. total memory took about 1GB. Whereas in 1.3, with hook ups, it took about 6.5GB for same amount of

solruser

2010-10-01 Thread ankita shinde
hello, I am trying to use solrj for interfacing with solr. I am trying to run the SolrjTest example. I have included all the following jar files- - commons-codec-1.3.jar - commons-fileupload-1.2.1.jar - commons-httpclient-3.1.jar - commons-io-1.4.jar - geronimo-stax-api_1.0_spec-

Re: DataImportHandler Error CHARBytesToJavaChars

2010-10-01 Thread Lance Norskog
Oracle has a bunch of functions you can use in the SELECT statement to translate types. You may want to translate a NULL into an empty string. harrysmith wrote: Anyone ever see this error on an import? Caused by: java.lang.NullPointerException at oracle.jdbc.driver.DBConversion._CHARB

Re: Faster loading to solr...

2010-10-01 Thread Lance Norskog
Please start a new email thread for this instead of replying to an existing one with a new subject and question. Sharma, Raghvendra wrote: I have been able to load around a million rows/docs in around 5+ minutes. The schema contains around 250+ fields. For the moment, I have kept everything

Re: Automatic xslt to responses ??

2010-10-01 Thread Lance Norskog
Please start a new email thread instead of replying to an existing one with a new subject and question. Sharma, Raghvendra wrote: Is there a way to specify a xslt at the server side, and make it default, i.e. whenever a response is returned, that xslt is applied to the response automatically.

Re: UpdateXmlMessage

2010-10-01 Thread Lance Norskog
Yes. stream.file and stream.url are independent of the request handler. They do their magic at the very top level of the request. However, there are no unit tests for these features, but they are widely used. Tod wrote: I can do this using GET: http://localhost:8983/solr/update?stream.body=

Fwd: solr-user

2010-10-01 Thread ankita shinde
-- Forwarded message -- From: ankita shinde Date: Sat, Oct 2, 2010 at 8:54 AM Subject: solr-user To: solr-user@lucene.apache.org hello, I am trying to use solrj for interfacing with solr. I am trying to run the SolrjTest example. I have included all the following jar files-

Re: Multiple Indexes and relevance ranking question

2010-10-01 Thread Lance Norskog
The score of a document has no scale: it only has meaning against other score in the same query. Solr does not rank these documents correctly. Without sharing the TF/DF information across the shards, it cannot. If the shards each have "a lot" of the same kind of document, this problem averag

Re: SolrCore / Index Searcher Instances

2010-10-01 Thread Mark Miller
On 10/1/10 9:16 PM, Chris Hostetter wrote: > > : > it's the same instance, it's just listed twice. > : > > : > : This comes up again and again - and it is confusing - I wonder if we > : can't improve what's displayed to make this more clear. > > On stats.jsp (or using /admin/mbeans?stats=true)

Re: How to tell Solr to return all fields including empty fields?

2010-10-01 Thread Erick Erickson
Nope. If a field is empty, it's just not in the document. You could index an empty value (some unique value for empty) if you really need this. But a more interesting question is why you need this. What is it about returning an empty field that's important? Couldn't you detect this through the abs

Re: Problem with Indexing

2010-10-01 Thread Erick Erickson
How much memory are you giving the JVM? Try adding the -Xmx parameter to your java invocation. e.g. -Xmx512M HTH Erick On Fri, Oct 1, 2010 at 7:20 AM, Jörg Agatz wrote: > Hallo, > > i tryed to index a lot of XML. 700.000 to 800.000 but i becom a Error: and > a > Java heap Space error.. > > i doo

Re: Facet Counts Issue when Using Dismax Query Parser in SOLR

2010-10-01 Thread Chris Hostetter
: So I add a fq param to filter the results by the chosen facet. This : seems logical, but the number of search results I get is NOT the same as : the count against the facet. can you give us some specific examples? Ie: show us a URL, show us the requestHandler config for the handler used in

Re: SolrCore / Index Searcher Instances

2010-10-01 Thread Chris Hostetter
: > it's the same instance, it's just listed twice. : > : : This comes up again and again - and it is confusing - I wonder if we : can't improve what's displayed to make this more clear. On stats.jsp (or using /admin/mbeans?stats=true) it's fairly obvious -- they have the exact same searcherNa

Re: SolrCore / Index Searcher Instances

2010-10-01 Thread Mark Miller
On 10/1/10 8:01 PM, Chris Hostetter wrote: > > it's the same instance, it's just listed twice. > This comes up again and again - and it is confusing - I wonder if we can't improve what's displayed to make this more clear. - Mark

Re: Solr with example Jetty and score problem

2010-10-01 Thread Chris Hostetter
: But when I issue the query with shard(two instances), the response XML will : be like following. : as you can see, that score has bee tranfer to a element of ... : : 1.9808292 : The root cause of these seems to be your catchall dynamic field declaration... : ...that line (spe

Tuning solr

2010-10-01 Thread Stavros Korokithakis
Hello all, I'm sure this has been asked many times before, but I couldn't find anything from browsing a few months: Is there a guide for tuning solr somewhere? We have about a million documents (the documents are 8 fields, one of which is the full text of webpages) and we'd like to give solr a bit

Re: SolrCore / Index Searcher Instances

2010-10-01 Thread Chris Hostetter
: This may seem like a stupid question, but why on the info / stats pages do we : see two instances on SolrIndexSearcher? There will always be two "SolrIndexSearcher" entries on the info & stats pages -- the curret search is listed once under it's "real" name and once as "searcher" (i think it

Re: Sorting individually by each keyword.

2010-10-01 Thread Chris Hostetter
Your email has been specific about one example, but vague about what your general goal is. Based purely on what youv'e said the QueryElevationComponent seems like it would meet your needs -- but it's hard to be sure w/o more details. : I want to sort individually by each keyword. : : ex:) :

Re: Solr Cluster Indexing data question

2010-10-01 Thread Chris Hostetter
: Subject: Solr Cluster Indexing data question : References: <1285812108551-1606159.p...@n3.nabble.com> : <1e8e3ada-ed02-410d-b4c6-c061846c2...@progressivealliance.co.uk> http://people.apache.org/~hossman/#threadhijack Thread Hijacking on Mailing Lists When starting a new discussion on a mai

Estimating memory use for Solr caches

2010-10-01 Thread Burton-West, Tom
We are having some memory and GC issues. I'm trying to get a handle on the contribution of the Solr caches. Is there a way to estimate the amount of memory used by the documentCache and the queryResultCache? I assume if we know the average size of our stored fields we can just multiply the s

RE: any working SolrJ code example for Solr 1.4.1

2010-10-01 Thread Xin Li
Thanks, Jon. I have just made my Perl version of solution working, will try your tips this weekend. Xin -Original Message- From: Sharp, Jonathan [mailto:jsh...@coh.org] Sent: Fri 10/1/2010 1:20 PM To: solr-user@lucene.apache.org Subject: RE: any working SolrJ code example for Solr 1.

Question

2010-10-01 Thread Bill Bell
I have a question concerning SOLR 5 Spatial. I would like to have a field with multiple LAT/LONG values tied to one ID (multiple offices for a Doctor) is this possible? Can I do multi-values? I think I could add multiple fields, but those fields would not be sortable across 3 fiel

Re: Problems with RAMDirectory in Solr

2010-10-01 Thread Chris Hostetter
: Hello. We just recently started using a RAMDirectory with Solr and found a : problem. When we restart the Solr, the RAMDirectory is refreshed as : expected. Hwoever when we use the snapinstaller script to update the index, : the RAMDirectory is not updated. Is there any way to update the RAMDire

Re: Upgrade to Solr 1.4, very slow at start up when loading all cores

2010-10-01 Thread Yonik Seeley
On Thu, Sep 30, 2010 at 4:52 PM, Renee Sun wrote: >  - do you have any warming queries configured? > > no, all autowarmingcount are set to 0 for all caches Any static warming requests though (newSearcher / firstSearcher hooks in solrconfig.xml)? Is anything at all querying these cores while y

How to tell Solr to return all fields including empty fields?

2010-10-01 Thread Khai Doan
Hello everyone, Right now, I am using fl=*,score to get all fields from Solr. However empty fields are not returned. Is there a way for me to tell Solr to return all fields including empty fields? Thanks, Khai

SolrJ - CommonsHttpSolrServer

2010-10-01 Thread Christopher Gross
I have SolrJ working, but it is running slower than I'd like. The logger spits out a lot of things that make me feel like I don't have something configured quite right. Instead of holding on to the connection, I think that it closes it each time. I'd want it to stay open -- I'm trying to do a cl

Re: ANNOUNCE: Stump Hoss @ Lucene Revolution

2010-10-01 Thread Chris Hostetter
FYI, Less then a week to go to submit your questions for my "Stump The Chump" session at Lucene Revolution: http://bit.ly/stump-hoss Some updates since my last email: 1) Prizes have been announced: http://bit.ly/stump-hoss-prizes 2) The Judges will be Mark Miller, Grant Ingersoll, and Yonik Se

RE: any working SolrJ code example for Solr 1.4.1

2010-10-01 Thread Sharp, Jonathan
Xin, I also had a similar error when I picked up SolrJ. See the first section of this wiki page for the extra jars (the ones not found in the dist directory): http://wiki.apache.org/solr/Solrj -Jon -Original Message- From: Xin Li [mailto:x...@book.com] Sent: Friday, October 01, 2010 8

DebugComponent behavour in a distributed environment

2010-10-01 Thread Edoardo Tosca
Hello everybody, i have some doubts about the current behaviour of DebugComponent at coordinator level in a sharded environment. I'm actually using Solr 1.4 While trying to test our current system using debugQuery=on i have seen that at coordinator level the timing element contains riduculous value

Re: any working SolrJ code example for Solr 1.4.1

2010-10-01 Thread Allistair Crossley
did you miss the page here http://wiki.apache.org/solr/Solrj ? this tells you the jars required for your classpath as well as usage examples On Oct 1, 2010, at 11:57 AM, Xin Li wrote: > That's precisely the reason I was asking about JARs too. It seems that I > am the minority that ran into Solr

RE: any working SolrJ code example for Solr 1.4.1

2010-10-01 Thread Xin Li
That's precisely the reason I was asking about JARs too. It seems that I am the minority that ran into SolrJ issue. If that's the case, I will grab Perl solution, and come back to SolrJ later. Thanks, Xin -Original Message- From: Allistair Crossley [mailto:a...@roxxor.co.uk] Sent: Frida

Re: any working SolrJ code example for Solr 1.4.1

2010-10-01 Thread Allistair Crossley
no example anyone gives you will solve your class not found exception .. you need to ensure the relevant jars (in dist) are included in your solr instance's lib folder i guess? On Oct 1, 2010, at 10:50 AM, Xin Li wrote: > Hi, there, > > Just picked up SolrJ few days ago. I have my Solr Server

UpdateXmlMessage

2010-10-01 Thread Tod
I can do this using GET: http://localhost:8983/solr/update?stream.body=%3Cdelete%3E%3Cquery%3Eoffice:Bridgewater%3C/query%3E%3C/delete%3E http://localhost:8983/solr/update?stream.body=%3Ccommit/%3E ... but can I pass a stream.url parameter using an UpdateXmlMessage? I looked at the schema and

any working SolrJ code example for Solr 1.4.1

2010-10-01 Thread Xin Li
Hi, there, Just picked up SolrJ few days ago. I have my Solr Server set up, data loaded, and everything worked fine with the web admin page. Then problem came when I was trying to use SolrJ to interact with the Solr server. I was stuck with "NoClassNotFoundException" yesterday. Being new to the d

Re: error sending a delete all request

2010-10-01 Thread Christopher Gross
Looking into it more, trying to issue a *:* doesn't work on either index. I went back to a different setup of tomcat and solr (just one solr instance), and the command did run. So I'm guessing that I did misconfigure something, but I have no idea what. Now I'm trying to set up a separate instanc

Facet Counts Issue when Using Dismax Query Parser in SOLR

2010-10-01 Thread Jason Brown
I am retrieving facet counts against a specific column in my index and these look accurate. The query for retrieving these counts is also running a dismax search using the q param (against 4 columns in my index, 1 of which I am facet counting on as mentioned above). So far, so good.

Re: Solr UIMA integration

2010-10-01 Thread Tommaso Teofili
Hi Mahesh , 2010/10/1 maheshkumar > > Thanks a lot for uploading the relevant dependencies jars. The issue was > bcoz of java heap size i increased the heap and the issue was resolved. > > I am happy it solved your issue. > Now i am getting 403 error while connecting to > http://api.opencalais

Re: Solr UIMA integration

2010-10-01 Thread maheshkumar
Hi Tommaso, Thanks a lot for uploading the relevant dependencies jars. The issue was bcoz of java heap size i increased the heap and the issue was resolved. Now i am getting 403 error while connecting to http://api.opencalais.com/enlighten/calais.asmx/Enlighten webservice. Do i need to registry

Re: Any way to "append" new text to an existing indexed field?

2010-10-01 Thread Allistair Crossley
if your question + answers form a compound document then the whole document (with given unique id, e.g. question id) needs to be reindexed i think. best i could find with google was this ... https://issues.apache.org/jira/browse/SOLR-139 On Oct 1, 2010, at 8:23 AM, Andy wrote: > Well I want to

Re: Any way to "append" new text to an existing indexed field?

2010-10-01 Thread Andy
Well I want to just display the "title" of the question in my search results and users can then just click on it to see the detals of the question and all the answers. For example, say a question has the title "What is the meaning of life?" and then one of the answers to that question is "solr.

Re: Local Solr, Spatial Search, and LatLonType clarification

2010-10-01 Thread webdev1977
Thank you!! I am not sure if I can take advantage of this however, because my entry point into the solr index is through nutch. I have to map the nutch --> solr fields in the solrindex-mapping.xml file. I am not sure if dynamic fields are supported :-( What version of solr should I look at u

Re: Any way to "append" new text to an existing indexed field?

2010-10-01 Thread Allistair Crossley
i would say question and answer are 2 different entities. if you are using the data import handler, i would personally create them as separate entities with their own queries to the database using the deltaQuery method to pick up only new rows. i guess it depends if you need question + answers t

Re: [PECL-DEV] Re: PHP Solr API

2010-10-01 Thread Israel Ekpo
Scott, You can also use the SolrClient::setServlet() method with SolrClient::TERMS_SERVLET_TYPE as the type http://www.php.net/manual/en/solrclient.setservlet.php On Fri, Oct 1, 2010 at 12:57 AM, Scott Yeadon wrote: > Hi, > > Sorry, scrap that, just found that SolrQuery is a subclass of > Mo

Any way to "append" new text to an existing indexed field?

2010-10-01 Thread Andy
I'm building a Q&A application. There's a "Question" database table and an "Answer" table. For each question, I'm putting the question itself plus all the answers into a single field "text" to be indexed and searched. Say I have a question that has 10 existing answers that are already indexed.

Problem with Indexing

2010-10-01 Thread Jörg Agatz
Hallo, i tryed to index a lot of XML. 700.000 to 800.000 but i becom a Error: and a Java heap Space error.. i doo this: find -maxdepth 1 -mindepth 1 -name '*_SEARCH*' -exec java -jar /opt/solr/apache-solr-1.4.0/example/exampledocs/post.jar '{}' + and i get this: ... ... ... ... SimplePostTool:

Too many SocketTimeoutException from solr server

2010-10-01 Thread 高甜甜
Hi, I using solr as my indexing server, but i get some error now. I deploy solr to tomcat alone, i use solrj to communication with solr server. I get many socketTimeoutException when i use solrj to execute a query. The exception occurs very often when the query and update operation executed at

Re: can i have more update processors with solr

2010-10-01 Thread Jan Høydahl / Cominvent
I think the parameter name is confusing. I have proposed renaming it to processor.chain: https://issues.apache.org/jira/browse/SOLR-2105 -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com On 30. sep. 2010, at 22.25, Markus Jelsma wrote: > Almost, you can define a update

RE: Solr Cluster Indexing data question

2010-10-01 Thread ZAROGKIKAS,GIORGOS
Thanks Steve I'll check it out -Original Message- From: Steve Cohen [mailto:mail4st...@gmail.com] Sent: Thursday, September 30, 2010 9:30 PM To: solr-user@lucene.apache.org Subject: Re: Solr Cluster Indexing data question So how would one set it up to use multiple nodes for building an i

RE: Solr Cluster Indexing data question

2010-10-01 Thread ZAROGKIKAS,GIORGOS
Thank you very much Jak I 'll take a look -Original Message- From: Jak Akdemir [mailto:jakde...@gmail.com] Sent: Thursday, September 30, 2010 5:58 PM To: solr-user@lucene.apache.org Subject: Re: Solr Cluster Indexing data question If you want to use both of your nodes for building inde

Prioritizing advectives in solr search

2010-10-01 Thread Hasnain
Hi, My question is related to search results giving less importance to adjectives, here is my scenario, im using dismax handler and my understanding is when I query "Blue hammer", solr brings me results for "blue hammer", "blue" and "hammer", and in the same hierarchy, which is understandabl