using HTTPClient sending solr ping request wont timeout as specified

2010-10-12 Thread Renee Sun
I am using the following code to send out solr request from a webapp. please notice the timeout setting: HttpClient client = new HttpClient(); HttpMethod method = new GetMethod(solrReq); method.getParams().setParameter(HttpConnectionParams.SO_TIMEOUT,

Re: using HTTPClient sending solr ping request wont timeout as specified

2010-10-12 Thread Renee Sun
I also added the following timeout for the connection, still not working: client.getParams().setSoTimeout(httpClientPingTimeout); client.getParams().setConnectionManagerTimeout(httpClientPingTimeout); -- View this message in context: http://lucene.472066.n3.nabble.com/

Re: Prioritizing adjectives in solr search

2010-10-12 Thread Ron Mayer
Erick Erickson wrote: > You can do some interesting things with payloads. You could index a > particular value as the payload that identified the "kind" of word it was, > where "kind" is something you define. Then at query time, you could > boost depending on what part kind of word you identified i

Re: Problem with Indexing

2010-10-12 Thread Gora Mohanty
On Tue, Oct 12, 2010 at 5:57 PM, Jörg Agatz wrote: > java -Xmx1024m -jar start.jar Hmm, could you try increasing the RAM, though this out to be enough? Maybe I missed this earlier, but what is the size of the data being indexed? Regards, Gora

About setting solrconfig.xml

2010-10-12 Thread Jason, Kim
Hi, all. I got some question about solrconfig.xml. I have 10 fields in a document for index. (Suppose that field names are f1, f2, ... , f10.) Some user will want to search in field f1 and f5. Another user will want to search in field f2, f3 and f7. I am going to use dismax handler for this. How

Re: Spatial search in Solr 1.5

2010-10-12 Thread Yonik Seeley
On Tue, Oct 12, 2010 at 8:07 PM, PeterKerk wrote: > > Ok, so does this actually say: > for now you have to do calculations based on bounding box instead of great > circle? I tried to make the documentation a little simpler... there's - geofilt... filters within a radius of "d" km (i.e. "great c

Re: Error loading class 'solr.ASCIIFoldingFilterFactory'

2010-10-12 Thread Lance Norskog
Correct. We used the Latin1 filter back then. Chris Hostetter wrote: : I am using solr 1.3. I get the below mentioned error when included the : solr.ASCIIFoldingFilterFactory on 'text' field while index and query : time: I fairly certain ASCIIFoldingFilterFactory did not exist in Solr 1.3. -Ho

Re: Yet another date format problem

2010-10-12 Thread Lance Norskog
2007-06-19 09:08:48 Solr's input format is '2007-06-19T09:08:48Z'. More to the point: you are creating a string and passing that in. The date type will accept this, but the DIH has code to accept Java JDBC datetime values directly. So, in your select you want to somehow cast your field data

Re: Yet another date format problem

2010-10-12 Thread Erick Erickson
Hmmm. If this is a Solr date, then these are not in a proper Solr date format, see: http://lucene.apache.org/solr/api/org/apache/solr/schema/DateField.html Solr took the path of eliminating all the fuzziness about times and #requiring# that all dates be in Zulu time since that's unambiguous. Or at

Re: Spatial search in Solr 1.5

2010-10-12 Thread PeterKerk
Ok, so does this actually say: for now you have to do calculations based on bounding box instead of great circle? And the fact that on top of the page it says "Solr4.0", does that imply I cant use this right now? Or where could I find the latest trunk for this? (and ofcourse this might all change

Re: Spatial search in Solr 1.5

2010-10-12 Thread Yonik Seeley
You may want to check the docs, which were recently updated to reflect the state of trunk: http://wiki.apache.org/solr/SpatialSearch -Yonik http://www.lucidimagination.com On Tue, Oct 12, 2010 at 7:49 PM, PeterKerk wrote: > > Hey Grant, > > Just came accross this post of yours. > > Run a query

Re: Spatial search in Solr 1.5

2010-10-12 Thread PeterKerk
Hey Grant, Just came accross this post of yours. Run a query: http://localhost:8983/solr/select/?q=_val_:"recip(dist(2, store, vector(34.0232,-81.0664)),1,1,0)"&fl=*,score // Note, I just updated this, it used to be point instead of vector and that was wrong. What does your suggested query ac

Re: dynamic "stop" words?

2010-10-12 Thread Chris Hostetter
: Is it possible to have certain query terms not effect score, if that : same query term is present in a field? For example, I have an index of that use case is precisely what the DisjunctionMaxQuery (generated by the dismax parser) does for you if you set the "tie" param to "0" when one of the

Re: Trouble with exception Document [Null] missing required field DocID

2010-10-12 Thread Erick Erickson
You mean I should actually read the #entire# post before responding? What an idea! Thanks for helping out here, I was completely misleading Ahson. Erick On Mon, Oct 11, 2010 at 7:25 PM, Chris Hostetter wrote: > > : Right. You're requiring that every document have an ID (via uniqueKey), > but >

Re: Accented Search in Solr

2010-10-12 Thread Chris Hostetter
: Subject: Accented Search in Solr : References: : In-Reply-To: http://people.apache.org/~hossman/#threadhijack Thread Hijacking on Mailing Lists When starting a new discussion on a mailing list, please do not reply to an existing message, instead start a fresh email. Even if you change the

Re: Error loading class 'solr.ASCIIFoldingFilterFactory'

2010-10-12 Thread Chris Hostetter
: I am using solr 1.3. I get the below mentioned error when included the : solr.ASCIIFoldingFilterFactory on 'text' field while index and query : time: I fairly certain ASCIIFoldingFilterFactory did not exist in Solr 1.3. -Hoss

NPE for a MLT query on a missing doc due to null facet_counts in solrj

2010-10-12 Thread Wilkes, Chris
With solrj doing a more like this query for a missing document: /mlt?q=docId:SomeMissingId always throws a null pointer exception: Caused by: org.apache.solr.client.solrj.SolrServerException: Error executing query at org .apache .solr.client.solrj.request.QueryRequest.process(Query

Re: useFastVectorHighlighter creates fragments with cut off terms, incomplete

2010-10-12 Thread Trey Hyde
That must be it, thanks for the find. The patch must not have made it into 3.1 yet. I'll take a look at the diff and see what I can make work for our app in the meantime. On Oct 12, 2010, at 12:01 PM, Markus Jelsma wrote: > I believe it was this one > https://issues.apache.org/jira/browse/

Re: Webservice for push indexing

2010-10-12 Thread Jan Høydahl / Cominvent
Hi, I would advise you to get involved in the SolrCloud initiative (see http://wiki.apache.org/solr/SolrCloud) and start designing a native indexing distributor component. I envision something like an integration in UpdateHandler which knows about all collections and shards from ZK config, and

Re: Solr PHP PECL Extension going to Stable Release - Wishing for Any New Features?

2010-10-12 Thread Israel Ekpo
On Tue, Oct 12, 2010 at 12:44 PM, Ken Stanley wrote: > > > > > > If you are using Solr via PHP and would like to see any new features > in > > > the > > > > extension please feel free to send me a note. > > > > I'm new to this list, but in seeing this thread - and using PHP SOLR - I > wanted to m

Re: Use of ExternalFileField

2010-10-12 Thread Lance Norskog
EFF was written in 2007. It's quite solid. It can be used for sorting and boosts. I don't know if function queries can use it. The value will not be returned in queries. On Tue, Oct 12, 2010 at 5:14 AM, Roland Villemoes wrote: > Hi > > I have an databasefield that changes a lot which I want to u

Re: multi cores vs filter queries for a multi tenant deployment

2010-10-12 Thread Lance Norskog
For multi-tenant, all customers share the same words for text fields. A customer's search relevance might be useless. Also, spelling & autosuggest recommendations come from all of the entries in a field, so the customers can see each other's words. There are other quirks in memory and cache manage

Error loading class 'solr.ASCIIFoldingFilterFactory'

2010-10-12 Thread Sethi, Parampreet
Hi All, I am using solr 1.3. I get the below mentioned error when included the solr.ASCIIFoldingFilterFactory on 'text' field while index and query time: SEVERE: org.apache.solr.common.SolrException: Error loading class 'solr.ASCIIFoldingFilterFactory' at org.apache.solr.core.SolrResou

Re: dynamic "stop" words?

2010-10-12 Thread Matt Mitchell
Exactly yep. I think that'll work nicely. Thanks Jonathan, Matt On Tue, Oct 12, 2010 at 9:47 AM, Jonathan Rochkind wrote: > You can identify what words are the city name at index time, because they're > the ones in the "city" field, right? So why not just strip those words out at > index time?

Webservice for push indexing

2010-10-12 Thread bbarani
Hi, I am thinking of creating a webservice for performing push indexing across multiple solr instance. This is our SOLR configuration 3 Master instance ---> 3 slave instance. Each master maintains index of a particular item type. This is my idea.. We have a update UI which will push the SOL

Re: synonym configuration logic require

2010-10-12 Thread Jonathan Rochkind
I am not sure there's any way to get query-time multi-word synonyms to work right, because of the way the standard and dismax query parsers 'pre-tokenize' (I don't know what the solr developers call this phenomenon? I call it "pre-tokenizing"). If you use the field query parser instead, it w

Re: useFastVectorHighlighter creates fragments with cut off terms, incomplete

2010-10-12 Thread Markus Jelsma
I believe it was this one https://issues.apache.org/jira/browse/LUCENE-1824 > The natives have been getting restless since I turned off the highlighter > on our application quite a few months ago.In an attempt to rectify > that I've migrated to Solr 3.1 (found it quite stable so far) in hopes

useFastVectorHighlighter creates fragments with cut off terms, incomplete

2010-10-12 Thread Trey Hyde
The natives have been getting restless since I turned off the highlighter on our application quite a few months ago.In an attempt to rectify that I've migrated to Solr 3.1 (found it quite stable so far) in hopes to use the vector highlighter.So far, I'm not seeing a significant performa

Installation Solr 1.4 + Tika

2010-10-12 Thread Andreas Jung
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 What is the right way for installing Tika with Solr 1.4? Where do those jar files go? ./tika-app/target/tika-app-0.7.jar ./tika-bundle/target/tika-bundle-0.7.jar ./tika-core/target/tika-core-0.7.jar ./tika-parsers/target/tika-parsers-0.7.jar Documen

Re: Solr PHP PECL Extension going to Stable Release - Wishing for Any New Features?

2010-10-12 Thread Ken Stanley
> > > > If you are using Solr via PHP and would like to see any new features in > > the > > > extension please feel free to send me a note. > I'm new to this list, but in seeing this thread - and using PHP SOLR - I wanted to make a suggestion that - while minor - I think would greatly improve the

synonym configuration logic require

2010-10-12 Thread Jonty Rhods
Hi all, I am pretty new in solr. I want to configure synonym for forwarding result (for exact match). my schema is : There is name field which is define as text_sync. When I am searching by "newyork show" its working fine and givin

Re: Solr PHP PECL Extension going to Stable Release - Wishing for Any New Features?

2010-10-12 Thread Israel Ekpo
On Tue, Oct 12, 2010 at 8:43 AM, Stefan Matheis < matheis.ste...@googlemail.com> wrote: > Hi Isreal, > > On Mon, Oct 11, 2010 at 7:03 AM, Israel Ekpo wrote: > > > If you are using Solr via PHP and would like to see any new features in > the > > extension please feel free to send me a note. > > >

Re: Solr PHP PECL Extension going to Stable Release - Wishing for Any New Features?

2010-10-12 Thread Israel Ekpo
On Tue, Oct 12, 2010 at 7:42 AM, Peter Blokland wrote: > hi, > > On Mon, Oct 11, 2010 at 01:03:07AM -0400, Israel Ekpo wrote: > > > If you are using Solr via PHP and would like to see any new features in > the > > extension please feel free to send me a note. > > I'm currently testing a setup wit

Re: LuceneRevolution - NoSQL: A comparison

2010-10-12 Thread Dennis Gearon
Some very interesting scenarious cropping up here. Sounds to me like a minor architectural changes will be in order for them to be addressed. What's the usual delay from stage one to stage 2. IF that were some kind of constant value, old versions could be kept around for approx doulbe that time.

Re: LuceneRevolution - NoSQL: A comparison

2010-10-12 Thread Jan Høydahl / Cominvent
This is a different issue. You are seeing the latency between master index update and replication to slave(s). Solve this by pointing your monitoring script directly to slave instead of master. What this thread is about is a potential difference in state during the execution of a single sharded

Re: multicore replication slave

2010-10-12 Thread Christopher Bottaro
Answered my own question. Instead of naming each core in the replication handler, you use a variable instead: http://solr.mydomain.com:8983/solr/${solr.core.name}/replication 00:00:60 That will get all of your cores replicating. -- C On Mon, Oct 11, 2010 at 6:25 PM, Christopher

Re: LuceneRevolution - NoSQL: A comparison

2010-10-12 Thread Shawn Heisey
On 10/11/2010 6:32 PM, Peter Keegan wrote: When Solr does a distributed search across shards, it does this in 2 phases (correct me if I'm wrong): 1. 1st query to get the docIds and facet counts 2. 2nd query to retrieve the stored fields of the top hits The problem here is that the index could

Yet another date format problem

2010-10-12 Thread Allistair Crossley
Afternoon, I wonder if anyone could shed any light on this one that's stumping me. I have a table that is storing dates using seconds since epoch, so something like this: 1182262980 When I first wrote my queries for DIH it was complaining about this and rightly so as I had a mapper thus So

Solr SnapPuller fails to clear Old Index Directories ....

2010-10-12 Thread Jayendra Patil
We are working on the Solr trunk and have a Master and Two slaves configuration . Our indexing consists of Periodic Full and Incremental index building on the master and replication on the slaves. When a Full indexing (clean rebuilt) is performed, we always end with an extra index folder c

Re: Sorting individually by each keyword.

2010-10-12 Thread jjml jkl
I'm sorry to late reply.Thanks for the reply. I want to control the ranking of each keyword freely in a large number of documents. - regardless of the actual keywords in the document. - and doesn't using XML configure file. example:) There are Doc1,Doc2,Doc3Doc1000 - more than thousand. It

RE: dynamic "stop" words?

2010-10-12 Thread Jonathan Rochkind
You can identify what words are the city name at index time, because they're the ones in the "city" field, right? So why not just strip those words out at index time? Create a new field, name_search, and search on that, not name. Doc 1 name => "Holiday Inn" name_search => "Holiday Inn" [ana

Re: dynamic "stop" words?

2010-10-12 Thread Matt Mitchell
Thanks for the feedback. I thought about stop words but since I have a lot of documents spanning lots of different countries, I won't know all of the possible cities so stop-words could get hard to manage. Also, the city name is in the same field. I think I might try creating a new field called nam

Re: Solr PHP PECL Extension going to Stable Release - Wishing for Any New Features?

2010-10-12 Thread Stefan Matheis
Hi Isreal, On Mon, Oct 11, 2010 at 7:03 AM, Israel Ekpo wrote: > If you are using Solr via PHP and would like to see any new features in the > extension please feel free to send me a note. we actually tried to grab some informations from solr's dataimport-page, but therefore we had to generate

Re: Problem with Indexing

2010-10-12 Thread Jörg Agatz
java -Xmx1024m -jar start.jar

Re: Index time boosting is not working with boosting value in document level

2010-10-12 Thread Ahmet Arslan
> Thanks a ton Eric and Iorixxx. > Since I used DIH to import XML data, boosting was not > reflected in score. > When I used /update to import XML data, boosting was > considered for scoring > and able to see the intended data on top 10 results. > Could you please let me know how to make boosting v

Use of ExternalFileField

2010-10-12 Thread Roland Villemoes
Hi I have an databasefield that changes a lot which I want to use for ranking my results. Is the ExternalFileField the way to go and is it supported in the latest stable release of SOLR? (Do I have to go for trunk, or should I stick with 1.4 - the lucidWorks certified distribution?) My idea is

Re: Replication and CPU

2010-10-12 Thread Peter Karich
Hi Olivier, the index size is relative big and you enabled replication after startup: startup This could explain why the slave is replicating from the very beginning. Are the index versions/generations the same? (via command or admin/replication) If not, the slaves tries to replicate and if that

Re: Solr PHP PECL Extension going to Stable Release - Wishing for Any New Features?

2010-10-12 Thread Peter Blokland
hi, On Mon, Oct 11, 2010 at 01:03:07AM -0400, Israel Ekpo wrote: > If you are using Solr via PHP and would like to see any new features in the > extension please feel free to send me a note. I'm currently testing a setup with Solr via PHP, and was wondering if support for the ExtractingRequestH

Re: Solr PHP PECL Extension going to Stable Release - Wishing for Any New Features?

2010-10-12 Thread Israel Ekpo
On Mon, Oct 11, 2010 at 3:33 AM, Lukas Kahwe Smith wrote: > > On 11.10.2010, at 07:03, Israel Ekpo wrote: > > > I am currently working on a couple of bug fixes for the Solr PECL > extension > > that will be available in the next release 0.9.12 sometime this month. > > > > http://pecl.php.net/packa

Re: Replication and CPU

2010-10-12 Thread Olivier RICARD
Hello Peter, On the slave server http://slave/solr/core0/admin/replication/index.jsp Poll Interval00:30:00 Local Index Index Version: 1284026488242, Generation: 13102 Location: /solr/multicore/core0/data/index Size: 26.9 GB Times Replicated Since Startup: 289 Previous Replication

Re: deleteByQuery issue

2010-10-12 Thread Claudio Atzori
Hi Eick, thanks for replying. Yes I do commit after deleting-by-query, but since the IndexReader is an internal (at lest for my app point of view), is there a way to reopen it (or a new one) when I do a commit? Claudio On 10/11/2010 10:08 PM, Erick Erickson wrote: I'd guess that after you d

Re: Replication and CPU

2010-10-12 Thread Peter Karich
Hi Olivier, maybe the slave replicates after startup? check replication status here: http://localhost/solr/admin/replication/index.jsp what is your poll frequency (could you paste the replication part)? Regards, Peter. > Hello, > > I setup a server for the replication of Solr. I used 2 cores an

Re: multi cores vs filter queries for a multi tenant deployment

2010-10-12 Thread Tharindu Mathew
Basically, for a large number of users would using a single index or using a multi core approach be better? On Tue, Oct 12, 2010 at 11:39 AM, Tharindu Mathew wrote: > Hi everyone, > > I'm sort of looking in to a deployment which will support multi tenancy. > This means that there will be 1000s of

Re: Index time boosting is not working with boosting value in document level

2010-10-12 Thread Shanmugavel SRD
Thanks a ton Eric and Iorixxx. Since I used DIH to import XML data, boosting was not reflected in score. When I used /update to import XML data, boosting was considered for scoring and able to see the intended data on top 10 results. Could you please let me know how to make boosting value to be re

Re: Faceting and omitNorms=true

2010-10-12 Thread Markus Jelsma
You can safely set it to 0 (default) for solr.Textfield types that you don't do sloppy phrase queries on. From the schema.xml comments: > The optional positionIncrementGap puts space between multiple fields > of this type on the same document, with the purpose of preventing > false phrase matchin

Replication and CPU

2010-10-12 Thread Olivier RICARD
Hello, I setup a server for the replication of Solr. I used 2 cores and for each one I specified the replication. I followed the tutorial on http://wiki.apache.org/solr/SolrReplication. The replication is OK for each cores. However the CPU is used to 100% on the slave. The master and slave a

Re: Faceting and omitNorms=true

2010-10-12 Thread Paolo Castagna
Thank you Markus for your quick reply. Do you have a recommendation|suggestion about positionIncrementGap? Paolo Markus Jelsma wrote: You can omit norms in field that you only facet on. The matrix only lists mandatory values for those parameters. On Tuesday, October 12, 2010 10:15:35 am Paol

Re: Faceting and omitNorms=true

2010-10-12 Thread Markus Jelsma
You can omit norms in field that you only facet on. The matrix only lists mandatory values for those parameters. On Tuesday, October 12, 2010 10:15:35 am Paolo Castagna wrote: > Hi, > I am not completely sure on what's the recommended setting for fields > used for faceting regarding omitNorms and

Re: LuceneRevolution - NoSQL: A comparison

2010-10-12 Thread Jan Høydahl / Cominvent
This is what FAST does in ESP. When a new version of a partition is built, it is staged in its own process and co-exists alongside the old one. The query-dispatcher sees both and routes traffic based on requested "generation id". Should probably not invest in such a feature until there's a clea

Faceting and omitNorms=true

2010-10-12 Thread Paolo Castagna
Hi, I am not completely sure on what's the recommended setting for fields used for faceting regarding omitNorms and potitionIncrementGap. Should I used omitNorms="true"? What about positionIncrementGap? At the moment I have this in my schema.xml: stored="false" multiValued="true" />

Re: StatsComponent and multi-valued fields

2010-10-12 Thread Peter Karich
I'm not sure ... just reading it yesterday night ... but isn't the unapplied patch from Harish https://issues.apache.org/jira/secure/attachment/12400054/SOLR-680.patch what you want? Regards, Peter. > Running 1.4.1. > > I'm able to execute stats queries against multi-valued fields, but when > giv