Optional filter queries

2012-01-03 Thread Allistair Crossley
Evening all, A subset of my documents have a field, filterMinutes, that some other documents do not. filterMinutes stores a number. I often issue a query that contains a filter query range, e.g. q=filterMinutes:[* TO 50] I am finding that adding this query excludes all documents that do not fe

Re: Same index is ranking differently on 2 machines

2011-03-09 Thread Allistair Crossley
ight(text:dubai in 1551), product of: >> + 0.05489459 = queryWeight(text:dubai), product of: >> >> Regards, >> Jayendra >> >> On Wed, Mar 9, 2011 at 4:38 PM, Allistair Crossley wrote: >>> Thanks. Good to know, but even so my problem remains - the

Re: Same index is ranking differently on 2 machines

2011-03-09 Thread Allistair Crossley
ts are multiplied by the >> same factor), but rather just attempts to make scores from different >> queries (or even different indexes) comparable >> >> Regards, >> Jayendra >> >> On Wed, Mar 9, 2011 at 4:22 PM, Allistair Crossley wrote: >>> Hi, >>

Re: Same index is ranking differently on 2 machines

2011-03-09 Thread Allistair Crossley
y the > same factor), but rather just attempts to make scores from different > queries (or even different indexes) comparable > > Regards, > Jayendra > > On Wed, Mar 9, 2011 at 4:22 PM, Allistair Crossley wrote: >> Hi, >> >> I am seeing an issue I do not unde

Same index is ranking differently on 2 machines

2011-03-09 Thread Allistair Crossley
Hi, I am seeing an issue I do not understand and hope that someone can shed some light on this. The issue is that for a particular search we are seeing a particular result rank in position 3 on one machine and position 8 on the production machine. The position 3 is our desired and roughly expec

Re: [Adding] Entities when indexing a DB

2010-12-15 Thread Allistair Crossley
mission.id and event.id if the same value will be overwriting the indexed document. your ids need to be unique across all documents. i usually have a field id_original that i map the table id to, and then for id per entity i usually prefix it with the entity name in the value mapped to the schem

Re: Synchronizing Solr with a PostgreDB

2010-10-14 Thread Allistair Crossley
i would not cross-reference solr results with your database to merge unless you want to spank your database. nor would i load solr with all your data. what i have found is that the search results page is generally a small subset of data relating to the fuller document/result. therefore i store o

Re: which schema.xml to modify ?

2010-10-14 Thread Allistair Crossley
you will find it in the distribution at example/solr/config On Oct 14, 2010, at 3:04 PM, Ibrahim Diop wrote: > Hi All, > > I'm a new solr user and I just want to know which schema.xml file to modify > for this tutorial : http://lucene.apache.org/solr/tutorial.html > > Thanks, > > Ibrahim.

Re: What is the maximum number of documents that can be indexed ?

2010-10-14 Thread Allistair Crossley
me also. great book, just wanted a bit more on complex DIH :) On Oct 14, 2010, at 10:38 AM, Jason Brown wrote: > Not related to the opening thread - but wante to thank Eric for his book. > Clarified a lot of stuff and very useful. > > > -Original Message- > From: Eric Pugh [mailto:ep..

Re: search over two independent tables

2010-10-14 Thread Allistair Crossley
super On Oct 14, 2010, at 8:00 AM, Anthony Maudry wrote: > Sorry for the late answer. > > It works now thanks to you, Allistair. > > I needed to use your "uid" field, common to the two entities but built in > different ways. > > here is the result in a sample of the data-config.xml file > >

Re: What is the maximum number of documents that can be indexed ?

2010-10-14 Thread Allistair Crossley
i think you answered the question by yourself ... these questions usually get the response that there is no answer. solr/lucence scale and distribute to whatever hardware you want to throw them. you probably want to turn the question around - what is the maximum number of documents that your sy

Re: check if field CONTAINS a value, as opposed to IS of a value

2010-10-14 Thread Allistair Crossley
actuall no you don't .. if you want hi in a sentence of hi there this is me this is just normal tokenizing and should work .. check your field type/analysers On Oct 14, 2010, at 7:59 AM, Allistair Crossley wrote: > i think you need to look at ngram tokenizing > > On Oct 14, 20

Re: check if field CONTAINS a value, as opposed to IS of a value

2010-10-14 Thread Allistair Crossley
i think you need to look at ngram tokenizing On Oct 14, 2010, at 7:55 AM, PeterKerk wrote: > > I try to determine if a certain word occurs within a field. > > http://localhost:8983/solr/db/select/?indent=on&facet=true&fl=id,title&q=introtext:hi > > this works if an EXACT match was found on fie

Re: search over two independent tables

2010-10-14 Thread Allistair Crossley
e but I only get result from one table (actualy "Newsfeeds") > > > Le 14/10/2010 11:49, Allistair Crossley a écrit : >> actually your intention is unclear ... are you wanting to run a single >> search and get back results from BOTH newsfeed and message? or do you

Re: search over two independent tables

2010-10-14 Thread Allistair Crossley
27;ll get back counts for your Newsfeed Message results too. On Oct 14, 2010, at 5:44 AM, Allistair Crossley wrote: > your first example is correct > > > > > i have the same config for indexing 5 different tables > > what you don't have from what i

Re: search over two independent tables

2010-10-14 Thread Allistair Crossley
your first example is correct i have the same config for indexing 5 different tables what you don't have from what i can see is a field name mapped to each column, e.g. i always have to provide the destination field in schema.xml, e.g. On Oct 14, 2010, at 5:22 AM, Anthony Maudry wrote:

Yet another date format problem

2010-10-12 Thread Allistair Crossley
Afternoon, I wonder if anyone could shed any light on this one that's stumping me. I have a table that is storing dates using seconds since epoch, so something like this: 1182262980 When I first wrote my queries for DIH it was complaining about this and rightly so as I had a mapper thus So

Re: Strategy for re-indexing

2010-10-08 Thread Allistair Crossley
0/6/2010 10:49 AM, Allistair Crossley wrote: >> Hi, >> >> I was interested in gaining some insight into how you guys schedule updates >> for your Solr index (I have a single index). >> >> Right now during development I have added deltaQuery specifications to

Re: Getting an ngram fieldtype to work

2010-10-08 Thread Allistair Crossley
n the Prescott result. So ngramming is working but it is not working when the query is something far to the right of the indexed value. Is this another user-error or have I missed something else here? Cheers On Oct 8, 2010, at 9:02 AM, Allistair Crossley wrote: > Oh my. I am basically

Re: Getting an ngram fieldtype to work

2010-10-08 Thread Allistair Crossley
53 AM, Markus Jelsma wrote: > How come your query analyser spits out grams? It isn't configured to do so or > you posted an older field definition. Anyway, do you actually search on your > new field? > > On Friday, October 08, 2010 02:46:08 pm Allistair Crossley wrote: >

Re: Getting an ngram fieldtype to work

2010-10-08 Thread Allistair Crossley
s to go to the analysis page, enter your test > data, and report back what each analysis stage prints out: > http://localhost:8983/solr/admin/analysis.jsp > > -- > Jan Høydahl, search solution architect > Cominvent AS - www.cominvent.com > > On 8. okt. 2010, at 14.19, Allis

Getting an ngram fieldtype to work

2010-10-08 Thread Allistair Crossley
Morning all, I would like to ngram a company name field in our index. I have read about the costs of doing so in the great David Smiley Solr 1.4 book and just to get started I have followed his example in setting up an ngram field type as follows:

Invalid boolean value for query with exclamation

2010-10-07 Thread Allistair Crossley
Hi, Quick one ... we have some documents that have punctuation in their indexed title, e.g. !Fast Note the leading exclamation. Our tests show we cannot query for !Fast and find the document. I've just taken a look and Solr is coming back with Invalid boolean value: !Fast It's a valid quer

Strategy for re-indexing

2010-10-06 Thread Allistair Crossley
Hi, I was interested in gaining some insight into how you guys schedule updates for your Solr index (I have a single index). Right now during development I have added deltaQuery specifications to data import entities to control the number of rows being queries on re-indexes. However in terms o

Re: solrj

2010-10-04 Thread Allistair Crossley
i rewrote the top jar section at http://wiki.apache.org/solr/Solrj and the following code then runs fine. import java.net.MalformedURLException; import org.apache.solr.client.solrj.SolrQuery; import org.apache.solr.client.solrj.SolrServer; import org.apache.solr.client.solrj.SolrServerException;

Re: solr-user

2010-10-04 Thread Allistair Crossley
I updated the SolrJ JAR requirements to be clearer on the wiki page given how many of these SolrJ emails I saw coming through since joining the list. I just created a test java class and imported the removed JARs until I found out the minimal set required. On Oct 4, 2010, at 8:27 AM, Erick Eric

DIH sub-entity not indexing

2010-10-04 Thread Allistair Crossley
Hello list, I've been successful with DIH to a large extent but a seemingly simple extra column I need is posing problems. In a nutshell I have 2 entities let's say - Listing habtm Contact. I have copied the relevant parts of the configs below. I have run my SQL for the sub-entity Contact and t

Re: DIH sub-entity not indexing

2010-10-04 Thread Allistair Crossley
t; http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201008.mbox/%3 > c9f8b39cb3b7c6d4594293ea29ccf438b01702...@icq-mail.icq.il.office.aol.com > %3E > > Ephraim Ofir > > -Original Message- > From: Allistair Crossley [mailto:a...@roxxor.co.uk] > Sent: Monday,

Re: DIH sub-entity not indexing

2010-10-04 Thread Allistair Crossley
iki.apache.org/solr/DataImportHandler#Interactive_Development_Mo > de > > Ephraim Ofir > > > -Original Message- > From: Allistair Crossley [mailto:a...@roxxor.co.uk] > Sent: Monday, October 04, 2010 3:09 PM > To: solr-user@lucene.apache.org > Subject: Re: DIH sub

Re: DIH sub-entity not indexing

2010-10-04 Thread Allistair Crossley
missing the obvious? On Oct 4, 2010, at 8:22 AM, Allistair Crossley wrote: > Hello list, > > I've been successful with DIH to a large extent but a seemingly simple extra > column I need is posing problems. In a nutshell I have 2 entities let's say - > Listing habtm C

Re: DIH sub-entity not indexing

2010-10-04 Thread Allistair Crossley
Very clever thinking indeed. Well, that's certainly revealed the problem ... ${listing.id} is empty on my sub-entity query ... And this because I prefix the indexed ID with a letter This appears to modify the internal value of $listing.id for subsequent uses. Well, I can work around this now

Re: multi level faceting

2010-10-04 Thread Allistair Crossley
I think that is just sending 2 fq facet queries through. In Solr PHP I would do that with, e.g. $params['facet'] = true; $params['facet.fields'] = array('Size'); $params['fq'] => array('sex' => array('Men', 'Women')); but yes i think you'd have to send through what the current facet query is and

Re: any working SolrJ code example for Solr 1.4.1

2010-10-01 Thread Allistair Crossley
at ran into SolrJ issue. If that's the case, I will > grab Perl solution, and come back to SolrJ later. > > Thanks, > Xin > > -Original Message- > From: Allistair Crossley [mailto:a...@roxxor.co.uk] > Sent: Friday, October 01, 2010 11:52 AM > To: solr-user@l

Re: any working SolrJ code example for Solr 1.4.1

2010-10-01 Thread Allistair Crossley
no example anyone gives you will solve your class not found exception .. you need to ensure the relevant jars (in dist) are included in your solr instance's lib folder i guess? On Oct 1, 2010, at 10:50 AM, Xin Li wrote: > Hi, there, > > Just picked up SolrJ few days ago. I have my Solr Server

Re: Any way to "append" new text to an existing indexed field?

2010-10-01 Thread Allistair Crossley
g". If > each answer is indexed separately, I'd get two "What is the meaning of life?" > in my search results when someone searches for "solr". > > --- On Fri, 10/1/10, Allistair Crossley wrote: > >> From: Allistair Crossley >> Subject:

Re: Any way to "append" new text to an existing indexed field?

2010-10-01 Thread Allistair Crossley
i would say question and answer are 2 different entities. if you are using the data import handler, i would personally create them as separate entities with their own queries to the database using the deltaQuery method to pick up only new rows. i guess it depends if you need question + answers t

Re: SolrJ

2010-09-30 Thread Allistair Crossley
it's in the dist folder with the name provided by the wiki page you refer to On Sep 30, 2010, at 3:01 PM, Christopher Gross wrote: > Where can I get SolrJ? The wiki makes reference to it, and says that it is > a part of the Solr builds that you download, but I can't find it in the jars > that co

Re: Solr rate limiting / DoS attacks

2010-09-29 Thread Allistair Crossley
This kind of thing is not limited to Solr and you normally wouldn't solve it in software - it's more a network concern. I'd be looking at a web server solution such as Apache mod_evasive combined with a good firewall for more conventional DOS attacks. Just hide your Solr install behind the firew

Re: Missing facet values for zero counts

2010-09-29 Thread Allistair Crossley
Hi, For us this is a usability concern. You either don't show Sweden in a pick-list called Country and some users go away thinking you don't *ever* support Sweden (not true). OR you allow a user to execute an empty result search - but at least they know you do support Sweden. It is we believe

Re: Missing facet values for zero counts

2010-09-29 Thread Allistair Crossley
t;> possible, the facet.query parameter allows you to rely on other conditions >> to generate a facet count. But if the missing countries are not in the >> index, you cannot query against them. >> >> -Original message- >> From: Allistair Crossley >> Sent:

Missing facet values for zero counts

2010-09-29 Thread Allistair Crossley
Hello list, I am implementing a directory using Solr. The user is able to search with a free-text query or 2 filters (provided as pick-lists) for country. A directory entry only has one country. I am using Solr facets for country and I use the facet counts generated initially by a *:* search t