Re: Solr search by indexes on the basis of different search fields

2008-09-16 Thread prerna07
Can you explain more on this or forward some example of this scenario which can help us. Otis Gospodnetic wrote: > > Hi, > > You can search using different fields and you can make such clauses > required or prohibited or "should occur". > > You seem to be describing something more hierarchic

Re: StopFilterFactory bug?

2008-09-16 Thread Chris Hostetter
: anything older than 1.3, could you please try to reproduce this with : 1.3? I seem to recall something similar being mentioned in the past It was SOLR-261, caused by LUCENE-933 -Hoss

Re: [ANNOUNCE] Solr 1.3.0 Released

2008-09-16 Thread Jon Baer
Another +1 for Shalin and Noble for DIH ... On Sep 16, 2008, at 9:50 PM, Erik Hatcher wrote: +1 for Grant's efforts! He put a lot of sweat into making this release a reality. Erik On Sep 16, 2008, at 9:29 PM, Grant Ingersoll wrote: The Apache Solr team is happy to announce the av

Re: StopFilterFactory bug?

2008-09-16 Thread Sachin
We use solr 1.2 on our production environment so it seems like this was fixed in 1.3; let me try getting 1.3 and see if that fixes it: the only issue is convincing our IT team to move to 1.3 for production might not be that easy :(. -Sachin -- "Yonik Seeley" <[EMAIL PROTECTED]> wrote: I can't r

Re: [ANNOUNCE] Solr 1.3.0 Released

2008-09-16 Thread Erik Hatcher
+1 for Grant's efforts! He put a lot of sweat into making this release a reality. Erik On Sep 16, 2008, at 9:29 PM, Grant Ingersoll wrote: The Apache Solr team is happy to announce the availability of Solr 1.3.0 for public download. This version contains many enhancements and b

[ANNOUNCE] Solr 1.3.0 Released

2008-09-16 Thread Grant Ingersoll
The Apache Solr team is happy to announce the availability of Solr 1.3.0 for public download. This version contains many enhancements and bug fixes, including: - Distributed search capabilities - Numerous Lucene and other performance improvements - Support for multiple indexes in a single dep

Re: How to copy a solr index to another index with a different schema collapsing stored data?

2008-09-16 Thread ristretto . rb
is it possible to query out the stored data as, uh, tokens I suppose. Then, index those tokens in the next index? thanks gene On Wed, Sep 17, 2008 at 1:14 PM, Gene Campbell <[EMAIL PROTECTED]> wrote: > I was pretty sure you'd say that. But, I means lots that you take the > time to confirm it.

Re: Filtering results

2008-09-16 Thread Gene Campbell
OK thanks Otis. Any gut feeling on the best approach to get this collapsed data? I hate to ask you to do my homework, but I'm coming to the end of my Solr/Lucene knowledge. I don't code java too well - used to, but switched to Python a while back. gene On Wed, Sep 17, 2008 at 12:47 PM, Otis

Re: Adding bias to Distributed search feature?

2008-09-16 Thread Yonik Seeley
On Mon, Sep 15, 2008 at 5:43 PM, Lance Norskog <[EMAIL PROTECTED]> wrote: > As to why we have the same document in different shards with different > contents: once you hit a certain index size and ingest rate, it is easiest > to create a series of indexes and leave the older ones alone. In the futu

Re: How to copy a solr index to another index with a different schema collapsing stored data?

2008-09-16 Thread Gene Campbell
I was pretty sure you'd say that. But, I means lots that you take the time to confirm it. Thanks Otis. I don't want to give details, but we crawl for our data, and we don't save it in a DB or on disk. It goes from download to index. Was a good idea at the time; when we thought our designs were

Re: StopFilterFactory bug?

2008-09-16 Thread Yonik Seeley
I can't reproduce this with Solr 1.3 http://localhost:8983/solr/select?q=%2b(text:solr)+%2b(text:of)+%2b(text:works)&debugQuery=true&rows=0 And the relevant fragment of the debugQuery is: +(text:solr) +(text:of) +(text:works) +(text:solr) +(text:of) +(text:works) +text:solr +text:work +text:solr

Re: How to copy a solr index to another index with a different schema collapsing stored data?

2008-09-16 Thread Otis Gospodnetic
You can't copy+merge+flatten indices like that. Reindexing would be the easiest. Indexing taking weeks sounds suspicious. How much data are you reindexing and how big are your indices? Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: ris

Re: StopFilterFactory bug?

2008-09-16 Thread Otis Gospodnetic
I can't double check this, but assuming this is indeed a problem, please tell us which version of Solr you are using, and if you are using anything older than 1.3, could you please try to reproduce this with 1.3? I seem to recall something similar being mentioned in the past Otis -- Semate

Re: Filtering results

2008-09-16 Thread Otis Gospodnetic
Gene, The latest patch from Bojan for SOLR-236 works with whatever revision of Solr he used when he made the patch. I didn't follow this thread to know your original requirements, but running 1+10 queries doesn't sound good to me from scalability/performance point of view. Otis -- Sematext --

Re: cron job update index

2008-09-16 Thread Otis Gospodnetic
Hi Sunny, There is a very detailed page about this on the Wiki. Have you seen it? Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: sunnyfr <[EMAIL PROTECTED]> > To: solr-user@lucene.apache.org > Sent: Tuesday, September 16, 2008 5:59:01 PM

How to copy a solr index to another index with a different schema collapsing stored data?

2008-09-16 Thread ristretto . rb
Is it possible to copy stored index data from index to another, but concatenating it as you go. Suppose 2 categories A and B both with 20 docs, for a total of 40 docs in the index. The index has a stored field for the content from the docs. I want a new index with only two docs in it, one for A

StopFilterFactory bug?

2008-09-16 Thread Sachin
Hi folks, We use the StopFilterFactory with the stopwords.txt provided as part of the solr getting started example and just realized that if the stop word is enclosed within a parentheses then solr just strips out the stop word and not the parens and this causes 0 results being returned. for ex

Re: Filtering results

2008-09-16 Thread ristretto . rb
thanks. very interesting. The plot thickens. And, yes, I think field collapsing is exactly what I'm after. I'm am considering now trying this patch. I have a solr 1.2 instance on Jetty. I looks like I need to install the patch. Does anyone use that patch? Recommend it? The wiki page (http:/

cron job update index

2008-09-16 Thread sunnyfr
Hi Can you give me more explanation about how to manage index update automatically I would like to automatically update solr, obviously I have to run snapuller ?? And I would like to past data from master to slave's servers ? Can you explain me a bit more about it ? Thanks Sunny -- View this

Re: Filtering results

2008-09-16 Thread Chris Hostetter
: 1. Identify all records that would match search terms. (Suppose I : search for 'dog', and get 450,000 matches) : 2. Of those records, find the distinct list of groups over all the : matches. (Suppose there are 300.) : 3. Now get the top ranked record from each group, as if you search : just

Questions about Indexes and Hardware Config

2008-09-16 Thread Andrey Shulinskiy
Hi, folks, We're planning to use SOLR for our project, got some questions. It's a very new experience for us so any help is really appreciated. 1) We're storing two types of documents; both have pretty much the same fields (with a few extra fields for one type). The important thing is that

Re: Filtering results

2008-09-16 Thread Gene Campbell
Thanks for the reply Erik Sorry for being vague. To be clear we have 1-2 million records, and rough 12000-14000 groups. Each record is in one and only one group. I see it working something like this 1. Identify all records that would match search terms. (Suppose I search for 'dog', and get 45

Re: Filtering results

2008-09-16 Thread Erik Hatcher
Personally, I'd send three requests for solr, one for each group. &rows=1&fq=category:A ... and so on. But that'd depend on how many groups you have. One can always hack custom request handlers to do this sort of thing all as a single request, but I'd guess it ain't that much slower to ju

Re: AW: Searching for future or "null" dates

2008-09-16 Thread Chris Maxwell
Yes your right I mistyped when I said [* : *]. I did mean [* TO *] hossman wrote: > > > : present and have a end date in the present/future or none at all. I had > : looked into searching for null dates and had come across the -date[* : > *] > : syntax, which searches for the absence of the in

Re: Solr stops listening

2008-09-16 Thread Fuad Efendi
In this case someone outside SOLR is improperly dealing with java.lang.VirtualMachineError in a thread outside of 'servlet'-pool. It could be: JVM, Tomcat (dispatcher thread etc.), Lucene, STAX, Jakarta Commons. (of course not-stable hardware could also be a problem). In some cases 'servlet

Filtering results

2008-09-16 Thread ristretto . rb
Hello All, I'm looking for a way to filter results by some ranking mechanism. For example... Suppose you have 30 docs in an index, and they are in groups of 10, like this A, 1 A, 2 : A, 10 B, 1 B, 2 : B, 10 C, 1 C, 2 : C, 10 I would like to get 3 records back such that I get a single, "best"

Re: Solr stops listening

2008-09-16 Thread Yonik Seeley
On Tue, Sep 16, 2008 at 2:43 PM, Fuad Efendi <[EMAIL PROTECTED]> wrote: > SOLR catches everything and tries to > output (sometimes with great success) HTTP 500 with XML-formatted stacktrace > as content. Right, as well as logging the exception. But, it's never foolproof because the OOM exception c

Re: Solr stops listening

2008-09-16 Thread Fuad Efendi
I seen it so many times; after moving to JRockit JVM I had clean error logs; after increasing memory such problems disappeared. In most cases OOME happens when someone executes 'sorted' query. JVM behaviour in case of catched OOME is very unpredictable... SOLR catches everything and tries t

Re: More details on my MoreLikeThis mlt.qf boosting problem

2008-09-16 Thread Clas Rydergren
Hoss, thanks for the response and confirmation. Yes, from reading the comments in the Lucene source of MoreLikeThis.java I have now realized that the field used in the TermQuery is "the top field that this word comes from". The SimilarityQuery is restricted to this field only for the specific word

Re: recip(myfield,m,a,b)

2008-09-16 Thread Chris Hostetter
: Is there a way to convert to integer to check if a = b ... like : recip(myfield,m,language,lang) : But I would like to boost(scoring) field which have the same user language : and book language ... : : But for that I need to know convert.int(language) There is an OrdFieldSource that can be use

Re: Solr 1.3 and Lucene 2.4 dev

2008-09-16 Thread Lars Kotthoff
Some highlighting stuff, most notably maxAnalyzedChars=-1 (SOLR-610) requires Lucene 2.4 to work correctly. Lars

Re: More details on my MoreLikeThis mlt.qf boosting problem

2008-09-16 Thread Chris Hostetter
: Document 1 is probably a better match since the word yahoo is present : two times. That seems fine, although I did not expect to see the : "content:" part in the list of interestingTerms. ... : but the response is exactly the same as for the query without the mlt.qf. : : The problem see

Re: Date field mystery

2008-09-16 Thread Chris Hostetter
: We have big problem searching out solr index and filtering for the date. : Let me give you an example: there is a record with date 30.04.2008, : 15:32:00. My query contains "+date:[20080101T12:00:00Z TO : 20080915T13:59:00Z]" but the record is not found. But when I search : "+date:[20071231T

Re: DIH + RSS (@attributes)

2008-09-16 Thread Jon Baer
That was it, thanks Shalin. On Sep 16, 2008, at 1:41 PM, Shalin Shekhar Mangar wrote: On Tue, Sep 16, 2008 at 10:41 PM, Jon Baer <[EMAIL PROTECTED]> wrote: For some reason my XPath attribute keeps failing to get picked up here (is that the proper format?): Put a slash between node and

Re: AW: Searching for future or "null" dates

2008-09-16 Thread Chris Hostetter
: present and have a end date in the present/future or none at all. I had : looked into searching for null dates and had come across the -date[* : *] : syntax, which searches for the absence of the index value for the field : specified. to clarify: that isn't valid syntax. i think you are confus

RE: Searching for future or "null" dates

2008-09-16 Thread Chris Hostetter
: If the query stars with a negative clause Lucene returns nothing. that's not true. If a "Query" in lucene is a BooleanQuery that only contains negative clauses, then Lucene returns nothing (because nothing is positively selected) ... but it if there is a mix of negative lcauses and positive

Re: DIH + RSS (@attributes)

2008-09-16 Thread Shalin Shekhar Mangar
On Tue, Sep 16, 2008 at 10:41 PM, Jon Baer <[EMAIL PROTECTED]> wrote: > For some reason my XPath attribute keeps failing to get picked up here (is > that the proper format?): > > > > Put a slash between node and attribute. -- Regards, Shalin Shekhar Mangar.

DIH + RSS (@attributes)

2008-09-16 Thread Jon Baer
Hi, For some reason my XPath attribute keeps failing to get picked up here (is that the proper format?): - Jon

RE: Searching for future or "null" dates

2008-09-16 Thread Lance Norskog
If the query stars with a negative clause Lucene returns nothing. endDate[NOW TO *] OR -endDate:[* TO *] Might work -Original Message- From: Kolodziej Christian [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 16, 2008 12:01 AM To: solr-user@lucene.apache.org Subject: AW: Searching f

Re: Some new SOLR features

2008-09-16 Thread Henrib
ryantxu wrote: > > > Yes, include would get us some of the way there, but not far enough > (IMHO). The problem is that (as written) you still need to have all > the configs spattered about various directories. > > I does not allow us to go *all* the way but it does allow to put configu

Re: Some new SOLR features

2008-09-16 Thread Ryan McKinley
ryantxu wrote: ... Yes, I would like to see a way to specify all the fieldtypes / handlers in one location and then only specify what fields are available for each core. So yes -- I agree. In 2.0, I hope to flush out configs so they are not monstrous. ... What about using "include" so each

Re: Some new SOLR features

2008-09-16 Thread Henrib
ryantxu wrote: > > ... > Yes, I would like to see a way to specify all the fieldtypes / > handlers in one location and then only specify what fields are > available for each core. > > So yes -- I agree. In 2.0, I hope to flush out configs so they are > not monstrous. > ... > What

Re: AW: Searching for future or "null" dates

2008-09-16 Thread Chris Maxwell
Christian Kolodziej wrote: > > Hi Chris, > >>I'm having a lot of trouble getting this query syntax to work correctly. >>How >>can I search for a date, which is either in the future OR missing >>completely >>(meaning open ended) >> >>I've tried -endDate:[* TO *] OR endDate[NOW TO *] but that do

Re: Solr 1.3 and Lucene 2.4 dev

2008-09-16 Thread Shalin Shekhar Mangar
SpellCheckComponent requires 2.4, apart from that I'm not sure. I guess one can just try and see. On Tue, Sep 16, 2008 at 10:31 AM, Lance Norskog <[EMAIL PROTECTED]> wrote: > Is it possible to run Solr 1.3 with Lucene 2.3.2, the last official release > of Lucene? We're running into a problem wit

Re: Solr stops listening

2008-09-16 Thread Otis Gospodnetic
I've seen plenty of OOMs logged :) But you can run things like jconsole or jmap (with -heap or -histo or -histo:live options) to see how the JVM is doing with memory. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: Peter Williams <[EMAIL PR

Re: Help with Dismax query Handler

2008-09-16 Thread Otis Gospodnetic
But you can't really have a good multi-lingual search with a single index and without having content in different languages in different fields and analyzed differently. So filtering by language field will "work", but won't work very well as far as the "search experience" is concerned. Otis -

Re: Solr stops listening

2008-09-16 Thread Peter Williams
Thanks. Does a message about what went wrong not get written to the log because by the OOME happens there is not enough memory to actually construct the message? Peter On Mon, Sep 15, 2008 at 2:40 PM, Fuad Efendi <[EMAIL PROTECTED]> wrote: > > SOLR main servlet catches all Throwable. In case of

Re: Solr 1.3 and Lucene 2.4 dev

2008-09-16 Thread Otis Gospodnetic
I'm not 100% sure, but can't recall anything that would absolutely require 2.4. However, a "memory leak" fix was committed to Lucene trunk only a few days ago - if memory is your problem. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: La

Re: Solr search by indexes on the basis of different search fields

2008-09-16 Thread Otis Gospodnetic
Hi, You can search using different fields and you can make such clauses required or prohibited or "should occur". You seem to be describing something more hierarchical, but perhaps you can just make all clauses required. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch

Re: Some new SOLR features

2008-09-16 Thread Ryan McKinley
On Sep 16, 2008, at 10:12 AM, Jason Rutherglen wrote: Hello Ryan, SQL database such as H2 Mainly to offer joins and be able to perform hierarchical queries. Also any other types of queries a hybrid SQL search system would offer. This is something that is best built into SOLR rather than Lu

Re: Some new SOLR features

2008-09-16 Thread Jason Rutherglen
Hello Ryan, > SQL database such as H2 Mainly to offer joins and be able to perform hierarchical queries. Also any other types of queries a hybrid SQL search system would offer. This is something that is best built into SOLR rather than Lucene. It seems like a lot of the users of SOLR work with

Solr search by indexes on the basis of different search fields

2008-09-16 Thread prerna07
We are doing serach in solr on indexes, is it possible to search on the basis of different fields. For example we have a index created with fields author, title, ISBN and phase. we want the search result first on the basis of author then title match and then ISBN. -- View this message in cont

Re: Turkish stemming ??? stemming?

2008-09-16 Thread sunnyfr
thanks a lot, Grant Ingersoll-6 wrote: > > trunk version is just the latest development version, and can be > obtained via Subversion: svn checkout > http://svn.apache.org/repos/asf/lucene/solr/trunk > > See also http://wiki.apache.org/solr/HowToContribute > > Or, you could just wait for So

AW: Searching for future or "null" dates

2008-09-16 Thread Kolodziej Christian
Hi Chris, >I'm having a lot of trouble getting this query syntax to work correctly. >How >can I search for a date, which is either in the future OR missing >completely >(meaning open ended) > >I've tried -endDate:[* TO *] OR endDate[NOW TO *] but that doesn't work. >Adding parentheses doesn't help