unloading a solr core doesn't free any memory

2014-11-17 Thread Ofer Fort
Getting a lot of those today. Is it all from the same site we saw last week? OFER FORT Head of R&D 437 Fifth Avenue 9th floor, New York, NY 10016 cell: ISR +972-54-5678339 US +1 212 738 9594 ext 34 skype: oferfort tracx social intelligence www.tracx.com<http://www.tracx.com/> Follow

Re: core sleep/wake

2012-05-01 Thread Ofer Fort
My random searches can be a bit slow on startup, so i still would like to get that lazy load but have more cores available. I'm actually trying now the LotsOfCores way of handling things. Had to work a bit to get the patch suitable for 3.5 but it seems to be doing what i need. On Tue, May 1, 2012

public constructor for KStemmer

2011-09-22 Thread Ofer Fort
He all, I was very happy to see that Kstemmer implementation was added to lucene. I was wondering why is the constructor not public? I have a case where i want to create an analyzer that uses the stemmer itself, and in order to construct a new instance, it has to be in the same package and be loade

SOLR-1155 on 3.1

2011-05-30 Thread Ofer Fort
Hey all, In the last comment on SOLR-1155 by Jayson Minard ( https://issues.apache.org/jira/browse/SOLR-1155?focusedCommentId=13019955&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13019955 ) "I'll look at updating this for 3.1" was it integrated into 3.1? if not is

Re: Index upgrade from 1.4.1 to 3.1 and 4.0

2011-04-22 Thread Ofer Fort
gt; > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch > Lucene ecosystem search :: http://search-lucene.com/ > > > > - Original Message >> From: Ofer Fort >> To: "solr-user@lucene.apache.org" >> Sent: Fri, April 22, 2011 1:00:05 P

Re: Index upgrade from 1.4.1 to 3.1 and 4.0

2011-04-22 Thread Ofer Fort
> Otis > > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch > Lucene ecosystem search :: http://search-lucene.com/ > > > > - Original Message >> From: Ofer Fort >> To: "solr-user@lucene.apache.org" >> Sent: Fri, April

Re: Index upgrade from 1.4.1 to 3.1 and 4.0

2011-04-22 Thread Ofer Fort
Nobody? Am I the only one in need of upgrading an index that was created with 1.4.1? Thanks for any info Ofer On Friday, April 22, 2011, Ofer Fort wrote: > Hi all, > While doing some tests, I realized that an index that was created with > solr 1.4.1 is readable by solr 3.1, but nt re

Re: Highest frequency terms for a subset of documents

2011-04-21 Thread Ofer Fort
Ok, thanks On Friday, April 22, 2011, Yonik Seeley wrote: > On Thu, Apr 21, 2011 at 6:50 PM, Ofer Fort wrote: >> Ok, I'll give it a try, as this is a server I am willing to risk. >> How is the competability between solrj of bulkpostings, trunk, 3.1 and 1.4.1? > > bu

Re: Highest frequency terms for a subset of documents

2011-04-21 Thread Ofer Fort
Ok, I'll give it a try, as this is a server I am willing to risk. How is the competability between solrj of bulkpostings, trunk, 3.1 and 1.4.1? On Friday, April 22, 2011, Yonik Seeley wrote: > On Thu, Apr 21, 2011 at 6:34 PM, Ofer Fort wrote: >> So I'm guessing my best appro

Re: Highest frequency terms for a subset of documents

2011-04-21 Thread Ofer Fort
So I'm guessing my best approach now would be to test trunk, and hope that as 3.1 cut the performance in half, trunk will do the same Thanks for the info Ofer On Friday, April 22, 2011, Yonik Seeley wrote: > On Thu, Apr 21, 2011 at 6:25 PM, Ofer Fort wrote: >> Well, it was

Index upgrade from 1.4.1 to 3.1 and 4.0

2011-04-21 Thread Ofer Fort
Hi all, While doing some tests, I realized that an index that was created with solr 1.4.1 is readable by solr 3.1, but nt readable by solr 4.0. If I plan to migrate my index to 4.0, and I prefer not to reindex it all, what would be my best course of action? Will it be possible to continue to write

Re: Highest frequency terms for a subset of documents

2011-04-21 Thread Ofer Fort
Well, it was worth the try;-) But will using the facet.method=fc, will reducing the subset size reduce the time and memory? Meaning is it an O( ndocs of the set)? Thanks On Thursday, April 21, 2011, Yonik Seeley wrote: > On Thu, Apr 21, 2011 at 11:15 AM, Ofer Fort wrote: >> So if i wa

Re: Highest frequency terms for a subset of documents

2011-04-21 Thread Ofer Fort
So if i want to use the facet.method=fc, is there a way to speed it up? and remove the bucket size limitation? On Thu, Apr 21, 2011 at 5:58 PM, Yonik Seeley wrote: > On Thu, Apr 21, 2011 at 10:41 AM, Ofer Fort wrote: > > I see, thanks. > > So if I would want to implement somet

Re: Highest frequency terms for a subset of documents

2011-04-21 Thread Ofer Fort
AM, Ofer Fort wrote: > > Not sure i fully understand, > > If "facet.method=enum steps over all terms in the index for that field", > > than what does setting the q=field:subset do? if i set the q=*:*, than > how > > do i get the frequency only on my subset? >

Re: Highest frequency terms for a subset of documents

2011-04-21 Thread Ofer Fort
n Thu, Apr 21, 2011 at 9:24 AM, Ofer Fort wrote: > > Another strange behavior is that the Qtime seems pretty stable, no matter > > how many object match my query. 200K and 20K both take about 17s. > > I would have guessed that since the time is going over all the terms of > all &g

Re: Highest frequency terms for a subset of documents

2011-04-21 Thread Ofer Fort
more documents, the more time. Thanks for any insights ofer On Thu, Apr 21, 2011 at 3:07 AM, Ofer Fort wrote: > my documents are user entries, so i'm guessing they vary a lot. > Tomorrow i'll try 3.1 and also 4.0, and see if they have an improvement. > thanks guys! > > >

Re: Highest frequency terms for a subset of documents

2011-04-20 Thread Ofer Fort
my documents are user entries, so i'm guessing they vary a lot. Tomorrow i'll try 3.1 and also 4.0, and see if they have an improvement. thanks guys! On Thu, Apr 21, 2011 at 3:02 AM, Yonik Seeley wrote: > On Wed, Apr 20, 2011 at 7:45 PM, Ofer Fort wrote: > > Thanks > &

Re: Highest frequency terms for a subset of documents

2011-04-20 Thread Ofer Fort
BTW, i'm using solr 1.4.1, does 3.1 or 4.0 contain any performance improvements that will make a difference as far as facet search? thanks again Ofer On Thu, Apr 21, 2011 at 2:45 AM, Ofer Fort wrote: > Thanks > but i've disabled the cache already, since my concern is speed and

Re: Highest frequency terms for a subset of documents

2011-04-20 Thread Ofer Fort
Thanks but i've disabled the cache already, since my concern is speed and i'm willing to pay the price (memory), and my subset are not fixed. Does the facet search do any extra work that i don't need, that i might be able to disable (either by a flag or by a code change), Somehow i feel, or rather

Re: Highest frequency terms for a subset of documents

2011-04-20 Thread Ofer Fort
sure how to start. On Thu, Apr 21, 2011 at 2:23 AM, Ofer Fort wrote: > thanks, but that's what i started with, but it took an even longer time and > threw this: > Approaching too many values for UnInvertedField faceting on field 'text' : > bucket size=15560140 > A

Re: Highest frequency terms for a subset of documents

2011-04-20 Thread Ofer Fort
hen facet.method=fc became available, it was nearly > impossible to facet on very high arity fields, facet.method=fc is the magic. > I think facet.method=fc is even the default in Solr 1.4+, if you hadn't > explicitly set it to enum instead! > > Jonathan > _

Highest frequency terms for a subset of documents

2011-04-20 Thread Ofer Fort
Hi, I am looking for the best way to find the terms with the highest frequency for a given subset of documents. (terms in the text field) My first thought was to do a count facet search , where the query defines the subset of documents and the facet.field is the text field, this gives me the result

Re: KStemmer for Solr 3.x +

2011-04-20 Thread Ofer Fort
Seems like it isn't. In my installation (1.4.1) i used LucidKStemFilterFactory, and when switching the solr.war file to the 3.1 one i get: 14:42:31.664 ERROR [pool-1-thread-1]: java.lang.AbstractMethodError: org.apache.lucene.analysis.TokenStream.incrementToken()Z at org.apache.lucene.analy

Re: mixing version of solr

2011-03-03 Thread Ofer Fort
gt; aren't alone, but it makes me nervous, seems like asking for trouble. When > the indexing instances writes new indexes, when and how is the read-only > Solr going to figure that out and load new searchers for it? It just gets > confusing and complicated. > > > > &g

mixing version of solr

2011-03-03 Thread Ofer Fort
Hey all, I have a master slave using the same index folder, the master only writes, and the slave only reads. Is it possible to use different versions of solr for those two servers? Let's say i want to gain from the improved search speed of solr4.0 but since it's my production system, am not willin

Re: Efficient boolean query

2011-03-02 Thread Ofer Fort
That's great, just what I needed, I was debugging and was expecting to see something like this. i'll look through the SVN history to see in which version it was added. Thanks On Wednesday, March 2, 2011, Yonik Seeley wrote: > On Wed, Mar 2, 2011 at 2:43 PM, Ofer Fort wrote: &g

Re: Efficient boolean query

2011-03-02 Thread Ofer Fort
I didn't see this behavior, running solr 1.4.1, was that implemented after this release? On Wednesday, March 2, 2011, Yonik Seeley wrote: > On Wed, Mar 2, 2011 at 1:58 PM, Ofer Fort wrote: >> Thanks, >> But each query tries to see if there is something new since the las

Re: Efficient boolean query

2011-03-02 Thread Ofer Fort
I'm guessing what i was describing is a short-circuit evaluation and i see that lucene doesn't have it: http://lucene.472066.n3.nabble.com/Short-circuit-in-query-td738551.html Still would love to hear any suggestions for my type of query ofer On Wed, Mar 2, 2011 at 8:58 PM, Ofer F

Re: Efficient boolean query

2011-03-02 Thread Ofer Fort
ndpoint is *not* reused across many queries, then you > can still use the same strategy as above by adding another small "fq" > for the lower endpoint. > > -Yonik > http://lucidimagination.com > > > > On Wed, Mar 2, 2011 at 1:11 PM, Ofer Fort wrote: > > you a

Re: Efficient boolean query

2011-03-02 Thread Ofer Fort
timestamp is of type: On Wed, Mar 2, 2011 at 8:11 PM, Ofer Fort wrote: > you are correct that my query is a tange one, probably should have > mentioned it in the first post. > this is the debug data: > > > > > > 0 > 4173 > > on > on > &g

Re: Efficient boolean query

2011-03-02 Thread Ofer Fort
0.0 4171.0 4171.0 0.0 0.0 0.0 0.0 0.0 On Wed, Mar 2, 2011 at 7:48 PM, Yonik Seeley wrote: > On Wed, Mar 2, 2011 at 12:11 PM, Ofer Fort wrote: > > Hey all, > >

Re: Efficient boolean query

2011-03-02 Thread Ofer Fort
quickly. > So your FIRST query will probably still be in the 'few seconds'- range, but > all following queries involving X will return much quicker. > > hth, > Geert-Jan > > 2011/3/2 Ofer Fort > > > Hey all, > > I have an index with a lot of documents w

Efficient boolean query

2011-03-02 Thread Ofer Fort
Hey all, I have an index with a lot of documents with the term X and no documents with the term Y. If i query for X it take a few seconds and returns the results. If I query for Y it takes a millisecond and returns an empty set. If i query for Y AND X it takes a few seconds and returns an empty set

Re: Does Solr supports indexing & search for Hebrew.

2011-01-18 Thread Ofer Fort
take a look at : http://github.com/synhershko/HebMorph with more info at http://www.code972.com/blog/hebmorph/ On Tue, Jan 18, 2011 at 11:04 AM, prasad deshpande < prasad.deshpand...@gmail.com> wrote: > Hello, > > With reference to below links I haven't found Hebrew support in Solr. > > http://w

Re: Master/Slave High CPU Usage

2010-11-23 Thread Ofer Fort
ducing the resources needed, as you don't have 2 indexes and you don't need to copy the data from one to the other. Thanks Lance for that proposal. On Sun, Nov 21, 2010 at 2:12 PM, Ofer Fort wrote: > do i really need a commit? or can i use the > *readercycle*<http://w

Re: Master/Slave High CPU Usage

2010-11-21 Thread Ofer Fort
ys reloads the index. And it > always throws away the Solr caches: queryresult, document, filter > query. > > If you do this, please post your results. > > On Sat, Nov 20, 2010 at 11:16 PM, Ofer Fort wrote: > > OK, > > so to make sure i understand, even though the &quo

Re: Master/Slave High CPU Usage

2010-11-21 Thread Ofer Fort
se post your results. > > On Sat, Nov 20, 2010 at 11:16 PM, Ofer Fort wrote: > > OK, > > so to make sure i understand, even though the "slave" doesn't do any > > indexing, i will call commit and it will do nothing to the index itself, > but > > will

Re: Master/Slave High CPU Usage

2010-11-20 Thread Ofer Fort
sure to make all of the wait options false to this command; you > don't want the master to block while the slave loads up the new index. > Or, to control the maximum load on your server, you might actually > want to make the master wait while the slave loads up > > Lance > >

Re: Master/Slave High CPU Usage

2010-11-20 Thread Ofer Fort
using the same index files. Ofer On Sat, Nov 20, 2010 at 8:52 PM, Erick Erickson wrote: > The slave polls. See: http://wiki.apache.org/solr/SolrReplication > > Best > Erick > > On Sat, Nov 20, 2010 at 1:13 PM, Ofer Fort wrote: > > > Another question on that conf

Re: Master/Slave High CPU Usage

2010-11-20 Thread Ofer Fort
has read-only access to the index; that way it cannot touch the index. On Wed, Nov 17, 2010 at 11:28 PM, Ofer Fort wrote: anybody? On Wed, Nov 17, 2010 at 12:09 PM, Ofer Fort wrote: Hi, I'm working with Erez, we experienced this again, and this time the slave index folder didn'

Re: Master/Slave High CPU Usage

2010-11-19 Thread Ofer Fort
; that way it cannot touch the index. On Wed, Nov 17, 2010 at 11:28 PM, Ofer Fort wrote: anybody? On Wed, Nov 17, 2010 at 12:09 PM, Ofer Fort wrote: Hi, I'm working with Erez, we experienced this again, and this time the slave index folder didn't contain the index.XXX folder, only

Re: Master/Slave High CPU Usage

2010-11-17 Thread Ofer Fort
anybody? On Wed, Nov 17, 2010 at 12:09 PM, Ofer Fort wrote: > > Hi, I'm working with Erez, > we experienced this again, and this time the slave index folder didn't > contain the index.XXX folder, only one index folder. > if we shutdown the slave, the CPU on the master

Re: Master/Slave High CPU Usage

2010-11-17 Thread Ofer Fort
Hi, I'm working with Erez, we experienced this again, and this time the slave index folder didn't contain the index.XXX folder, only one index folder. if we shutdown the slave, the CPU on the master was normal, as soon as we started the slave again, the CPU went up to 100% again. thanks for any hel