firstSearcher and newSearcher events

2008-09-18 Thread oleg_gnatovskiy
Hello. I am using the spellcheck component (https://issues.apache.org/jira/browse/SOLR-572). Since the spell checker index is kept in RAM, it gets erased every time the Solr server gets restarted. I was thinking of using either the firstSearcher or the newSearcher to reload the index every time So

Re: firstSearcher and newSearcher events

2008-09-19 Thread oleg_gnatovskiy
Shalin Shekhar Mangar wrote: > > On Fri, Sep 19, 2008 at 5:55 AM, oleg_gnatovskiy < > [EMAIL PROTECTED]> wrote: > >> >> Hello. I am using the spellcheck component >> (https://issues.apache.org/jira/browse/SOLR-572). Since the spell checker >> index

Re: firstSearcher and newSearcher events

2008-09-19 Thread oleg_gnatovskiy
On Fri, Sep 19, 2008 at 10:07 PM, oleg_gnatovskiy < > [EMAIL PROTECTED]> wrote: > >> >> Is there any way to do it for an external (file-based) dictionary? >> > > SpellCheckComponent always reload on the dictionary in the firstSearcher > event. This works if y

File based index doesn't work in spellcheck component

2008-09-19 Thread oleg_gnatovskiy
Hello, I tried to have the spellcheck component to write to a drive index. My config is a s follows: org.apache.solr.spelling.FileBasedSpellChecker external spellings.txt UTF-8 true ./spellIndex org.apache.lucene.search.spell.JaroWinklerDistance

Re: File based index doesn't work in spellcheck component

2008-09-19 Thread oleg_gnatovskiy
oleg_gnatovskiy wrote: > > Hello, > > I tried to have the spellcheck component to write to a drive index. My > config is a s follows: > > >name="classname">org.apache.solr.spelling.FileBasedSpellChecker > external >

Different XML format for multi-valued fields?

2008-10-16 Thread oleg_gnatovskiy
Hello. I have an index built in Solr with several multi-value fields. When the multi-value field has only one value for a document, the XML returned looks like this: 5693 However, when there are multiple values for the field, the XMl looks like this: arr name="someIds"> 11199 1722 Is there a

Query Performance while updating teh index

2008-11-11 Thread oleg_gnatovskiy
Hello. We have an index with 15 million documents working on a distributed environment, with an index distribution setup. While an index on a slave server is being updated, query response times become extremely slow (upwards of 5 seconds). Is there any way to decrease the hit query response times

Re: Query Performance while updating teh index

2008-11-12 Thread oleg_gnatovskiy
this. Does this mean that it happens during warming? Yonik Seeley wrote: > > On Tue, Nov 11, 2008 at 9:31 PM, oleg_gnatovskiy > <[EMAIL PROTECTED]> wrote: >> Hello. We have an index with 15 million documents working on a >> distributed >> environment, with an i

Re: Query Performance while updating teh index

2008-11-12 Thread oleg_gnatovskiy
Yonik Seeley wrote: > > On Wed, Nov 12, 2008 at 2:06 PM, oleg_gnatovskiy > <[EMAIL PROTECTED]> wrote: >> The rsync seems to have nothing to do with slowness, because while the >> rsync >> is going on, there isn't any reload occurring, once the files

Re: Query Performance while updating teh index

2008-11-12 Thread oleg_gnatovskiy
cache autowarming. >> >> We turned this off and staged separate queries that pre-warm our standard >> queries. We are looking at pulling the query server out of the load >> balancer >> during this process; it is the most effective way to give fixed response >> time. &g

RE: Query Performance while updating teh index

2008-12-11 Thread oleg_gnatovskiy
We are still having this problem. I am wondering if it can be fixed with autowarm settings. Is there a reliable formula for determining the autowarm settings? -- View this message in context: http://www.nabble.com/Query-Performance-while-updating-the-index-tp20452835p20968516.html Sent from the

Re: Query Performance while updating teh index

2008-12-12 Thread oleg_gnatovskiy
Hey Otis, Do you think our problem is slow warm time, or too few items that are being copied? Oleg -- View this message in context: http://www.nabble.com/Query-Performance-while-updating-the-index-tp20452835p20980523.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Query Performance while updating teh index

2008-12-12 Thread oleg_gnatovskiy
Here’s what we have on one of the data slaves for the autowarming. -- Dec 12, 2008 8:46:02 AM org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming searc...@3f32ca2b main from searc...@443ad545 main filterCache{lookups=351993,hits=347055,hitratio=0.98,inserts=8332,eviction

RE: Query Performance while updating teh index

2008-12-12 Thread oleg_gnatovskiy
may > have to set that value in the hundreds, as opposed to tens of thousands. > > -Todd Feak > > -Original Message- > From: oleg_gnatovskiy [mailto:oleg_gnatovs...@citysearch.com] > Sent: Friday, December 12, 2008 10:08 AM > To: solr-user@lucene.apache.org >

RE: Query Performance while updating teh index

2008-12-12 Thread oleg_gnatovskiy
I just verified this. The slowness occurs after auto warm is done. Oleg -- View this message in context: http://www.nabble.com/Query-Performance-while-updating-the-index-tp20452835p20982068.html Sent from the Solr - User mailing list archive at Nabble.com.

RE: Query Performance while updating teh index

2008-12-12 Thread oleg_gnatovskiy
20,000 and see if it helps. > > Second look at your document cache. Document caches don't use autowarm. > But you can add queries to your firstSeacher and newSearcher entries in > your solrconfig to pre-populate the document cache during warming. > > -Todd Feak > >

Re: Query Performance while updating teh index

2009-01-20 Thread oleg_gnatovskiy
Hello again. It seems that we are still having these problems. Queries take as long as 20 minutes to get back to their average response time after a large index update, so it doesn't seem like the problem is the 12 second autowarm time. Are there any more suggestions for things we can try? Taking

Re: Using Threading while Indexing.

2009-01-20 Thread oleg_gnatovskiy
I can verify that multithreaded loading using HTTP does work. That's probably the way to go. zayhen wrote: > > Your 3 instances are trying to acquire the physical lock to the index. > If you want to use multi-threaded indexing, I would suggest http > interface, > as Solr will control the reque

Re: Query Performance while updating teh index

2009-01-21 Thread oleg_gnatovskiy
ndex before the update? Do you *really* need to > do that? > How large is your update, what makes it big, and could you make it > smaller? > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > - Original Message >>

Re: Random queries extremely slow

2009-01-22 Thread oleg_gnatovskiy
My aplogies, this is likely the same issue as "Intermittent high response times by hbi dev " oleg_gnatovskiy wrote: > > Hello. Our production servers are operating relatively smoothly most of > the time running Solr with 19 million listings. However every once in a >

Re: Random queries extremely slow

2009-01-22 Thread oleg_gnatovskiy
which holds true based on the tests we ran. oleg_gnatovskiy wrote: > > My aplogies, this is likely the same issue as "Intermittent high response > times by hbi dev " > > > > oleg_gnatovskiy wrote: >> >> Hello. Our production servers are operat

Re: Random queries extremely slow

2009-01-22 Thread oleg_gnatovskiy
What are some things that could happen to force files out of the cache on a Linux machine? I don't know what kinds of events to look for... yonik wrote: > > On Thu, Jan 22, 2009 at 1:46 PM, oleg_gnatovskiy > wrote: >> Hello. Our production servers are operating relativ

Re: Random queries extremely slow

2009-01-22 Thread oleg_gnatovskiy
large newly optimized index onto the > server. > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > - Original Message >> From: oleg_gnatovskiy >> To: solr-user@lucene.apache.org >> Sent: Thursday, January 22,

Re: Query Performance while updating teh index

2009-01-22 Thread oleg_gnatovskiy
our index before the update? Do you *really* need to > do that? > How large is your update, what makes it big, and could you make it > smaller? > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > - Original Message >>

Re: Query Performance while updating teh index

2009-01-22 Thread oleg_gnatovskiy
hatever > queries are in flight? Try jconsole. > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > - Original Message >> From: oleg_gnatovskiy >> To: solr-user@lucene.apache.org >> Sent: Thursday, January 22, 2009

Re: Random queries extremely slow

2009-01-26 Thread oleg_gnatovskiy
Can you expand on this? Mirroring delay on what? zayhen wrote: > > Use multiple boxes, with a mirroring delaay from one to another, like a > pipeline. > > 2009/1/22 oleg_gnatovskiy > >> >> Well this probably isn't the cause of our random slow queries,

Re: Query Performance while updating teh index

2009-01-26 Thread oleg_gnatovskiy
Just to calrify - we do not optimize on the slaves at all. We only optimize on the master. hossman wrote: > > > : We do optimize the index before updates but we get tehse performance > issues > : even when we pull an empty snapshot. Thus even when our update is tiny, > the > : performance issue

Re: Query Performance while updating teh index

2009-01-26 Thread oleg_gnatovskiy
Just to calrify - we do not optimize on teh slaves at all. We only optimize on the master. hossman wrote: > > > : We do optimize the index before updates but we get tehse performance > issues > : even when we pull an empty snapshot. Thus even when our update is tiny, > the > : performance issue

Re: Query Performance while updating teh index

2009-01-26 Thread oleg_gnatovskiy
Just to clarify - we do not optimize on the slaves at all. We only optimize on the master. hossman wrote: > > > : We do optimize the index before updates but we get tehse performance > issues > : even when we pull an empty snapshot. Thus even when our update is tiny, > the > : performance issue

Re: Random queries extremely slow

2009-04-14 Thread oleg_gnatovskiy
t; thanks a lot, > > > oleg_gnatovskiy wrote: >> >> Can you expand on this? Mirroring delay on what? >> >> >> >> zayhen wrote: >>> >>> Use multiple boxes, with a mirroring delaay from one to another, like a >>> pipeline. &

Re: Reverse querying

2009-06-24 Thread oleg_gnatovskiy
AlexElba wrote: > > Hello, > > I have problem which I am trying to solve using solr. > > I have search text (term) and I have index full of words which are mapped > to ids. > > Is there any query that I can run to do this? > > Example: > > Term > "3) A recommendation to use VAR=value in th

Re: Integrated Spellchecking

2008-02-15 Thread oleg_gnatovskiy
dsteiger wrote: > > I've got a couple search components for automatic spell correction that > I've been working on. > > I've converted most of the SpellCheckerRequestHandler to a search > component (hopefully will throw a > patch out soon for this). Then another search component that will do

Question regarding Solr ranking

2008-02-27 Thread oleg_gnatovskiy
Hello everyone. I've run into a weird problem with Solr's ranking engine. In a nutshell, the problem involves certain results getting EXTREMELY high rank scores. Here is an example: locRvwText:"Pizza Pizza"^10 OR locName:"Pizza Pizza"^30 The way I understand it is that the locName part of th

Re: Question regarding Solr ranking

2008-02-27 Thread oleg_gnatovskiy
Sorry about the previous message, I had some formatting issues. Below is the actual message! oleg_gnatovskiy wrote: > > Hello everyone. > > I've run into a weird problem with Solr's ranking engine. In a nutshell, > the problem involves certain results getting EXTREMELY

Re: Companies Using Solr

2008-02-27 Thread oleg_gnatovskiy
Clay Webster wrote: > > Hey Folks, > > Reminder: http://wiki.apache.org/solr/PublicServers lists the sites using > Solr. The listing is a bit thin. I know many people don't know about the > list or don't have the time to add themselves to the list. I'd like to be > able to promote open sour

Re: Question regarding Solr ranking

2008-02-29 Thread oleg_gnatovskiy
Otis Gospodnetic wrote: > > It's a little hard to read that message, but if I were you I'd go to the > Solr admin page, analysis section, enter your query, and see what index > and query time analyzers spit out. I think that should at least give you > some hints. > > Otis > > -- > Sematext -

Solr-J problem

2008-03-07 Thread oleg_gnatovskiy
Hello. I just started using solrJ recently and ran into a problem. I execute the following line after creating a SolrQuery: SolrDocumentList solrResults = engine.query(solrQuery).getResults();. solrResults.size() is always 10, while solrResults.getNumFound() varies based on the query. My question

Re: Solr-J problem

2008-03-10 Thread oleg_gnatovskiy
> method that you can call before running the query? > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > - Original Message > From: oleg_gnatovskiy <[EMAIL PROTECTED]> > To: solr-user@lucene.apache.org > Sent: Friday, March 7, 2

Query Level Boosting

2008-03-11 Thread oleg_gnatovskiy
Hello. I was wondering if anyone knew a way to do query level boosting with SolrJ. On the http client I could just do something like sku:123^2.3 which would boost the sky query 2.3 points. -- View this message in context: http://www.nabble.com/Query-Level-Boosting-tp15995005p15995005.html Sent f

Replication of Segmented indexes

2008-03-26 Thread oleg_gnatovskiy
Hello, this is actually a repost of a question posed by Swarag. I don't think he made the question quite clear, so let me give it a shot. It is known that Solr has support for index replication, and it has support for index segmentation. The question is, how would you use the replication tools wit

solr commit command questions

2008-04-03 Thread oleg_gnatovskiy
Hello. I was wondering what happens when an add command is done without a commit command. Is there any way to roll back? -- View this message in context: http://www.nabble.com/solr-commit-command-questions-tp16467824p16467824.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: solr commit command questions

2008-04-04 Thread oleg_gnatovskiy
So, what is the point of the commit? oleg_gnatovskiy wrote: > > Hello. I was wondering what happens when an add command is done without a > commit command. Is there any way to roll back? > -- View this message in context: http://www.nabble.com/solr-commit-comma

Nightly build compile error?

2008-04-08 Thread oleg_gnatovskiy
Hello everyone. I downloaded the latest nightly build from http://people.apache.org/builds/lucene/solr/nightly/. When I tried to compile it, I got the following errors: [javac] Compiling 189 source files to /home/csweb/apache-solr-nightly/build/core [javac] /home/csweb/apache-solr-nightly/src

Re: Distributed Search

2008-04-08 Thread oleg_gnatovskiy
We are using the Chain Collapse patch as well. Will that not work over a distributed index? swarag wrote: > > Hi, > I am trying to search through a distributed index and when I enter this > link: > > http://wil1devsch1.cs.tmcs:8983/select?shards=wil1devsch1.cs.tmcs:8983,wil1devsch1.cs.tmcs:8080

Re: Nightly build compile error?

2008-04-09 Thread oleg_gnatovskiy
hossman wrote: > > : > : Hello everyone. I downloaded the latest nightly build from > : http://people.apache.org/builds/lucene/solr/nightly/. When I tried to > : compile it, I got the following errors: > : > : [javac] Compiling 189 source files to > : /home/csweb/apache-solr-nightly/build/cor

Re: Distributed Search

2008-04-09 Thread oleg_gnatovskiy
: > > On Wed, Apr 9, 2008 at 2:00 AM, oleg_gnatovskiy > <[EMAIL PROTECTED]> wrote: >> We are using the Chain Collapse patch as well. Will that not work over a >> distributed index? > > Since there is no explicit distributed support for it, it would only

Re: Distributed Search

2008-04-09 Thread oleg_gnatovskiy
Yonik Seeley wrote: > > On Wed, Apr 9, 2008 at 1:57 PM, oleg_gnatovskiy > <[EMAIL PROTECTED]> wrote: >> Do you have any suggestions as to how we would be able to implement >> chain >> collapse over the entire distributed index? Our collection is 27 GB, 15 &g

Re: too many queries?

2008-04-16 Thread oleg_gnatovskiy
Hello. I am having a similar problem as the OP. I see that you recommended setting 4GB for the index, and 2 for Solr. How do I allocate memory for the index? I was under the impression that Solr did not support a RAMIndex. Walter Underwood wrote: > > Do it. 32-bit OS's went out of style five ye

Re: too many queries?

2008-04-16 Thread oleg_gnatovskiy
/sematext.com/ -- Lucene - Solr - Nutch > > - Original Message > From: oleg_gnatovskiy <[EMAIL PROTECTED]> > To: solr-user@lucene.apache.org > Sent: Wednesday, April 16, 2008 2:05:23 PM > Subject: Re: too many queries? > > > Hello. I am having a similar p

MultiThreaded Document Loader?

2008-04-24 Thread oleg_gnatovskiy
Hello. I was wondering if Solr has some kind of a multi-threaded document loader? I've been using post.sh (curl) to post documents to my Solr server, and it's pretty slow. I know it should be pretty easy to write one up, but I was just wondering if one already existed. -- View this message in con

Field Grouping

2008-05-12 Thread oleg_gnatovskiy
Hello. I was wondering if there is a way to get solr to return fields with the same value for a particular field together. For example I might want to have all the documents with exactly the same name field all returned next to each other. Is this possible? Thanks! -- View this message in context

Re: Field Grouping

2008-05-12 Thread oleg_gnatovskiy
e top... Yonik Seeley wrote: > > On Mon, May 12, 2008 at 9:58 PM, oleg_gnatovskiy > <[EMAIL PROTECTED]> wrote: >> Hello. I was wondering if there is a way to get solr to return fields >> with >> the same value for a particular field together. For example I might want

Re: Field Grouping

2008-05-13 Thread oleg_gnatovskiy
There is an XSLT example here: http://wiki.apache.org/solr/XsltResponseWriter , but it doesn't seem like that would work either... This example would only do a group by for the current page. If I use Solr for pagination, this would not work for me. oleg_gnatovskiy wrote: > > But I

Re: Field Grouping

2008-05-14 Thread oleg_gnatovskiy
e one for trunk needs > some work before it can work... > > ryan > > > On May 13, 2008, at 2:46 PM, oleg_gnatovskiy wrote: >> >> There is an XSLT example here: >> http://wiki.apache.org/solr/XsltResponseWriter >> , but it doesn't seem like that would

Problem getting spelling suggestions to work

2008-05-19 Thread oleg_gnatovskiy
Hello. I am having some trouble getting spelling suggestions to work. I am running the latest nightly build of Solr. The URL I am hitting is: http://localhost:8983/solr/select/?q=pizzza&qt=spellchecker&cmd=rebuild and the response I am getting is 0 14 rebuild pizzza spellc

Re: Problem getting spelling suggestions to work

2008-05-19 Thread oleg_gnatovskiy
ou asking the handler > for spellchecking (q=pizzza) or are you asking it to rebuild the index > (cmd=rebuild)? Asking for both at the same time might not be the best > thing. > > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > -