Re: Reverse querying

2009-06-24 Thread oleg_gnatovskiy
AlexElba wrote: > > Hello, > > I have problem which I am trying to solve using solr. > > I have search text (term) and I have index full of words which are mapped > to ids. > > Is there any query that I can run to do this? > > Example: > > Term > "3) A recommendation to use VAR=value in th

Re: Random queries extremely slow

2009-04-14 Thread oleg_gnatovskiy
t; thanks a lot, > > > oleg_gnatovskiy wrote: >> >> Can you expand on this? Mirroring delay on what? >> >> >> >> zayhen wrote: >>> >>> Use multiple boxes, with a mirroring delaay from one to another, like a >>> pipeline. &

Re: Query Performance while updating teh index

2009-01-26 Thread oleg_gnatovskiy
Just to clarify - we do not optimize on the slaves at all. We only optimize on the master. hossman wrote: > > > : We do optimize the index before updates but we get tehse performance > issues > : even when we pull an empty snapshot. Thus even when our update is tiny, > the > : performance issue

Re: Query Performance while updating teh index

2009-01-26 Thread oleg_gnatovskiy
Just to calrify - we do not optimize on teh slaves at all. We only optimize on the master. hossman wrote: > > > : We do optimize the index before updates but we get tehse performance > issues > : even when we pull an empty snapshot. Thus even when our update is tiny, > the > : performance issue

Re: Query Performance while updating teh index

2009-01-26 Thread oleg_gnatovskiy
Just to calrify - we do not optimize on the slaves at all. We only optimize on the master. hossman wrote: > > > : We do optimize the index before updates but we get tehse performance > issues > : even when we pull an empty snapshot. Thus even when our update is tiny, > the > : performance issue

Re: Random queries extremely slow

2009-01-26 Thread oleg_gnatovskiy
Can you expand on this? Mirroring delay on what? zayhen wrote: > > Use multiple boxes, with a mirroring delaay from one to another, like a > pipeline. > > 2009/1/22 oleg_gnatovskiy > >> >> Well this probably isn't the cause of our random slow queries,

Re: Query Performance while updating teh index

2009-01-22 Thread oleg_gnatovskiy
hatever > queries are in flight? Try jconsole. > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > - Original Message >> From: oleg_gnatovskiy >> To: solr-user@lucene.apache.org >> Sent: Thursday, January 22, 2009

Re: Query Performance while updating teh index

2009-01-22 Thread oleg_gnatovskiy
our index before the update? Do you *really* need to > do that? > How large is your update, what makes it big, and could you make it > smaller? > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > - Original Message >>

Re: Random queries extremely slow

2009-01-22 Thread oleg_gnatovskiy
large newly optimized index onto the > server. > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > - Original Message >> From: oleg_gnatovskiy >> To: solr-user@lucene.apache.org >> Sent: Thursday, January 22,

Re: Random queries extremely slow

2009-01-22 Thread oleg_gnatovskiy
What are some things that could happen to force files out of the cache on a Linux machine? I don't know what kinds of events to look for... yonik wrote: > > On Thu, Jan 22, 2009 at 1:46 PM, oleg_gnatovskiy > wrote: >> Hello. Our production servers are operating relativ

Re: Random queries extremely slow

2009-01-22 Thread oleg_gnatovskiy
which holds true based on the tests we ran. oleg_gnatovskiy wrote: > > My aplogies, this is likely the same issue as "Intermittent high response > times by hbi dev " > > > > oleg_gnatovskiy wrote: >> >> Hello. Our production servers are operat

Re: Random queries extremely slow

2009-01-22 Thread oleg_gnatovskiy
My aplogies, this is likely the same issue as "Intermittent high response times by hbi dev " oleg_gnatovskiy wrote: > > Hello. Our production servers are operating relatively smoothly most of > the time running Solr with 19 million listings. However every once in a >

Re: Query Performance while updating teh index

2009-01-21 Thread oleg_gnatovskiy
ndex before the update? Do you *really* need to > do that? > How large is your update, what makes it big, and could you make it > smaller? > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > - Original Message >>

Re: Using Threading while Indexing.

2009-01-20 Thread oleg_gnatovskiy
I can verify that multithreaded loading using HTTP does work. That's probably the way to go. zayhen wrote: > > Your 3 instances are trying to acquire the physical lock to the index. > If you want to use multi-threaded indexing, I would suggest http > interface, > as Solr will control the reque

Re: Query Performance while updating teh index

2009-01-20 Thread oleg_gnatovskiy
Hello again. It seems that we are still having these problems. Queries take as long as 20 minutes to get back to their average response time after a large index update, so it doesn't seem like the problem is the 12 second autowarm time. Are there any more suggestions for things we can try? Taking

RE: Query Performance while updating teh index

2008-12-12 Thread oleg_gnatovskiy
20,000 and see if it helps. > > Second look at your document cache. Document caches don't use autowarm. > But you can add queries to your firstSeacher and newSearcher entries in > your solrconfig to pre-populate the document cache during warming. > > -Todd Feak > >

RE: Query Performance while updating teh index

2008-12-12 Thread oleg_gnatovskiy
I just verified this. The slowness occurs after auto warm is done. Oleg -- View this message in context: http://www.nabble.com/Query-Performance-while-updating-the-index-tp20452835p20982068.html Sent from the Solr - User mailing list archive at Nabble.com.

RE: Query Performance while updating teh index

2008-12-12 Thread oleg_gnatovskiy
may > have to set that value in the hundreds, as opposed to tens of thousands. > > -Todd Feak > > -Original Message- > From: oleg_gnatovskiy [mailto:oleg_gnatovs...@citysearch.com] > Sent: Friday, December 12, 2008 10:08 AM > To: solr-user@lucene.apache.org >

Re: Query Performance while updating teh index

2008-12-12 Thread oleg_gnatovskiy
Here’s what we have on one of the data slaves for the autowarming. -- Dec 12, 2008 8:46:02 AM org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming searc...@3f32ca2b main from searc...@443ad545 main filterCache{lookups=351993,hits=347055,hitratio=0.98,inserts=8332,eviction

Re: Query Performance while updating teh index

2008-12-12 Thread oleg_gnatovskiy
Hey Otis, Do you think our problem is slow warm time, or too few items that are being copied? Oleg -- View this message in context: http://www.nabble.com/Query-Performance-while-updating-the-index-tp20452835p20980523.html Sent from the Solr - User mailing list archive at Nabble.com.

RE: Query Performance while updating teh index

2008-12-11 Thread oleg_gnatovskiy
We are still having this problem. I am wondering if it can be fixed with autowarm settings. Is there a reliable formula for determining the autowarm settings? -- View this message in context: http://www.nabble.com/Query-Performance-while-updating-the-index-tp20452835p20968516.html Sent from the

Re: Query Performance while updating teh index

2008-11-12 Thread oleg_gnatovskiy
cache autowarming. >> >> We turned this off and staged separate queries that pre-warm our standard >> queries. We are looking at pulling the query server out of the load >> balancer >> during this process; it is the most effective way to give fixed response >> time. &g

Re: Query Performance while updating teh index

2008-11-12 Thread oleg_gnatovskiy
Yonik Seeley wrote: > > On Wed, Nov 12, 2008 at 2:06 PM, oleg_gnatovskiy > <[EMAIL PROTECTED]> wrote: >> The rsync seems to have nothing to do with slowness, because while the >> rsync >> is going on, there isn't any reload occurring, once the files

Re: Query Performance while updating teh index

2008-11-12 Thread oleg_gnatovskiy
this. Does this mean that it happens during warming? Yonik Seeley wrote: > > On Tue, Nov 11, 2008 at 9:31 PM, oleg_gnatovskiy > <[EMAIL PROTECTED]> wrote: >> Hello. We have an index with 15 million documents working on a >> distributed >> environment, with an i

Query Performance while updating teh index

2008-11-11 Thread oleg_gnatovskiy
Hello. We have an index with 15 million documents working on a distributed environment, with an index distribution setup. While an index on a slave server is being updated, query response times become extremely slow (upwards of 5 seconds). Is there any way to decrease the hit query response times

Different XML format for multi-valued fields?

2008-10-16 Thread oleg_gnatovskiy
Hello. I have an index built in Solr with several multi-value fields. When the multi-value field has only one value for a document, the XML returned looks like this: 5693 However, when there are multiple values for the field, the XMl looks like this: arr name="someIds"> 11199 1722 Is there a

Re: File based index doesn't work in spellcheck component

2008-09-19 Thread oleg_gnatovskiy
oleg_gnatovskiy wrote: > > Hello, > > I tried to have the spellcheck component to write to a drive index. My > config is a s follows: > > >name="classname">org.apache.solr.spelling.FileBasedSpellChecker > external >

File based index doesn't work in spellcheck component

2008-09-19 Thread oleg_gnatovskiy
Hello, I tried to have the spellcheck component to write to a drive index. My config is a s follows: org.apache.solr.spelling.FileBasedSpellChecker external spellings.txt UTF-8 true ./spellIndex org.apache.lucene.search.spell.JaroWinklerDistance

Re: firstSearcher and newSearcher events

2008-09-19 Thread oleg_gnatovskiy
On Fri, Sep 19, 2008 at 10:07 PM, oleg_gnatovskiy < > [EMAIL PROTECTED]> wrote: > >> >> Is there any way to do it for an external (file-based) dictionary? >> > > SpellCheckComponent always reload on the dictionary in the firstSearcher > event. This works if y

Re: firstSearcher and newSearcher events

2008-09-19 Thread oleg_gnatovskiy
Shalin Shekhar Mangar wrote: > > On Fri, Sep 19, 2008 at 5:55 AM, oleg_gnatovskiy < > [EMAIL PROTECTED]> wrote: > >> >> Hello. I am using the spellcheck component >> (https://issues.apache.org/jira/browse/SOLR-572). Since the spell checker >> index

firstSearcher and newSearcher events

2008-09-18 Thread oleg_gnatovskiy
Hello. I am using the spellcheck component (https://issues.apache.org/jira/browse/SOLR-572). Since the spell checker index is kept in RAM, it gets erased every time the Solr server gets restarted. I was thinking of using either the firstSearcher or the newSearcher to reload the index every time So

Re: Problem getting spelling suggestions to work

2008-05-19 Thread oleg_gnatovskiy
ou asking the handler > for spellchecking (q=pizzza) or are you asking it to rebuild the index > (cmd=rebuild)? Asking for both at the same time might not be the best > thing. > > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > -

Problem getting spelling suggestions to work

2008-05-19 Thread oleg_gnatovskiy
Hello. I am having some trouble getting spelling suggestions to work. I am running the latest nightly build of Solr. The URL I am hitting is: http://localhost:8983/solr/select/?q=pizzza&qt=spellchecker&cmd=rebuild and the response I am getting is 0 14 rebuild pizzza spellc

Re: Field Grouping

2008-05-14 Thread oleg_gnatovskiy
e one for trunk needs > some work before it can work... > > ryan > > > On May 13, 2008, at 2:46 PM, oleg_gnatovskiy wrote: >> >> There is an XSLT example here: >> http://wiki.apache.org/solr/XsltResponseWriter >> , but it doesn't seem like that would

Re: Field Grouping

2008-05-13 Thread oleg_gnatovskiy
There is an XSLT example here: http://wiki.apache.org/solr/XsltResponseWriter , but it doesn't seem like that would work either... This example would only do a group by for the current page. If I use Solr for pagination, this would not work for me. oleg_gnatovskiy wrote: > > But I

Re: Field Grouping

2008-05-12 Thread oleg_gnatovskiy
e top... Yonik Seeley wrote: > > On Mon, May 12, 2008 at 9:58 PM, oleg_gnatovskiy > <[EMAIL PROTECTED]> wrote: >> Hello. I was wondering if there is a way to get solr to return fields >> with >> the same value for a particular field together. For example I might want

Field Grouping

2008-05-12 Thread oleg_gnatovskiy
Hello. I was wondering if there is a way to get solr to return fields with the same value for a particular field together. For example I might want to have all the documents with exactly the same name field all returned next to each other. Is this possible? Thanks! -- View this message in context

MultiThreaded Document Loader?

2008-04-24 Thread oleg_gnatovskiy
Hello. I was wondering if Solr has some kind of a multi-threaded document loader? I've been using post.sh (curl) to post documents to my Solr server, and it's pretty slow. I know it should be pretty easy to write one up, but I was just wondering if one already existed. -- View this message in con

Re: too many queries?

2008-04-16 Thread oleg_gnatovskiy
/sematext.com/ -- Lucene - Solr - Nutch > > - Original Message > From: oleg_gnatovskiy <[EMAIL PROTECTED]> > To: solr-user@lucene.apache.org > Sent: Wednesday, April 16, 2008 2:05:23 PM > Subject: Re: too many queries? > > > Hello. I am having a similar p

Re: too many queries?

2008-04-16 Thread oleg_gnatovskiy
Hello. I am having a similar problem as the OP. I see that you recommended setting 4GB for the index, and 2 for Solr. How do I allocate memory for the index? I was under the impression that Solr did not support a RAMIndex. Walter Underwood wrote: > > Do it. 32-bit OS's went out of style five ye

Re: Distributed Search

2008-04-09 Thread oleg_gnatovskiy
Yonik Seeley wrote: > > On Wed, Apr 9, 2008 at 1:57 PM, oleg_gnatovskiy > <[EMAIL PROTECTED]> wrote: >> Do you have any suggestions as to how we would be able to implement >> chain >> collapse over the entire distributed index? Our collection is 27 GB, 15 &g

Re: Distributed Search

2008-04-09 Thread oleg_gnatovskiy
: > > On Wed, Apr 9, 2008 at 2:00 AM, oleg_gnatovskiy > <[EMAIL PROTECTED]> wrote: >> We are using the Chain Collapse patch as well. Will that not work over a >> distributed index? > > Since there is no explicit distributed support for it, it would only

Re: Nightly build compile error?

2008-04-09 Thread oleg_gnatovskiy
hossman wrote: > > : > : Hello everyone. I downloaded the latest nightly build from > : http://people.apache.org/builds/lucene/solr/nightly/. When I tried to > : compile it, I got the following errors: > : > : [javac] Compiling 189 source files to > : /home/csweb/apache-solr-nightly/build/cor

Re: Distributed Search

2008-04-08 Thread oleg_gnatovskiy
We are using the Chain Collapse patch as well. Will that not work over a distributed index? swarag wrote: > > Hi, > I am trying to search through a distributed index and when I enter this > link: > > http://wil1devsch1.cs.tmcs:8983/select?shards=wil1devsch1.cs.tmcs:8983,wil1devsch1.cs.tmcs:8080

Nightly build compile error?

2008-04-08 Thread oleg_gnatovskiy
Hello everyone. I downloaded the latest nightly build from http://people.apache.org/builds/lucene/solr/nightly/. When I tried to compile it, I got the following errors: [javac] Compiling 189 source files to /home/csweb/apache-solr-nightly/build/core [javac] /home/csweb/apache-solr-nightly/src

Re: solr commit command questions

2008-04-04 Thread oleg_gnatovskiy
So, what is the point of the commit? oleg_gnatovskiy wrote: > > Hello. I was wondering what happens when an add command is done without a > commit command. Is there any way to roll back? > -- View this message in context: http://www.nabble.com/solr-commit-comma

solr commit command questions

2008-04-03 Thread oleg_gnatovskiy
Hello. I was wondering what happens when an add command is done without a commit command. Is there any way to roll back? -- View this message in context: http://www.nabble.com/solr-commit-command-questions-tp16467824p16467824.html Sent from the Solr - User mailing list archive at Nabble.com.

Replication of Segmented indexes

2008-03-26 Thread oleg_gnatovskiy
Hello, this is actually a repost of a question posed by Swarag. I don't think he made the question quite clear, so let me give it a shot. It is known that Solr has support for index replication, and it has support for index segmentation. The question is, how would you use the replication tools wit

Query Level Boosting

2008-03-11 Thread oleg_gnatovskiy
Hello. I was wondering if anyone knew a way to do query level boosting with SolrJ. On the http client I could just do something like sku:123^2.3 which would boost the sky query 2.3 points. -- View this message in context: http://www.nabble.com/Query-Level-Boosting-tp15995005p15995005.html Sent f

Re: Solr-J problem

2008-03-10 Thread oleg_gnatovskiy
> method that you can call before running the query? > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > - Original Message > From: oleg_gnatovskiy <[EMAIL PROTECTED]> > To: solr-user@lucene.apache.org > Sent: Friday, March 7, 2

Solr-J problem

2008-03-07 Thread oleg_gnatovskiy
Hello. I just started using solrJ recently and ran into a problem. I execute the following line after creating a SolrQuery: SolrDocumentList solrResults = engine.query(solrQuery).getResults();. solrResults.size() is always 10, while solrResults.getNumFound() varies based on the query. My question

Re: Question regarding Solr ranking

2008-02-29 Thread oleg_gnatovskiy
Otis Gospodnetic wrote: > > It's a little hard to read that message, but if I were you I'd go to the > Solr admin page, analysis section, enter your query, and see what index > and query time analyzers spit out. I think that should at least give you > some hints. > > Otis > > -- > Sematext -

Re: Companies Using Solr

2008-02-27 Thread oleg_gnatovskiy
Clay Webster wrote: > > Hey Folks, > > Reminder: http://wiki.apache.org/solr/PublicServers lists the sites using > Solr. The listing is a bit thin. I know many people don't know about the > list or don't have the time to add themselves to the list. I'd like to be > able to promote open sour

Re: Question regarding Solr ranking

2008-02-27 Thread oleg_gnatovskiy
Sorry about the previous message, I had some formatting issues. Below is the actual message! oleg_gnatovskiy wrote: > > Hello everyone. > > I've run into a weird problem with Solr's ranking engine. In a nutshell, > the problem involves certain results getting EXTREMELY

Question regarding Solr ranking

2008-02-27 Thread oleg_gnatovskiy
Hello everyone. I've run into a weird problem with Solr's ranking engine. In a nutshell, the problem involves certain results getting EXTREMELY high rank scores. Here is an example: locRvwText:"Pizza Pizza"^10 OR locName:"Pizza Pizza"^30 The way I understand it is that the locName part of th

Re: Integrated Spellchecking

2008-02-15 Thread oleg_gnatovskiy
dsteiger wrote: > > I've got a couple search components for automatic spell correction that > I've been working on. > > I've converted most of the SpellCheckerRequestHandler to a search > component (hopefully will throw a > patch out soon for this). Then another search component that will do