Re: Near real-time search of user data

2009-02-19 Thread Mark Ferguson
Thanks Noble and Otis for your suggestions. After reading more messages on the mailing list relating to this problem, I decided to implement one suggestion which was to keep an archive index and a smaller delta index containing only recent updates, then do a distributed search across them. The del

Re: Near real-time search of user data

2009-02-19 Thread Noble Paul നോബിള്‍ नोब्ळ्
we have a similar usecase and I have raised an issue for the same (SOLR-880) currently we are using an internal patch and we hopw to submit one soon. we also use an LRU based automatic loading unloading feature. if a request comes up for a core that is 'STOPPED' . the core is 'STARTED' and the req

Re: Near real-time search of user data

2009-02-19 Thread Otis Gospodnetic
I've used a similar strategy for Simpy.com, but with raw Lucene and not Solr. The crucial piece is to close (inactive) user indices periodically and thus free the memory. Are you doing the same with your per-user Solr cores and still running into memory issues? Otis -- Sematext -- http://sem

Re: Long query causes problems

2009-02-19 Thread Otis Gospodnetic
Hi, Could you paste an example of the query that fails? Have you tried shortening it and finding the point where adding another AND/OR clause causes the query to break? Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: Jonathan Huizingh >

Re: Realtime Searching..

2009-02-19 Thread Otis Gospodnetic
Yes, the two are similar. As a matter of fact, Zoie is one of the case studies you'll find in the soon to be published Lucene in Action 2nd edition. I just reviewed this very informative case study a few weeks ago and I think people will like it and will likely end up using Zoie until we get

Re: Updating a single field of a document

2009-02-19 Thread Otis Gospodnetic
Amit, This is still the case. I believe 2 separate issues related to this exist in JIRA, but none is in a finished state. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: Amit Nithian > To: solr-user@lucene.apache.org > Sent: Friday, Feb

Re: Dynamically calculated range facet

2009-02-19 Thread Rohit Gandhe
I know this is an old thread, but I am not sure if this thread concluded with any concrete result. Martin, I see you started some changes to Solr Code in "RangeRequestHandler". Did you ever release them as a patch? Has anybody been working on this issue. I read through most of Hossman's posts an

Re: foreign characters equivalent in solr search

2009-02-19 Thread Chris Hostetter
: if a user searches for Tiesto which is indexed in this format Tiësto in our : solr. we want solr also return result This is what the ISOLatin1AccentFilter is for. It's been included in Solr since 1.1. It's been deprecated in favor of the newer ASCIIFoldingFilter which does a better job with

Re: Facet search on Multi-Valued Fields

2009-02-19 Thread Chris Hostetter
: I am trying to do facet search on 3 fields (all multivalued fields) in one : query. field1 has 2 million distinct values, field2 has 1.5 million distinct : values, field3 has 50,000 distinct values. : : I already set the filterCache to 3,000,000, But the searching speed is still making your fi

Re: Distributed search

2009-02-19 Thread Chris Hostetter
: for exampe if we have 3 cores -core0 ,core1 and core2 for 3 different : languages and to search across all the 3 indexes : use the shard parameter as : shard=localhost:8080/solr/core0,localhost:8080/solr/core1,localhost:8080/solr/core2 Distributed search requires that all of the shards have "c

Re: suggestion queries

2009-02-19 Thread Chris Hostetter
: What's the best way to set up a suggestion box with solr ? : I mean, if i type one letter, it would resquest for all the "categories" : beginning with that letter, and so on as the user adds letters. there really sin't a one-size-fits-all answer to this question, but as a starting point take a

Updating a single field of a document

2009-02-19 Thread Amit Nithian
Is there a way in Solr 1.2 (or Solr 1.3) to update a single field of an existing document if I know the primary key? Reason I ask is that I construct a document from multiple sources and some fields may need periodic updating from one of those sources. I would prefer not to have to reconstruct the

Re: Snowball and protected words

2009-02-19 Thread Erik Hatcher
On Feb 19, 2009, at 8:37 AM, Leonardo Dias wrote: Erik just said it wouldn't be hard to bring that functionality to Snowball. Erik, do you know what needs to be done in order to achieve that? Don't you guys have plans for that? I'm sure that I'm not the only one with that problem using SOLR

Shard Query Problem

2009-02-19 Thread Anshul jain
Hi, I'm using distributed Solr with two machines. when I query in URL: http://lca2-s5-pc04:8080/solr/select?&q=name:john I get and 10 documents are shown. when I query in URL: http://lca2-s5-pc04:8080/solr/select?shards=lca2-s5-pc04:8080/solr&q=name:john I get and 0 documents are shown. when

Re: Query on Level of Access to lucene in Solr

2009-02-19 Thread Chris Hostetter
: Can someone please help me understand the extent to which solr allows : access to lucene? ... : I have my own data structures (in java) that handle efficient graph : walks,etc and eventually : apply a whole bunch of math to re-rank results/result trees. : In a more traditional setting,

Re: Using a modified IndexReader

2009-02-19 Thread Mark Miller
Nickolai Toupikov wrote: Hello! Our application requires the use of a custom IndexReader. SolrCore is final and the IndexReader creation is hardcoded. Is there a way I can have our IndexReader instantiated by the SolrCore without having to actually modify the code in Solr or Lucene? Thanks

Using a modified IndexReader

2009-02-19 Thread Nickolai Toupikov
Hello! Our application requires the use of a custom IndexReader. SolrCore is final and the IndexReader creation is hardcoded. Is there a way I can have our IndexReader instantiated by the SolrCore without having to actually modify the code in Solr or Lucene? Thanks alot! Nickolai

Near real-time search of user data

2009-02-19 Thread Mark Ferguson
Hi, I am trying to come up with a strategy for a solr setup in which a user's indexed data can be nearly immediately available to them for search. My current strategy (which is starting to cause problems) is as follows: - each user has their own personal index (core), which gets committed after

Re: why don't we have a forum for discussion?

2009-02-19 Thread Erik Hatcher
On Feb 19, 2009, at 10:13 AM, Erick Erickson wrote: I admit it takes a while, but Google mail, especially with its threading format has been very useful for me. I rarely need to go to the searchable archive any more since I've been collecting Lucene and SOLR user-list email for quite some tim

Query results problem

2009-02-19 Thread Anshul jain
Hi All, I'm using Solr in distributed setup with two masters and two slaves for each master. This is the code I use for querying the SHRADS: SolrQuery query = new SolrQuery(); query.setQuery(q); query.setParam(ShardParams.SHARDS,shards); query.setRows(new Integer(maxResults)); query.setInc

Re: Realtime Searching..

2009-02-19 Thread Jon Baer
This part: The part of Zoie that enables real-time searchability is the fact that ZoieSystem contains three IndexDataLoader objects: * a RAMLuceneIndexDataLoader, which is a simple wrapper around a RAMDirectory, * a DiskLuceneIndexDataLoader, which can index directly to the FSDire

Re: Good strategy for news in Solr?

2009-02-19 Thread Jon Baer
Yes, more or less, most of my tries have not been function query based just basic dismax handler stuff. I have a bit of a unique case where Im dealing w/ last names + multiple players (think Staal ;-) and non tagged content (feeds) so it's a bit tricky than just news articles and some tend

Long query causes problems

2009-02-19 Thread Jonathan Huizingh
I am using the solr search server for a project that I am working on. I am making pretty heavy use of facets, so the query string is already pretty long but it does work. I am working on a new feature that requires a lot of 'OR's and 'AND's in the query, which makes the query string a lot longer.

Re: LocalSolr distributed search

2009-02-19 Thread pjaol
Hi Most of the localsolr / locallucene doc's are a little out of date I'll get to updating them soon the most relevant ones are on http://www.gissearch.com/ To use it in a distributed form, it should already be built into the trunk version Use the standard query component as your primary entry

Re: indexing entire text but only storing first N characters?

2009-02-19 Thread Koji Sekiguchi
Mike, It will come with Solr 1.4, but if you visit SOLR-538, you can get a patch for Solr 1.3 branch. I'm using the patch with Solr 1.3 on my production site and works well so far. https://issues.apache.org/jira/browse/SOLR-538 Note: you should use "maxLength" attribute instead of "maxChars" wh

Re: why don't we have a forum for discussion?

2009-02-19 Thread Erick Erickson
I admit it takes a while, but Google mail, especially with its threading format has been very useful for me. I rarely need to go to the searchable archive any more since I've been collecting Lucene and SOLR user-list email for quite some time. Please, please, please don't make me have to visit yet

Re: Realtime Searching..

2009-02-19 Thread Genta Kaneyama
Michael, I think you might be get interested in "zoie". zoie: real-time search and indexing system built on Apache Lucene http://code.google.com/p/zoie/ Zoie is realtime search project for lucene by Linkedin. Basically, I think it is similar technique to a Otis's trick. >>In the mean time you

Re: indexing entire text but only storing first N characters?

2009-02-19 Thread Mike Topper
Cool, we are actually still on 1.2 but were planning on upgrading to 1.3 is this a feature of 1.3 or just on the nightly builds? -Mike Koji Sekiguchi wrote: > Mike Topper wrote: >> Hello, >> >> In one of the fields in my schema I am sending somewhat large texts. I >> want to be able to index al

Re: delta-import or update for one special ID

2009-02-19 Thread sunnyfr
It looks like it did update the document, I've just done : book/dataimport?command=full-import&clean=false&start=9327553&rows=1&debug=on&verbose=true&commit=true and I've : Feb 19 15:20:02 search-07 jsvc.exec[3233]: Feb 19, 2009 3:20:02 PM org.apache.solr.core.SolrCore execute INFO: [group] weba

Re: delta-import or update for one special ID

2009-02-19 Thread Noble Paul നോബിള്‍ नोब्ळ्
I was referring to the DIH debug page. But apparently, in some cases it seems to be working for you. can you elaborate , when does it work and when it doesn't? On Thu, Feb 19, 2009 at 5:38 PM, sunnyfr wrote: > > Hi, > Thanks Paul > > I did that : > book/dataimport?command=full-import&clean=false

Re: Snowball and protected words

2009-02-19 Thread Leonardo Dias
Hello Walter. We believe this kind of thing is better managed by a content team that works with user feedback. It would be costly everytime we find a word that brings irrelevant results the fact that, to correct that, we'd need to build a new stemmer. It's a lot better to create a simple inter

Re: foreign characters equivalent in solr search

2009-02-19 Thread AHMET ARSLAN
> we will try that and post the results here but it seems we > may get problem with highlight function. No highlighting works fine with that. I am also using similar filter for turkish chars. I replace ç with c, ş with s and so on at index time. Another (easier but less efficient ) way to imple

Re: indexing entire text but only storing first N characters?

2009-02-19 Thread Koji Sekiguchi
Mike Topper wrote: Hello, In one of the fields in my schema I am sending somewhat large texts. I want to be able to index all of it since I want to search on the entire text, but I only need the first N characters to be returned to me. Is there a way to do this with one field or would I just c

Re: Good strategy for news in Solr?

2009-02-19 Thread Grant Ingersoll
Hey Jon, If I understand right, you want news about a particular player, right? And you need it to be fresh. Can you share more about what you've done so far? It sounds like you have tried out some function query stuff, but can you share what you did there? -Grant On Feb 18, 2009, at

Re: delta-import or update for one special ID

2009-02-19 Thread sunnyfr
It looks like books which have no link with entity are not took in consideration ??? part of my data-config.xml: Is it normal ? sunnyfr wrote: > > Hi, > Thanks Paul > > I did that : > book/dataimport?command=full-import&clean=false&start=9327553&rows=1&debugQuery=true > > but i

Re: delta-import or update for one special ID

2009-02-19 Thread sunnyfr
Hi, Thanks Paul I did that : book/dataimport?command=full-import&clean=false&start=9327553&rows=1&debugQuery=true but it stays stuck like that : 0:2:10.979 1 0 0 0 2009-02-19 13:06:23 but with another Id it works and I've no error ...? and if I make the request straight in MySql database i've

Re: indexing entire text but only storing first N characters?

2009-02-19 Thread Shalin Shekhar Mangar
Change the 'maxFieldLength' in solrconfig.xml to a large value. On Thu, Feb 19, 2009 at 5:11 PM, Mike Topper wrote: > Hello, > > In one of the fields in my schema I am sending somewhat large texts. I > want to be able to index all of it since I want to search on the entire > text, but I only ne

Re: indexing entire text but only storing first N characters?

2009-02-19 Thread revathy arun
I have also done this and i used two separate fields like the ones you mentioned On 2/19/09, Mike Topper wrote: > > Hello, > > In one of the fields in my schema I am sending somewhat large texts. I > want to be able to index all of it since I want to search on the entire > text, but I only need

indexing entire text but only storing first N characters?

2009-02-19 Thread Mike Topper
Hello, In one of the fields in my schema I am sending somewhat large texts. I want to be able to index all of it since I want to search on the entire text, but I only need the first N characters to be returned to me. Is there a way to do this with one field or would I just create two fields, one

Re: delta-import or update for one special ID

2009-02-19 Thread Noble Paul നോബിള്‍ नोब्ळ्
the start and rows is supposed to work . If you put it into debug you may see what is happening On Thu, Feb 19, 2009 at 3:47 PM, sunnyfr wrote: > > Hi > > I looked for a book that I couldn't find in solr's databe. > How can I update just this one by the command in the Url ... I tried : > But it d

delta-import or update for one special ID

2009-02-19 Thread sunnyfr
Hi I looked for a book that I couldn't find in solr's databe. How can I update just this one by the command in the Url ... I tried : But it doesn't seems to work ?? dataimport?command=full-import&clean=false&start=11289500&rows=100 Is there another way ??? maybe the book can't be updated but

Re: Unified search of relational data on Solr?

2009-02-19 Thread Kalidoss MM
Its for searching with almost all the fields we used for seaching, stats we used for list the most viewd image(gallery). thanks, kalidoss.m, On Thu, Feb 19, 2009 at 12:50 PM, Noble Paul നോബിള്‍ नोब्ळ् < noble.p...@gmail.com> wrote: > do you wish to search on the image names or is it that you onl