Re: batch indexing takes more time than shown on SOLR output --> something to do with IO?

2008-01-17 Thread Otis Gospodnetic
Paging - quite possible. Run vmstat 2' on that server when its slow and see if there is any paging (look at the 'si' and 'so' columns). Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Geert-Jan Brits <[EMAIL PROTECTED]> To: solr-user@lucene.ap

Re: Some sort of join in SOLR?

2008-01-17 Thread Michael Lackhoff
On 17.01.2008 23:48 Chris Hostetter wrote: assuming these are simple delimited files, something like the unix "join" command can do this for you ... then your indexing code can just process on file linerally. (if they aren't simple delimited files, you can preprocess them to strip out the exc

Re: Getting SolrSharp to work

2008-01-17 Thread Jeff Rodenburg
Thanks for the updates Peter. Can you pass along your Solr instance information? thanks, jeff r. On Jan 17, 2008 7:22 AM, Peter Thygesen <[EMAIL PROTECTED]> wrote: > Version solrsharp-Dec-30-2007.zip > > I had to make a few changes to get this version to work. Here is what I > did. > > ---

Re: SolrSharp and UTF-8, danish letters are messed up

2008-01-17 Thread Jeff Rodenburg
Hi Peter - Hmm, not sure what's going on here. Please take a look in the SolrConfigurationSection class, which sets the Culture setting. thanks, jeff r. On Jan 17, 2008 6:05 AM, Peter Thygesen <[EMAIL PROTECTED]> wrote: > Norwegian and danish letters like "ÆØÅ" are messed up when indexing us

Re: copyField limitation

2008-01-17 Thread Chris Hostetter
: But, the directive in the schema has a limitation. It will only : copy data between fields with the same type. If the two fields are a : different type, the copy is ignored. This example would require : to translate 'sint' to 'integer'. i can't reproduce this problem. with the following add

Related searches

2008-01-17 Thread anuvenk
How to accomplish this with solr? Say for a search term like 'alternative dispute resolution', i want to be able to display 'see related searches' and display related search terms like 'mediation', 'arbitration' etc. Has anyone done something like this before? If so please share your ideas. -- V

Re: Related Search

2008-01-17 Thread anuvenk
I'm also looking in to displaying 'related search terms' for every search term. How to do it with solr? The thread explains about some FuzzyLikeThisQuery. Can this be used with solr? Has anyone done this before? Could you please share your ideas. ajaxtrend wrote: > > Take a look at this thread >

Re: copyField limitation

2008-01-17 Thread Yonik Seeley
On Jan 17, 2008 4:53 PM, Lance Norskog <[EMAIL PROTECTED]> wrote: > Because sort works much faster on type 'integer', but range queries do not > work on type 'integer', Really? The sort speed should be identical. -Yonik

Re: Some sort of join in SOLR?

2008-01-17 Thread Chris Hostetter
: I don't have an index to check. Both sources come in huge text files, one of : them daily, the other irregular. One has the ID, the other has a different ID : that must be mapped first to the ID of the first source. So there is no easy : way of saying: "Give me the record to this ID from the oth

Re: Big number of conditions of the search

2008-01-17 Thread Mike Klaas
My first suggestion would be to find something semantic to select on other than arbitrary ids. For instance, are they all of a set of categories, results from another query, etc? It is odd to have a list of 10k completely arbitrary items. Other than that, I don't think there is any way to

Re: copyField limitation

2008-01-17 Thread Ryan McKinley
But, the directive in the schema has a limitation. It will only copy data between fields with the same type. If the two fields are a different type, the copy is ignored. This example would require to translate 'sint' to 'integer'. really? what version are you running? what error do you ge

copyField limitation

2008-01-17 Thread Lance Norskog
Hi- Because sort works much faster on type 'integer', but range queries do not work on type 'integer', I want to do this: > ... So, a_number_sort always contains the same data as a_number, and now we can do a fast sort on 'a_number_sort' AND do a range query o

Re: Big number of conditions of the search

2008-01-17 Thread evgeniy . strokin
Sorry,.. my fault, please disregard previous email,.. It returns right number of documents always,... I sent duplicated IDs after first 1000. Sorry to bother,.. I go get some coffee... But any suggestions of solving this problem of running big queries are still welcome))) Thank you...

Re: Integrated Spellchecking

2008-01-17 Thread Grant Ingersoll
On Jan 17, 2008, at 3:01 PM, Doug Steigerwald wrote: I've got a couple search components for automatic spell correction that I've been working on. I've converted most of the SpellCheckerRequestHandler to a search component (hopefully will throw a patch out soon for this). Then another s

Re: Big number of conditions of the search

2008-01-17 Thread evgeniy . strokin
I still want to give a try to the simple idea just to combine all those IDs in one query. Again, lets say I have a list of IDs in some file. I need to get all docs from Solr with those IDs. I run query: id:(123 234 456 ***more IDs goes like this, I was planning to have about 10K IDs***) Solr has

Re: Some sort of join in SOLR?

2008-01-17 Thread Erick Erickson
Good Luck! You're right, there *is* a lot to learn. I get both excited and frustrated by new stuff, which is much better than having my skill set only comprised of, say, only being able to work with old "C" code. Before getting fancy at all, I'd find out what the total size of my index will

Re: Integrated Spellchecking

2008-01-17 Thread Doug Steigerwald
I've got a couple search components for automatic spell correction that I've been working on. I've converted most of the SpellCheckerRequestHandler to a search component (hopefully will throw a patch out soon for this). Then another search component that will do auto correction for a query if

Re: Integrated Spellchecking

2008-01-17 Thread Yonik Seeley
On Jan 17, 2008 2:33 PM, Ryan McKinley <[EMAIL PROTECTED]> wrote: > Yes -- this is what search components are for! > > Depending on where you put it in the chain, it could only return spell > checked results if there are too few results (or the top score is below > some threshold) Score thresholds

Re: Integrated Spellchecking

2008-01-17 Thread Ryan McKinley
Yes -- this is what search components are for! Depending on where you put it in the chain, it could only return spell checked results if there are too few results (or the top score is below some threshold) ryan Grant Ingersoll wrote: Is it feasible to submit a query to any of the various ha

Integrated Spellchecking

2008-01-17 Thread Grant Ingersoll
Is it feasible to submit a query to any of the various handlers and have it bring back results and spelling suggestions all in one response? Is this something the query components piece would handle, assuming one exists for the spell checker? Thanks, Grant

Re: Spell checker index rebuild

2008-01-17 Thread Doug Steigerwald
Seemed to be able to fix the below problem with the following patch in lucene-2.2. Going to try the lucene 2.3 branch. Index: contrib/spellchecker/src/java/org/apache/lucene/search/spell/SpellChecker.java === --- contrib/spellche

Re: Some sort of join in SOLR?

2008-01-17 Thread Michael Lackhoff
On 17.01.2008 18:32 Erick Erickson wrote: There's some cost here, and I don't know how this all plays with the sizes of your indexes. It may be totally impractical. Anyway, back to work. I think I will have to play with the different possibilities and see what fits best to my situation. Ther

Re: Some sort of join in SOLR?

2008-01-17 Thread Erick Erickson
See below: On Jan 17, 2008 11:42 AM, Michael Lackhoff <[EMAIL PROTECTED]> wrote: > On 17.01.2008 16:53 Erick Erickson wrote: > > > I would *strongly* encourage you to store them together > > as one document. There's no real method of doing > > DB like joins in the underlying Lucene search engine.

Re: Some sort of join in SOLR?

2008-01-17 Thread Michael Lackhoff
On 17.01.2008 16:53 Erick Erickson wrote: I would *strongly* encourage you to store them together as one document. There's no real method of doing DB like joins in the underlying Lucene search engine. Thanks, that was also my preference. But that's generic advice. The question I have for you

Re: Some sort of join in SOLR?

2008-01-17 Thread Erick Erickson
I would *strongly* encourage you to store them together as one document. There's no real method of doing DB like joins in the underlying Lucene search engine. But that's generic advice. The question I have for you is "What's the big deal about coordinating the sources?" That is, you have to have s

Getting SolrSharp to work

2008-01-17 Thread Peter Thygesen
Version solrsharp-Dec-30-2007.zip I had to make a few changes to get this version to work. Here is what I did. -- // fix 1,. Solr server messed up Danish/Norwegian letters like "æøå" because the HttpWbRequest object wasn't sending in utf-8 to Solr server Class: SolrSe

SolrSharp and UTF-8, danish letters are messed up

2008-01-17 Thread Peter Thygesen
Norwegian and danish letters like "ÆØÅ" are messed up when indexing using SolrSharp??? I've checked the C# code, and to me it looks right. If I instead write my documents to disk (in utf-8), uploads the file to my solr server and runs post.jar from the tutorial, everything works out correctly

Re: Spell checker index rebuild

2008-01-17 Thread Doug Steigerwald
It's in the index. Can see it with a query: q=word:blackjack And in luke: − 29 The actual index data seems to disappear. First rebuild: $ ls spell/ _2.cfs segments.gen segments_i Second rebuild: $ ls spell segments_2z segments.gen doug Otis Gospodnetic wrote: Do you trust the

Re: XSLT to preprocess XML documents into 'update xml documents' ?

2008-01-17 Thread Karen Loughran
Thanks Ryan, we'll have a go at this, Karen On Tuesday 15 January 2008 16:41:02 Ryan McKinley wrote: > I have not tried it, but check: > https://issues.apache.org/jira/browse/SOLR-285 > > Karen Loughran wrote: > > Hi all, > > > > I noticed some recent discussion with regard to using XSLT to pr

Re: batch indexing takes more time than shown on SOLR output --> something to do with IO?

2008-01-17 Thread Geert-Jan Brits
No Solr-commit is sent until the end. Since client and server on this moment are on the same machine network IO should be small as well I think. Also as you mentioned the response is very small so it can't be that either. As to what IO-activity I was thinking about: I was merely guessing here, but