Re: DocSet: BitDocSet or HashDocSet ?

2008-10-28 Thread Noble Paul നോബിള്‍ नोब्ळ्
bitdocset does not take ~ 14M * sizeof(int) in memory it may take a maximum of 14M/8 bytes in memory ~= 1.75MB On Tue, Oct 28, 2008 at 6:06 PM, Jérôme Etévé <[EMAIL PROTECTED]> wrote: > Hi all, > > In my code, I'd like to keep a subset of my 14M docs which is around > 100k large. > > What i

Re: replication handler - compression

2008-10-28 Thread Noble Paul നോബിള്‍ नोब्ळ्
The new replication feature does not use any unix commands , it is pure java. On the fly compression is hard but possible. I wish to repeat the question. Did you optimize the index? Because a 10:1 compression is not usually observed in an optimized index. Our own experiments showed compression of

RE: Query integer type

2008-10-28 Thread Nguyen, Joe
Never mind. I misused the syntax. :-) -Original Message- From: Nguyen, Joe [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 28, 2008 7:00 Joe To: solr-user@lucene.apache.org Subject: Query integer type SITE is defined as integer. I wanted to select all document whose SITE=3002, but S

Query integer type

2008-10-28 Thread Nguyen, Joe
SITE is defined as integer. I wanted to select all document whose SITE=3002, but SITE of the response was different. http://localhost:8080/solr/mysite/select?indent=on&qt=standard&fl=SITE&fq:SITE:3002 http://localhost:8080/solr/mysite/select?indent=on&qt=dismax&fl=SITE&fq:SITE:3002 http:/

RE: replication handler - compression

2008-10-28 Thread Lance Norskog
Aha! The hint to the actual problem: "When compressed with winzip". You are running Solr on Windows. Snapshots don't work on Windows: they depend on a Unix file system feature. You may be copying the entire index. Not just that, it could be inconsistent. This is a fine topic for a "best practice

RE: Changing field datatype

2008-10-28 Thread Nguyen, Joe
Thanks for your quick reply. What would be a reasonable way to handle this without affecting the end users? Create a new dynamic core with the new schema, load documents to the new core, then swap the cores? At some moments, two mostly identical cores co-exist on solr server, would that impact

Re: Changing field datatype

2008-10-28 Thread Shalin Shekhar Mangar
On Wed, Oct 29, 2008 at 1:55 AM, Nguyen, Joe <[EMAIL PROTECTED]> wrote: > > 1. If I modify datatype of a field 'foo' from string to a sint and > restart the server, what would happen to the existing documents? And > documents added with the new schema? At query time (sort=foo desc), > should I e

Changing field datatype

2008-10-28 Thread Nguyen, Joe
I have a solr core having 2 million lengthy documents. 1. If I modify datatype of a field 'foo' from string to a sint and restart the server, what would happen to the existing documents? And documents added with the new schema? At query time (sort=foo desc), should I expect the documents sorte

Re: Question about textTight

2008-10-28 Thread Stephen Weiss
OK, thanks everyone. Since this is the only thing this field is used for, I think we'll just reindex without the filters and go from there... Now if only I could just reindex that field! Oh well. -- Steve On Oct 28, 2008, at 3:32 PM, Yonik Seeley wrote: I'm wrong: I saw the punctuation

Re: Question about textTight

2008-10-28 Thread Yonik Seeley
I'm wrong: I saw the punctuation being left in for "m_*" and thought that the WordDelimiterFilter wasn't working. So as Todd pointed out, underscores are dropped during indexing and searching. The limitation you are running into is that things like prefix and wildcard queries are not analyzed (so

Re: Question about textTight

2008-10-28 Thread Stephen Weiss
That's strange then. The schema hasn't changed in well over a month, solr's been restarted several times since then to reload synonyms and the whole thing was reindexed just this past week to add in new chinese translations (the fields were already there but left blank).

Re: Question about textTight

2008-10-28 Thread Yonik Seeley
These query parsing results don't match with the config you've posted. Double-check the type of the "name" field and that you have restarted Solr since changing the schema.xml -Yonik On Tue, Oct 28, 2008 at 11:25 AM, Stephen Weiss <[EMAIL PROTECTED]> wrote: > Thanks for the reply. I've been look

Re: Question about textTight

2008-10-28 Thread Stephen Weiss
Thanks for the reply. I've been looking at the debug page... and I really don't see any clues there (maybe I don't know how to read it). 0 1 standard 10 0 on name:(stm 0810 m_*) *,score standard on 2.2 name:(stm 0810 m_*) name:(stm 0810 m_*) +name:stm +name:0810 +name:

Re: Sorting TEXT Field problems :-(

2008-10-28 Thread Kraus, Ralf | pixelhouse GmbH
Thomas Traeger schrieb: Kraus, Ralf | pixelhouse GmbH schrieb: Hello, Querry: {wt=json&rows=30&json.nl=map&start=0&sort=RezeptName+asc} Result : Doppeldecker Eiersalat Curry - Eiersalat Eiersalat Why is my second "Curry..." after "Doppeldecker" ??? RezeptName is a normal "text" field defined

RE: Question about textTight

2008-10-28 Thread Feak, Todd
You may want to take a very close look at what the WordDelimiterFilter is doing. I believe the underscore is dropped entirely during indexing AND searching as it's not alphanumeric. Wiki doco here http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters?highlight=(t okenizer)#head-1c9b83870ca78

Re: Override Similarity

2008-10-28 Thread Walter Underwood
I'm curious, why do you want to disable idf? --wunder On 10/28/08 5:37 AM, "Erik Hatcher" <[EMAIL PROTECTED]> wrote: > > On Oct 28, 2008, at 6:33 AM, Kraus, Ralf | pixelhouse GmbH wrote: >> is there a chance to override the Similarity in my search ? >> In fact I want that all result return a 1 (

Re: Sorting TEXT Field problems :-(

2008-10-28 Thread Thomas Traeger
Kraus, Ralf | pixelhouse GmbH schrieb: Hello, Querry: {wt=json&rows=30&json.nl=map&start=0&sort=RezeptName+asc} Result : Doppeldecker Eiersalat Curry - Eiersalat Eiersalat Why is my second "Curry..." after "Doppeldecker" ??? RezeptName is a normal "text" field defined as : positionInc

Re: Override Similarity

2008-10-28 Thread Erik Hatcher
On Oct 28, 2008, at 6:33 AM, Kraus, Ralf | pixelhouse GmbH wrote: is there a chance to override the Similarity in my search ? In fact I want that all result return a 1 (with the idf methode). Sure thing, see Solr 1.3.0's example/solr/conf/schema.xml

DocSet: BitDocSet or HashDocSet ?

2008-10-28 Thread Jérôme Etévé
Hi all, In my code, I'd like to keep a subset of my 14M docs which is around 100k large. What is according to you the best option in terms of speed and memory usage ? Some basic thoughts tells me the BitDocSet should be the fastest for lookup, but takes ~ 14M * sizeof(int) in memory, whereas

Sorting TEXT Field problems :-(

2008-10-28 Thread Kraus, Ralf | pixelhouse GmbH
Hello, Querry: {wt=json&rows=30&json.nl=map&start=0&sort=RezeptName+asc} Result : Doppeldecker Eiersalat Curry - Eiersalat Eiersalat Why is my second "Curry..." after "Doppeldecker" ??? RezeptName is a normal "text" field defined as : positionIncrementGap="100">

Re: solr 1.3 - MySql - Commit one greek document, no error but nothing showing up by request?

2008-10-28 Thread Grant Ingersoll
What is q=ALL? Is the name of your document ALL? And is that query going against that field? *:* is the way to get "all" documents. On Oct 28, 2008, at 6:45 AM, sunnyfr wrote: Hi everybody, I noticed that greek document wasn't there, so I tried to commit just one document and checked

Re: Override Similarity

2008-10-28 Thread Kraus, Ralf | pixelhouse GmbH
Rafał Kuć schrieb: Hello! You can do it, by extending the DefaultSimilarity class from org.apache.lucene.search package. After that, You need to add one line to schema.xml file, which might look like this: After that, Solr will recognize Your new similarity class and will use it to compute

solr 1.3 - MySql - Commit one greek document, no error but nothing showing up by request?

2008-10-28 Thread sunnyfr
Hi everybody, I noticed that greek document wasn't there, so I tried to commit just one document and checked it, so I did a full import which inserted one document then I commit it manually, checked the log, no error. When I check my statistics I've : numDocs : 1, commits : 1. But then When I t

Re: Override Similarity

2008-10-28 Thread Rafał Kuć
Hello! You can do it, by extending the DefaultSimilarity class from org.apache.lucene.search package. After that, You need to add one line to schema.xml file, which might look like this: After that, Solr will recognize Your new similarity class and will use it to compute score. -- Regards,

Commit some greek characters - solr 1.3

2008-10-28 Thread sunnyfr
Hi, When I try to commit greek characters I've an error : Βίντεο ενός μεγάλου αθλητικού τύπου από Kum All over language works well, If I try the request manually on MySql it works well, I tried as well with a little java function to write it in a document and I get it back properly ... any Idea ?

Override Similarity

2008-10-28 Thread Kraus, Ralf | pixelhouse GmbH
Hello, is there a chance to override the Similarity in my search ? In fact I want that all result return a 1 (with the idf methode). Greets -Ralf-

Override Similarity

2008-10-28 Thread Kraus, Ralf | pixelhouse GmbH
Hello, is there a chance to override the Similarity in my search ? In fact I want that all result return a 1 (with the idf methode). Greets -Ralf-

highlighting problem

2008-10-28 Thread Aleksey Gogolev
Hello. I enabled highlighting and it works perfect, but not for all queries. For example if ?q=canon+powershot I get this response: --- { "responseHeader":{ "status":0, "QTime":4}, "response":{"numFound":296,"start":0,"maxScore":4.3135004