Sorting on primitive types

2009-09-21 Thread Aleksander Stensby
Hey, I have a question regarding the primitive type definitions and use of those for sorting. I have an ID field in my index of type SortableLongField, and on my test index I have about 2 million documents. When doing a sort=id desc and q=*:* I'm getting out of memory (heap space)... running the i

Overlapping zipcodes

2009-09-21 Thread Anders Melchiorsen
We are in a situation where we are trying to match up documents based on a number of zipcodes. In our case, zipcodes are just integers, so that hopefully simplifies things. So, we might have a document listing a number of zipcodes: 1200-1450,2000,5000-5999 and we want to do a search of "1100

Highlighting in stemmed or n-grammed fields possible?

2009-09-21 Thread David Espinosa
Hi, Anybody knows how to get the highlighted field, when q term matches in a stemmed or n-grammed filtered field? Matching in a normal field (not stemmed or n-grammed) highlighting works perfectly as expected. But in stemmed matching cases, no highlighting fields are recovered, and in n-gramming

Re: Highlighting in stemmed or n-grammed fields possible?

2009-09-21 Thread Koji Sekiguchi
In stemmed matching cases, highlighter should work correctly. For example, using example data, I can get highlighted snippets that include "memory" when quering features:memories. With respect to highlighting in n-gram field, Solr cannot highlight terms in out-of-the-box. We have an open issue fo

Re: acts_as_solr integeration with solr separately

2009-09-21 Thread Erik Hatcher
acts_as_solr accesses the Solr server listed in the config solr.yml file. You don't have to use the start/stop Rake actions, they are really just conveniences for development/testing (I personally would launch Solr separately in production though). Out of curiosity, what acts_as_solr versi

Re: acts_as_solr integeration with solr separately

2009-09-21 Thread abhay kumar
Thanks Erik for your knd response. I am using acts_as_solr stable release v0.9 . I downloaded it from http://github.com/mattmatt/acts_as_solr/tree/master Regards Abhay On Mon, Sep 21, 2009 at 3:41 PM, Erik Hatcher wrote: > acts_as_solr accesses the Solr server listed in the config solr.yml fi

Re: Overlapping zipcodes

2009-09-21 Thread Avlesh Singh
Range queries? Cheers Avlesh On Mon, Sep 21, 2009 at 2:57 PM, Anders Melchiorsen wrote: > We are in a situation where we are trying to match up documents based on a > number of zipcodes. > > In our case, zipcodes are just integers, so that hopefully simplifies > things. > > > So, we might have

Re: Sorting on primitive types

2009-09-21 Thread Yonik Seeley
On Mon, Sep 21, 2009 at 3:30 AM, Aleksander Stensby wrote: > So I'm wondering if the Trie based field types are less memory expensive > than the "old" SortableXXFields? > sorting on the date field (which is a TrieDateField) works fine (and > fast)... In general, yes (assuming there are many uniqu

Re: Overlapping zipcodes

2009-09-21 Thread Anders Melchiorsen
Yeah, that takes care of the query side, but how can we index a list like that? It seems wrong to create a multivalue zipcode field and populate it with each individual number in all the ranges. Regards, Anders. On Mon, 21 Sep 2009 19:05:01 +0530, Avlesh Singh wrote: > Range queries? > > Chee

Re: Overlapping zipcodes

2009-09-21 Thread Erik Hatcher
Seems wrong, but actually is how I've done this sort of thing (with year ranges like 1860-1865). Denormalizing/expanding is a pretty common way to solve problems with Lucene/Solr. There's not that many zip codes, so expanding shouldn't be prohibitive. Erik On Sep 21, 2009, at 9:

define index at search time

2009-09-21 Thread DHast
is there a way i can actually tell solr which index i want it to search against with the query? I know it will cost a bit on performance, but it would be helpful i have many indexes and it would be nice to determine which one should be used by the user. thanks -- View this message in context: ht

Re: define index at search time

2009-09-21 Thread Avlesh Singh
Are you talking about multiple cores? Cheers Avlesh On Mon, Sep 21, 2009 at 9:15 PM, DHast wrote: > > is there a way i can actually tell solr which index i want it to search > against with the query? I know it will cost a bit on performance, but it > would be helpful > i have many indexes and i

XSD for Solr Response Format Version 2.2

2009-09-21 Thread Israel Ekpo
I am working on an XSD document for all the types in the response xml version 2.2 Do you think there is a need for this? -- "Good Enough" is not good enough. To give anything less than your best is to sacrifice the gift. Quality First. Measure Twice. Cut Once.

Re: Latest trunk locks execution thread in SolrCore.getSearcher()

2009-09-21 Thread Chris Harris
Solr now starts up fine in Tomcat with both JMX and ReplicationHandler enabled if I construct my working copy as follows: download solr-r815830 reverse merge r815587 ("SOLR-1427: fixed registry MBean issue") apply 1427.afterlatch.patch I haven't tried applying 1427.afterlatch.patch to the svn hea

Re: Sorting on primitive types

2009-09-21 Thread Aleksander Stensby
Perfect, thanks a heap Yonik! Cheers, Aleks On Mon, Sep 21, 2009 at 3:47 PM, Yonik Seeley wrote: > On Mon, Sep 21, 2009 at 3:30 AM, Aleksander Stensby > wrote: > > So I'm wondering if the Trie based field types are less memory expensive > > than the "old" SortableXXFields? > > sorting on the d

Re: How to leverage the LogMergePolicy "calibrateSizeByDeletes" patch in Solr ?

2009-09-21 Thread Jason Rutherglen
John, It would be great if Lucene's benchmark were used so everyone could execute the test in their own environment and verify. It's not clear the settings or code used to generate the results so it's difficult to draw any reliable conclusions. The steep spike shows greater evidence for the IO ca

Re: Questions on RandomSortField

2009-09-21 Thread Avlesh Singh
The question was either non-trivial or heavily uninteresting! No replies yet :) Thankfully, I figured out a solution for the problem at hand. For people who might be looking for a solution, here it goes - 1. Extended the RandomSortField to create your own YourCustomRandomField. 2. Override

Re: what is too large for an indexed field

2009-09-21 Thread Mark Miller
Park, Michael wrote: > I am trying to place the value of around 390,000 characters into a > single field. However, my search results have become inaccurate. Is > this too large? I tried bumping the maxFieldLength in the > solrconfig.xml file to 500,000 and it hasn't fixed the problem. > > > >

Re: what is too large for an indexed field

2009-09-21 Thread Yonik Seeley
On Mon, Sep 21, 2009 at 3:27 PM, Park, Michael wrote: > I am trying to place the value of around 390,000 characters into a > single field.  However, my search results have become inaccurate. Do you mean that the document should score higher, or that the document doesn't match a particular query?

RE: what is too large for an indexed field

2009-09-21 Thread Park, Michael
I get no results back on a search. But I can see the actual word or phrase in the stored doc. -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Monday, September 21, 2009 4:18 PM To: solr-user@lucene.apache.org Subject: Re: what is too

RE: what is too large for an indexed field

2009-09-21 Thread Park, Michael
I'm using the solr.WhitespaceTokenizerFactory and the solr.LowerCaseFilterFactory. Is it safe to assume that a token would be created for each word? I can't image anything that would be even close to 16383 chars. Is there a way to dissect the tokens? Thanks, Mike -Original Message- F

Re: what is too large for an indexed field

2009-09-21 Thread Yonik Seeley
On Mon, Sep 21, 2009 at 4:22 PM, Park, Michael wrote: > I get no results back on a search.  But I can see the actual word or phrase > in the stored doc. Ok cool - that should make it much easier to debug. #1) verify that you changed the maxFieldLength property in both places in solrconfig.xml, a

http://beerpla.net/2009/09/21/solr-how-to-fix-java-io-ioexception-directory-foo-exists-and-is-a-directory-but-cannot-be-listed-list-returned-null/

2009-09-21 Thread Archon810
-- View this message in context: http://www.nabble.com/http%3A--beerpla.net-2009-09-21-solr-how-to-fix-java-io-ioexception-directory-foo-exists-and-is-a-directory-but-cannot-be-listed-list-returned-null--tp25530499p25530499.html Sent from the Solr - User mailing list archive at Nabble.com.

How To Fix java.io.IOException: directory FOO exists and is a directory, but cannot be listed: list() returned null

2009-09-21 Thread Archon810
I recently hit the problem described in the topic and posted a quick blog post on how to fix it. Here it is, in case someone will ever encounter it. http://beerpla.net/2009/09/21/solr-how-to-fix-java-io-ioexception-directory-foo-exists-and-is-a-directory-but-cannot-be-listed-list-returned-null/ h

Limit number of docs that can be indexed (security)

2009-09-21 Thread Valdir Salgueiro
Hello, I need a way to limit the number of documents that can be indexed on my solr-based application. Here is what I have come up with: create a * UpdateRequestProcessor* and register it on *solrconfig.xml*. When the user tries to add a document, check if the docs limit has been reached. The prob

Re: Limit number of docs that can be indexed (security)

2009-09-21 Thread Israel Ekpo
Valdir, I think you are making it more complicated that it needs to be. As the administrator, if you don't want them to modify the contents of the solrconfig.xml file then you should not give them access to do so. If they already have access to change the contents of the file, you can revoke suc

Re: Exact word search in Solr

2009-09-21 Thread bhaskar chandrasekar
  Hi,   I followed with the below metioned steps and i am not getting the desired results. Let me know if anything else to be done.   Regards Bhaskar --- On Fri, 9/18/09, AHMET ARSLAN wrote: From: AHMET ARSLAN Subject: Re: Exact word search in Solr To: solr-user@lucene.apache.org Date: Friday

Re: Exact word search in Solr

2009-09-21 Thread Chris Hostetter
: I followed with the below metioned steps and i am not getting the desired results. : Let me know if anything else to be done. Ahmet's point was to declare the text_ws fieldtype and then use it on the field you want to have an "exact" match. FWIW: describing your goal as an "exact word search