Re: SolrCore.getSearcher() and postCommit()

2010-10-30 Thread Grant Ingersoll
On Oct 29, 2010, at 5:48 PM, Yonik Seeley wrote: > On Fri, Oct 29, 2010 at 5:36 PM, Grant Ingersoll wrote: >> Is it OK to call and increment a Searcher ref (i.e. SolrCore.getSearcher()) >> in a SolrEventListener.postCommit() hook as long as I decrement it when I am >> done? I need to get a ha

Re: Would it be nuts to store a bunch of large attachments (images, videos) in stored but-not-indexed fields

2010-10-30 Thread Grant Ingersoll
On Oct 29, 2010, at 6:00 PM, Ron Mayer wrote: > I have some documents with a bunch of attachments (images, thumbnails > for them, audio clips, word docs, etc); and am currently dealing with > them by just putting a path on a filesystem to them in solr; and then > jumping through hoops of keeping

Re: Would it be nuts to store a bunch of large attachments (images, videos) in stored but-not-indexed fields

2010-10-30 Thread Paul Libbrecht
I am quite interested by this story, including sample code. Back in Lucene 1.4 and 2.0 times, the reader vs string loading abilities was inconsistently handled and I switched to have one directory with thousands of files for our ActiveMath content storage. It works but fairly badly on smaller ma

solr 4.0 - pagination

2010-10-30 Thread Papp Richard
Hi all, I'm using Solr 4.0 with grouping (field collapsing), but unfortunately I can't solve the pagination. Mainly there are two problems: - the query fields "start" & "rows" doesn't work anymore - beside of the values, it always returns the data as the start would be 0 (sta

Re: solr 4.0 - pagination

2010-10-30 Thread Yonik Seeley
On Sat, Oct 30, 2010 at 12:22 PM, Papp Richard wrote: >  I'm using Solr 4.0 with grouping (field collapsing), but unfortunately I > can't solve the pagination. It's not implemented yet, but I'm working on that right now. -Yonik http://www.lucidimagination.com

RE: solr 4.0 - pagination

2010-10-30 Thread Papp Richard
Can you estimate please when it will be done? thanks, Rich -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Saturday, October 30, 2010 19:29 To: solr-user@lucene.apache.org Subject: Re: solr 4.0 - pagination On Sat, Oct 30, 2010 at

Re: solr 1.4 suggester component

2010-10-30 Thread Erick Erickson
Sorry it took so long to respond, I got wrapped up in work. I don't think you're going to be able to pull this off (but of course I've been wrong before). File based suggestions look like they do an awful lot of stuff for you, at some expense of customization (no surprise here). That said, though,

Basic Document Question

2010-10-30 Thread Eric Martin
HI everyone, I'm new which won't be hard to figure out after I ask this question: I use Drupal/Solr/Nutch http://svn.apache.org/viewvc/lucene/dev/trunk/solr/example/solr/conf/schema. xml?view=markup Solr specific: How do I re-index for specific content only? I am starting a legal

Re: Modelling Access Control

2010-10-30 Thread Erick Erickson
If that's in response to Lance's comment, the answer is that if you return autosuggest possibilities you effectively allow users to see data they shouldn't. Imagine you have a field of the real names of spies. You only want the persons way high up in the security chain to access these names and you

Re: Highlighting for non-stored fields

2010-10-30 Thread Erick Erickson
Also, consider what you'd be reconstructing from if you could try it. The indexed data has been transformed by, say, stemming, casing, etc. So any attempt to reconstruct the fields for highlighting would necessarily show the transformed version, which would not be pleasing. Plus you could have syno

Re: Strange search

2010-10-30 Thread Erick Erickson
What does the admin page say is in your text field (see solr/admin, the "schema browser" link Also, see what returns if you add &debugQuery=on to your URL, that often is useful... HTH Erick On Wed, Oct 27, 2010 at 3:53 AM, ramzesua wrote: > > Can anyone give me working schema.xml and solrc

Re: question about SolrCore

2010-10-30 Thread Erick Erickson
Why do you want to know? If there is a specific problem you're trying to solve, perhaps stating the problem itself will get you a better response. Best Erick On Thu, Oct 28, 2010 at 4:00 AM, Li Li wrote: > is there anyone could help me? > > 2010/10/11 Li Li : > > hi all, > >I want to know t

Re: solr stuck in writing to inexisting sockets

2010-10-30 Thread Erick Erickson
Are you saying that your Solr server is at times taking 5 minutes to complete? If so, I'd get to the bottom of that first off. My first guess would be you're either hitting memory issues and swapping horribly or..well, that would be my first guess. Best Erick On Thu, Oct 28, 2010 at 5:23 AM, Roxa

Re: No response from Solr on complex request (real issue explained)

2010-10-30 Thread Erick Erickson
This smells like a garbage collection issue. Here's an article from about a year ago by Mark Miller that might help: http://www.lucidimagination.com/blog/2009/09/19/java-garbage-collection-boot-camp-draft/ D

Re: Commit/Optimise question

2010-10-30 Thread Erick Erickson
What version of Solr are you using? About committing. I'd just let the solr defaults handle that. You configure this in the autocommit section of solrconfig.xml. I'm pretty sure this gets triggered even if you're using SolrJ. That said, it's probably wise to issue a commit after all your data is

Re: Consulting in Solr tuning, stop words, dictionary, etc

2010-10-30 Thread Erick Erickson
Well, that all depends on what you want. Here's a list of Solr consultants. http://wiki.apache.org/solr/Support HTH Erick On Thu, Oct 28, 2010 at 4:21 PM, Dennis Gearon wrote: > Speaking of jobs on this list . . . . > > How much does a good consultant fo

Re: Reverse range search

2010-10-30 Thread Erick Erickson
I wonder if function queries might help you here (be aware that these aren't my strong point though). See: http://wiki.apache.org/solr/FunctionQuery Best Erick On Thu, Oct 28, 2010 at 5:24 PM, kenf_nc wrote: > > Doing a range search is straightforward.

Re: QueryElevation Component is so slow

2010-10-30 Thread Lance Norskog
Now you got me interested. Always a bad thing ;) Looking at the QueryElevationComponent, I don't know enough to decide if it has algorithms that don't scale. It does something odd with sorting. It has a concurrent access path for each query, which should not be a problem. It has not changed much s

Re: Ensuring stable timestamp ordering

2010-10-30 Thread Erick Erickson
What are the actual values in your index? I'm wondering if they all get the same values somehow, perhaps due to the granularity of your dates? And (and I'm really grasping at straws here) your is causing enough delay to have time intervals be greater than your granularity. Unfortunately, that do

Re: Would it be nuts to store a bunch of large attachments (images, videos) in stored but-not-indexed fields

2010-10-30 Thread Lance Norskog
There is a "binary" field type for this problem. Trunk versions now do not have to base-64 encode but just store the bytes directly (I think). There is a quirk: Solr/Lucene field contents (the "stored" part) are stored in field order. So all of the fields are in order on the disk. So, when Lucene

Re: solr 4.0 - pagination

2010-10-30 Thread Erick Erickson
See: https://issues.apache.org/jira/browse/SOLR-2207 You can track the progress here, but it hasn't been committed to trunk yet. Your options are: 1> get a recent trunk source tree and apply the patch and compile. 2> wait until it gets put in the tr

Re: Ensuring stable timestamp ordering

2010-10-30 Thread Lance Norskog
Hi- NOW does not get re-run for each document. If you give a large upload batch, the same NOW is given to each document. It would be handy to have an auto-incrementing date field, so that each document would get a unique number and the timestamp would then be the unique ID of the document. On Sa

Re: eDismax result differs from Dismax

2010-10-30 Thread Lance Norskog
Does the eDismax parser design guarantee that it will give the same results as the old dismax for a "legal" query? If not, are the differences minor? On Fri, Oct 29, 2010 at 6:40 AM, Yonik Seeley wrote: > On Fri, Oct 29, 2010 at 9:30 AM, Ryan Walker wrote: >> >> We are launching a new version of

Re: QueryElevation Component is so slow

2010-10-30 Thread Chamnap Chhorn
Well, I use Solr 1.4. There are 30698 lines in my elevation file. I need only 20 results response back at a time. On Sun, Oct 31, 2010 at 9:12 AM, Lance Norskog wrote: > Now you got me interested. Always a bad thing ;) > > Looking at the QueryElevationComponent, I don't know enough to decide >

Re: QueryElevation Component is so slow

2010-10-30 Thread Lance Norskog
How many items for each query? On Sat, Oct 30, 2010 at 7:34 PM, Chamnap Chhorn wrote: > Well, I use Solr 1.4. > > There are 30698 lines in my elevation file. I need only 20 results response > back at a time. > > On Sun, Oct 31, 2010 at 9:12 AM, Lance Norskog wrote: > >> Now you got me interested

Re: QueryElevation Component is so slow

2010-10-30 Thread Chamnap Chhorn
20 On Sun, Oct 31, 2010 at 9:44 AM, Lance Norskog wrote: > How many items for each query? > > On Sat, Oct 30, 2010 at 7:34 PM, Chamnap Chhorn > wrote: > > Well, I use Solr 1.4. > > > > There are 30698 lines in my elevation file. I need only 20 results > response > > back at a time. > > > > On S