Re: how to retrieve all the documents in an index?
Hi Hui, I'm not 100% certain but I believe this syntax was added in 1.2 (it certainly works in the svn trunk code), can anyone confirm this? cheers, Piete On 14/08/07, Yu-Hui Jin <[EMAIL PROTECTED]> wrote: > > Piete, > > I tried and it doesn't work for Solr 1.1. Is it supported for 1.2 or at > all? > > (Right now, I'm using a work-around by a range query for a field whose > range > is known to be larger than 0.) > > > Thanks, > > -Hui > > > > On 8/12/07, Pieter Berkel <[EMAIL PROTECTED]> wrote: > > > > Try using q=*:* to match all documents in the index. > > > > Piete > > > > > > > > On 13/08/07, Yu-Hui Jin <[EMAIL PROTECTED]> wrote: > > > > > > Hi, there, > > > > > > I found the following post on the web. Is this still the simplest > > > get-around > > > to retrieve all documents in an index? (I'm asking just in case I > don't > > > know > > > there's a more standard way to do that now.) > > > > > > > > > thanks, > > > > > > -Hui > > > > > > > > > > > > From "Fuad Efendi" < [EMAIL PROTECTED]> > > > Subject RE: MatchAllDocsQuery in solr? > > > Date Wed, 29 Nov 2006 01:58:25 GMT > > > > > > Workaround > > > == > > > > > > Define a field abcd with constant > value > > > 'abcd' for all documents (choose value not listed in any 'stop-word' > > > etc.). > > > Lucene query 'scan_all:abcd' will retrieve 'all' documents. > > > Enjoy! > > > > > > > > > -Original Message- > > > From: Tom > > > Sent: Tuesday, November 21, 2006 5:08 PM > > > To: solr-user@lucene.apache.org > > > Subject: MatchAllDocsQuery in solr? > > > > > > > > > Is there a way to do a match all docs query in solr? > > > > > > I mean is there something I can put in a solr URL that will get > > > recognized by the SolrQueryParser as meaning a "match all"? > > > > > > Why? Because I'm porting unit tests from our internal Lucene > > > container to Solr, and the tests usually run such a query, upon > > > completion, to make sure the index is in the expected state (nothing > > > missing, nothing extra). > > > > > > Yes, I can create a query that will match all my docs, there are a > > > few fields that have a relatively small range of values. I was just > > > looking for a standard way to do it first. > > > > > > Thanks, > > > > > > Tom > > > > > > > > > -- > Regards, > > -Hui >
Re: how to retrieve all the documents in an index?
On 8/15/07, Pieter Berkel <[EMAIL PROTECTED]> wrote: > I'm not 100% certain but I believe this syntax was added in 1.2 (it > certainly works in the svn trunk code), can anyone confirm this? Yes, It was added to Lucene 2.1 (which Solr 1.2 uses) http://svn.apache.org/viewvc/lucene/java/trunk/CHANGES.txt?view=markup -Yonik
RE: solr doesn't find exe in postCommit event
Adding '.' to the PATH didn't work for me. I tried it many different combinations -- in my .bashrc, in the script which starts my tomcat and/or setting " PATH=.:/bin:/usr/bin:/usr/local/bin ". None of the following worked for me... In my scripts -- export PATH=.:$JAVA_HOME/bin:/usr/bin:/bin Different listener configurations -- snapshooter . true arg1 arg2 MYVAR=val1 snapshooter solr/bin true arg1 arg2 MYVAR=val1 snapshooter solr/bin true arg1 arg2 PATH=.:/bin:/usr/bin:/usr/local/bin This was run on Linux 2.6.9-34.0.2.Elsmp, my login shell is bash 3.00.15(1). The solr version I am using is apache-solr-1.2.0. Daniel -Original Message- From: Chris Hostetter [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 14, 2007 7:19 PM To: solr-user@lucene.apache.org Subject: Re: solr doesn't find exe in postCommit event : > Is it possible to update the example config file enclosed in the : > distribution as it is not working. : : done. just to be clear, the old example did in fact work *if* you had "." in your PATH ... (a bad assumption true, but i just want to make sure people understand why it worked for some people and not for others) -Hoss
RE: Solrsharp highlighting
Thanks for adding in those facet examples. That should help me out a great deal. As for the highlighting, did you have any ideas about a good way to go about it? I was thinking about taking a stab at it, but I want to get your input first. Thanks, Charlie -Original Message- From: Jeff Rodenburg [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 14, 2007 1:08 AM To: solr-user@lucene.apache.org Subject: Re: Solrsharp highlighting Pull down the latest example code from http://solrstuff.org/svn/solrsharpwhich includes adding facets to search results. It's really short and simple to add facets; the example application implements one form of it. The nice thing about the facet support is that it utilizes generics to allow you to have strongly typed name/value pairs for the fieldname/count data. Hope this helps. -- jeff r. On 8/10/07, Charlie Jackson <[EMAIL PROTECTED]> wrote: > > Also, are there any examples out there of how to use Solrsharp's > faceting capabilities? > > > Charlie Jackson > 312-873-6537 > [EMAIL PROTECTED] > -Original Message- > From: Charlie Jackson [mailto:[EMAIL PROTECTED] > Sent: Friday, August 10, 2007 3:51 PM > To: solr-user@lucene.apache.org > Subject: Solrsharp highlighting > > Trying to use Solrsharp (which is a great tool, BTW) to get some results > in a C# application. I see the HighlightFields method of the > QueryBuilder object and I've set it to my highlight field, but how do I > get at the results? I don't see anything in the SearchResults code that > does anything with the highlight results XML. Did I miss something? > > > > > > Thanks, > > Charlie > >
Re: how to retrieve all the documents in an index?
Great. Thanks, guys. That's how what I thought. Any of you know whether the new "q:*.*" query performs better than the get-around solutions like using a ranged query? I would guess so, but I haven't looked into the Lucene implementation. regards, -Hui On 8/15/07, Yonik Seeley <[EMAIL PROTECTED]> wrote: > > On 8/15/07, Pieter Berkel <[EMAIL PROTECTED]> wrote: > > I'm not 100% certain but I believe this syntax was added in 1.2 (it > > certainly works in the svn trunk code), can anyone confirm this? > > Yes, It was added to Lucene 2.1 (which Solr 1.2 uses) > http://svn.apache.org/viewvc/lucene/java/trunk/CHANGES.txt?view=markup > > -Yonik > -- Regards, -Hui
Query optimisation - multiple filter caches?
I'm trying to understand how best to integrate directly with Solr (Java-to-Java in the same JVM) to make the most of its query optimisation - chiefly, its caching of queries which merely filter rather than rank results. I notice that SolrIndexSearcher maintains a filter cache and so does LuceneQueryOptimiser. Shouldn't they be contributing to/using the same cache, or are they used for different things? Jon
Indexing speed: web v.s. solrj app
Is indexing via solrj faster than going through the web service? There are three cases: Read a file from a local file system and indexing it directly, Read a file on one machine and indexing it on another, and Run solrj and read a file, then directly update the index. I'm talking about the last case v.s. the middle case: we are already feeding solr from remote machines. Thanks, Lance
Overall performance: network v.s. SAN file system
Is anyone doing Solr installations with a SAN file system? Like IBM Storage Tank or Apple XSAN or Red Hat GFS? What are your experiences? Thanks, Lance
Re: Indexing speed: web v.s. solrj app
On 8/15/07, Lance Norskog <[EMAIL PROTECTED]> wrote: > Is indexing via solrj faster than going through the web service? There are > three cases: > Read a file from a local file system and indexing it directly, > Read a file on one machine and indexing it on another, and > Run solrj and read a file, then directly update the index. > > I'm talking about the last case v.s. the middle case: we are already feeding > solr from remote machines. solrj is (in my opinion) primarily as a convenience client for going through the web service. If you mean using solrj locally, I would guess that giving the CSV handler a local file (or perhaps even streaming remotely) might be a tad faster. If done correctly, the differences can be small enough that architectural decisions shouldn't be based on those differences. -Yonik
Re: [slightly ot] Looking for Lucene/Solr consultant in Germany
Dear Jan, I just saw your post on the SOLR mailing list. I hope I'm not too late. First of, I don't exactly match your required qualifications. I do have 9 years at Verity and 1 year at Autonomy in enterprise search, however. I'm in the middle of coming up to speed on SOLR and applying my considerable expertise in general Enterprise Search to the SOLR/Lucene platform. So, your specific requirements for a Lucene/SOLR expert are not quite met. But, I've been in the business of enterprise search for 10 years. Think if it as asking an Oracle expert to look at your MySQL implementation. My normal rate is USD 200/hour, and I do command that rate more often than not. I'd be interested in taking on the challenge in my spare time, free of charge, just to get my bearings and to see how my consulting skills translate from the closed-source Verity/IDOL world to the open source world. I think this could be beneficial to both of us: I would get some expertise in specific SOLR idiosyncrasies, and you would get the benefit of 10 years of general enterprise search experience. I've been studying SOLR and Lucene, and even developing my own project using them as a basis. That being said, I expect to make some mistakes as I try to match my existing skill set with what's available in SOLR. Fortunately, I found that with the transition from Verity K2 to Autonomy IDOL the underlying concepts of full-text search are pretty much universal. Another fly in the ointment is that I live in the USA (St. Pete Beach, Florida to be exact), so there would be some time zone issues. Also, I don't speak German, which will be a handicap when it comes to analyzing stemming options. If you can live with those limitations, I'd be happy to help. Let me know if you're interested. George Everitt Applied Relevance LLC [EMAIL PROTECTED] Tel: +1 (727) 641-4660 Fax: +1 (727) 233-0672 On Aug 8, 2007, at 12:43 PM, Jan Miczaika wrote: Hello, we are looking for a Lucene/Solr consultant in Germany. We have set up a Lucene/Solr server (currently live at http://www.hitflip.de). It returns search results, but the results are not really very good. We have been tweaking the parameters a bit, following suggestions from the mailing list, but are unsure of the effects this has. We are looking for someone to do the following: - analyse the search patterns on our website - define a methodology for defining the quality of search - analyse the data we have available - specify which data is required in the index - modify the search patterns used to query the data - test and evaluate the results The requirements: deep knowledge of Lucene/Solr, examples of implemented working search engines, theoretical knowledge Is anyone interested? Please feel free to circulate this offer. Thanks in advance Jan -- Geschäftsführer / Managing Director Hitflip Media Trading GmbH Gürzenichstr. 7, 50667 Köln http://www.hitflip.de - new: http://www.hitflip.co.uk Tel. +49-(0)221-272407-27 Fax. 0221-272407-22 (that's so 1990s) HRB 59046, Amtsgericht Köln Geschäftsführer: Andre Alpar, Jan Miczaika, Gerald Schönbucher
Re: Query optimisation - multiple filter caches?
On 8/15/07, Jonathan Woods <[EMAIL PROTECTED]> wrote: > I'm trying to understand how best to integrate directly with Solr > (Java-to-Java in the same JVM) to make the most of its query optimisation - > chiefly, its caching of queries which merely filter rather than rank > results. > > I notice that SolrIndexSearcher maintains a filter cache and so does > LuceneQueryOptimiser. Shouldn't they be contributing to/using the same > cache, or are they used for different things? LuceneQueryOptimiser is no longer used since one can directly specify filters via fq parameters. -Yonik
Solr, Lucene and patents
Does anyone know what the patent situation is with Lucene and Solr? What patents affect it, what you can and cannot do with it? Thanks, Lance
Re: Solrsharp highlighting
I've been working on the highlighting component, and it's a little odd how it works. For myself, if I want terms highlighted, I'd like those in the return results. Solr, on the other hand, returns a separate xml node that represents the portions of the results that are highlighted. I know that it's incorporated that way for other reasons, but it makes patching the highlighted portions together with the doc results in Solrsharp an out-of-band experience. Nonetheless, the approach I'm trying is one where the highlighted nodes are associated with the SearchResults object, and will have their highlighted text bits incorporated into the associated SearchRecord objects. At least that's what I'm initially trying to accomplish. -- j On 8/15/07, Charlie Jackson <[EMAIL PROTECTED]> wrote: > > Thanks for adding in those facet examples. That should help me out a > great deal. > > As for the highlighting, did you have any ideas about a good way to go > about it? I was thinking about taking a stab at it, but I want to get > your input first. > > > Thanks, > Charlie > > > -Original Message- > From: Jeff Rodenburg [mailto:[EMAIL PROTECTED] > Sent: Tuesday, August 14, 2007 1:08 AM > To: solr-user@lucene.apache.org > Subject: Re: Solrsharp highlighting > > Pull down the latest example code from > http://solrstuff.org/svn/solrsharpwhich includes adding facets to > search results. It's really short and > simple to add facets; the example application implements one form of it. > The nice thing about the facet support is that it utilizes generics to > allow > you to have strongly typed name/value pairs for the fieldname/count > data. > > Hope this helps. > > -- jeff r. > > On 8/10/07, Charlie Jackson <[EMAIL PROTECTED]> wrote: > > > > Also, are there any examples out there of how to use Solrsharp's > > faceting capabilities? > > > > > > Charlie Jackson > > 312-873-6537 > > [EMAIL PROTECTED] > > -Original Message- > > From: Charlie Jackson [mailto:[EMAIL PROTECTED] > > Sent: Friday, August 10, 2007 3:51 PM > > To: solr-user@lucene.apache.org > > Subject: Solrsharp highlighting > > > > Trying to use Solrsharp (which is a great tool, BTW) to get some > results > > in a C# application. I see the HighlightFields method of the > > QueryBuilder object and I've set it to my highlight field, but how do > I > > get at the results? I don't see anything in the SearchResults code > that > > does anything with the highlight results XML. Did I miss something? > > > > > > > > > > > > Thanks, > > > > Charlie > > > > >
