Re: Custom filter development

2011-05-09 Thread Tom Hill
On Mon, May 9, 2011 at 5:07 AM, solrfan wrote: > Hi, I would like to write my own filter. I try to use the following class: > But this is a problem for me. The one-to-one mapping. I want to map a given > Token, for example "a" to three Tokens "a1", "a2", "a3". I want to do a > one-to-one mapping t

Re: Is it possible to assign default value for a particular record when using multivalued field type?

2010-12-10 Thread Tom Hill
Could you give us a bit more information? How are you getting this information into Solr? SolrJ? DataImportHandler? It's hard to see where the null value is getting dropped, if we don't know the path that it is coming in. I suspect that the default attribute won't do it. It's possible that you mi

Re: command line parameters for solr

2010-12-10 Thread Tom Hill
java -jar start.jar --help More docs here http://docs.codehaus.org/display/JETTY/A+look+at+the+start.jar+mechanism Personally, I usually limit access to localhost by using whatever firewall the machine uses. Tom On Fri, Dec 10, 2010 at 7:55 PM, Jack O wrote: > Hello, > > For starting solr, fr

Re: singular/plurals

2010-12-10 Thread Tom Hill
Check out this page: http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters Look, in particular, for "stemming". On Fri, Dec 10, 2010 at 7:58 PM, Jack O wrote: > Hello, > > Need one more help: > > What do I have to do so that search will work for singulars and plurals ? > > > > I would real

Re: OutOfMemory GC: GC overhead limit exceeded - Why isn't WeakHashMap getting collected?

2010-12-10 Thread Tom Hill
Hi John, WeakReferences allow things to get GC'd, if there are no other references to the object referred to. My understanding is that WeakHashMaps use weak references for the Keys in the HashMap. What this means is that the keys in HashMap can be GC'd, once there are no other references to the

Re: Triggering a reload of replicated configuration files

2010-12-09 Thread Tom Hill
On Thu, Dec 9, 2010 at 4:49 AM, Ophir Adiv wrote: > On Thu, Dec 9, 2010 at 2:25 PM, Upayavira wrote: >> >> >> On Thu, 09 Dec 2010 13:34 +0200, "Ophir Adiv" wrote: >>> Hi, >>> >>> I added a configuration file which is updated on one of the master >>> cores' conf directory, and also added the file

Re: Delete by query or Id very slow

2010-12-09 Thread Tom Hill
e); >                server.setMaxRetries(1); // defaults to 0. > 1 not > recommended. > >            } catch (MalformedURLException mex) { >                throw new SolrCustomException("Cannot resolve Solr Server at > '" + url + "'\n", mex); >  

Re: Delete by query or Id very slow

2010-12-08 Thread Tom Hill
That''s a pretty low number of documents for auto complete. It means that when getting to 850,000 documents, you will create 8500 segments, and that's not counting merges. How big are your documents? I just created an 850,000 document (and a 3.5 m doc index) with tiny documents (id and title), and

Re: How badly does NTFS file fragmentation impact search performance? 1.1X? 10X? 100X?

2010-12-08 Thread Tom Hill
If you can benchmark before and after, please post the results when you are done! Things like your index's size, and the amount of RAM in your computer will help make it meaningful. If all of your index can be cached, I don't think fragmentation is going matter much, once you get warmed up. Tom

Re: Index version on slave nodes

2010-12-07 Thread Tom Hill
Just off the top of my head, aren't you able to use a slave as a repeater, so it's configured as both a master and a slave? http://wiki.apache.org/solr/SolrReplication#Setting_up_a_Repeater This would seem to require that the slave return the same values as its master for indexversion. What happe

Re: complex boolean filtering in fq queries

2010-12-07 Thread Tom Hill
For one thing, you wouldn't have fq= in there, except at the beginning. fq=location:national OR (location:CA AND city:"San Francisco") more below... On Tue, Dec 7, 2010 at 10:25 PM, Andy wrote: > Forgot to add, my defaultOperator is "AND". > > --- On Wed, 12/8/10, Andy wrote: > >> From: Andy

Re: customer ping response

2010-12-07 Thread Tom Hill
Hi Tri, Well, I wouldn't really recommend this, but I just tried making a custom XMLReponseWriter that wrote the response you wanted. So you can use it with any request handler you want. Works fine, but it's pretty hack-y. The downside is, you are writing code, and you have to modify SolrCore. Bu

Re: only index synonyms

2010-12-07 Thread Tom Hill
rocess anyway >> >> Best >> Erick >> >> On Tue, Dec 7, 2010 at 6:07 AM, lee carroll > >wrote: >> >> > Hi tom >> > >> > This seems to place in the index >> > This is a scenic line of words >> > I just wa

Re: only index synonyms

2010-12-06 Thread Tom Hill
Hi Lee, On Mon, Dec 6, 2010 at 10:56 PM, lee carroll wrote: > Hi Erik Nope, Erik is the other one. :-) > thanks for the reply. I only want the synonyms to be in the index > how can I achieve that ? Sorry probably missing something obvious in the > docs Exactly what he said, use the => syntax.

Re: Need help with spellcheck city name

2010-09-27 Thread Tom Hill
Maybe process the city name as a single token? On Mon, Sep 27, 2010 at 3:25 PM, Savannah Beckett wrote: > Hi, >   I have city name as a text field, and I want to do spellcheck on it.  I use > setting in http://wiki.apache.org/solr/SpellCheckComponent > > If I setup city name as text field and do

Re: Delete Dynamic Fields

2010-09-22 Thread Tom Hill
Delete all docs with the dynamic fields, and then optimize. On Wed, Sep 22, 2010 at 1:58 PM, Moiz Bhukhiya wrote: > Hi All: > > I had used dynamic fields for some of my fields and then later decided to > make it static. I removed that dynamic field from the schema but I still see > it on admin in

Re: Searching solr with a two word query

2010-09-20 Thread Tom Hill
It will probably be clearer if you don't use the pseudo-boolean operators, and just use + for required terms. If you look at your output from debug, you see your query becomes:     all_text:open +all_text:excel +presentation_id:294 +type:blob Note that "all_text:open" does not have a + sign, but

Re: Odd query result

2010-04-20 Thread Tom Hill
I agree that, if they are the same, you want to merge them. In this case, I don't think you want them to be the same. In particular, you usually don't want to catenateWords and catenateNumbers both index time AND at query time. You generate the permutations on one, or the other, but you don't need

Re: Odd query result

2010-04-20 Thread Tom Hill
When I run it, with that fieldType, it seems to work for me. Here's a sample query output 0 17 on 0 xtext:I-Car 2.2 10 ALLCAPS I-CAR CAMEL I-Car Did I miss something? Could you show the output with debugQuery=on for the user's failing query? Assuming I

Re: Problem with suggest search

2010-03-15 Thread Tom Hill
You need a query string with the standard request handler. (dismax has q.alt) Try q=*:*, if you are trying to get facets for all documents. And yes, a friendlier error message would be a good thing. Tom On Mon, Mar 15, 2010 at 9:03 AM, David Rühr wrote: > Hi List. > > We have two Servers dev

Re: java.lang.OutOfMemoryError, VM may need to be forcibly terminated

2010-03-12 Thread Tom Hill
Hi - The best way is probably to add more ram. :-) That error apparently results from running out of perm gen space, and with 512m, you may not have much perm gen space. Options for increasing this can be found http://java.sun.com/javase/technologies/hotspot/vmoptions.jsp But, if you don't have

Re: Warning : no lockType configured for...

2010-03-02 Thread Tom Hill.
Hi Mani, Mani EZZAT wrote: > I'm dynamically creating cores with a new index, using the same schema > and solrconfig.xml Does the problem occur if you use the same configuration in a single, static core? Tom -- View this message in context: http://old.nabble.com/Re%3A-Warning-%3A-no-lockTyp

Re: Plurals in solr indexing

2010-01-27 Thread Tom Hill
I recommend getting familiar with the analysis tool included with solr. From Solr's main admin screen, click on "analysis", Check verbose, and enter your text, and you can see the changes that happen during analysis. It's really helpful, especially when getting started. Tom On Wed, Jan 27, 2010

Re: Multiple Cores Vs. Single Core for the following use case

2010-01-27 Thread Tom Hill
Hi - I'd probably go with a single core on this one, just for ease of operations. But here are some thoughts: One advantage I can see to multiple cores, though, would be better idf calculations. With individual cores, each user only sees the idf for his own documents. With a single core, the idf

Re: Improvising solr queries

2010-01-04 Thread Tom Hill
Hi - Something doesn't make sense to me here: On Mon, Jan 4, 2010 at 5:55 AM, dipti khullar wrote: > - optimize runs on master in every 7 minutes > - using postOptimize , we execute snapshooter on master > - snappuller/snapinstaller on 2 slaves runs after every 10 minutes > > Why would you optim

Re: Case Insensitive search not working

2009-12-08 Thread Tom Hill
Did you rebuild the index? Changing the analyzer for the index doesn't affect already indexed documents. Tom On Tue, Dec 8, 2009 at 11:57 AM, insaneyogi3008 wrote: > > Hello, > > I tried to force case insensitive search by having the following setting in > my schema.xml file which I guess is st

Re: why no results?

2009-12-07 Thread Tom Hill
k Brown Fox", because it's not the whole field. The ID field probably works because it has one term in it. "1" finds "1" just fine. Try solr.TextField instead. Tom On Mon, Dec 7, 2009 at 7:47 PM, Tom Hill wrote: > Hi - > > That's a common one to

Re: why no results?

2009-12-07 Thread Tom Hill
Hi - That's a common one to get bit by. The string On Mon, Dec 7, 2009 at 7:44 PM, regany wrote: > > hi all - newbie solr question - I've indexed some documents and can search > / > receive results using the following schema - BUT ONLY when searching on the > "id" field. If I try searching on t

Re: deleteById without solrj?

2009-12-03 Thread Tom Hill
http://wiki.apache.org/solr/UpdateXmlMessages#A.22delete.22_by_ID_and_by_Query On Thu, Dec 3, 2009 at 11:57 AM, Joel Nylund wrote: > Is there a url based approach to delete a document? > > thanks > Joel > >

Re: Multi-Term Synonyms

2009-11-24 Thread Tom Hill
Hi Brad, I suspect that this section from the wiki for SynonymFilterFactory might be relevant: http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.SynonymFilterFactory *"Keep in mind that while the SynonymFilter will happily work with synonyms containing multiple words (ie: "**sea

Webinar: An Introduction to Basics of Search and Relevancy with Apache Solr hosted by Lucid Imagination

2009-11-23 Thread Tom Hill
In this introductory technical presentation, renowned search expert Mark Bennett, CTO of Search Consultancy New Idea Engineering, will present practical tips and examples to help you quickly get productive with Solr, including: * Working with the "web command line" and controlling your inputs and

Re: Talk on Solr - Oakland, CA June 18, 2008

2008-06-21 Thread Tom Hill - Solr
Thanks Brad! Hoss, I've added the link to the wiki. I've got the presentation on slideshare.net working. http://www.slideshare.net/tomhill/an-introduction-to-solr I don't see my comments when I view the presentation online, they are in the .ppt you can download, and they add some more content.

Talk on Solr - Oakland, CA June 18, 2008

2008-06-17 Thread Tom Hill - Solr
Hi - I'll be giving a talk on Solr at the East Bay Innovations Group (eBig) Java SIG on Wed, June 18. http://www.ebig.org/index.cfm?fuseaction=Calendar.eventDetail&eventID=16 This is an introductory / overview talk intended to get you from "What is Solr & Why Would I Use It" to "Cool, now I kno

Re: Solr + Tomcat Undeploy Leaks

2007-10-18 Thread Tom Hill
I certainly have seen memory problems when I just drop a new war file in place. So now I usually stop tomcat and restart. I used to see problems (pre-1.0) when I just redeployed repeatedly, without even accessing the app, but I've got a little script running in the background that has done that 50

Re: Availability Issues

2007-10-08 Thread Tom Hill
Hi - We're definitely not seeing that. What do your logs show? What do your schema/solrconfig look like? Tom On 10/8/07, David Whalen <[EMAIL PROTECTED]> wrote: > > Hi All. > > I'm seeing all these threads about availability and I'm > wondering why my situation is so different than others'. > >

Re: Solr live at Netflix

2007-10-02 Thread Tom Hill
Nice! And there seem to be some improvements. For example, "Gamers" and "Gamera" no longer stem to the same word :-) Tom On 10/2/07, Walter Underwood <[EMAIL PROTECTED]> wrote: > > Here at Netflix, we switched over our site search to Solr two weeks ago. > We've seen zero problems with the server

Re: pluggable functions

2007-09-18 Thread Tom Hill
Hi - I'm not sure what you mean by a reflection based approach, but I've been thinking about doing this for a bit, since we needed it, too. I'd just thought about listing class names in the config file. The functions would probably need to extend a subclass of ValueSource which will handle argume

Re: Slow response

2007-09-14 Thread Tom Hill
Hi Mike, Thanks for clarifying what has been a bit of a black box to me. A couple of questions, to increase my understanding, if you don't mind. If I am only using fields with multiValued="false", with a type of "string" or "integer" (untokenized), does solr automatically use approach 2? Or is

Re: Query for German "Special Characters" (i.e., ä, ö, ß)

2007-09-14 Thread Tom Hill
uot;this" and "is" work fine, whereas > >> queries to "really" and "fünny" do not return the result. Fünnily ;-) , > >> after extending my sometext to "This is really fünny kraßen.", queries > >> to "really" and "fünn

Re: Query for German "Special Characters" (i.e., ä, ö, ß)

2007-09-14 Thread Tom Hill
not work, but "kraßen" is found. > Now I am somehow confused -- hopefully anyone has a good explanation ;-) > > Regards, > > marc > > > Tom Hill schrieb: > >> If you are using tomcat, try adding "URIEncoding="UTF-8" to your > >> tomc

Re: Query for German "Special Characters" (i.e., ä, ö, ß)

2007-09-13 Thread Tom Hill
If you are using tomcat, try adding "URIEncoding="UTF-8" to your tomcat connector. use the analysis page of the admin interface to check to see what's happening to your queries, too. http://localhost:8080/solr/admin/analysis.jsp?highlight=on (your port # may vary) Tom On 9/13/07, Marc Be

Re: solr/home

2007-09-06 Thread Tom Hill
It works for me. (fragments with solr 1.2 on tomcat 5.5.20) Could you post your fragment file? Tom On 9/6/07, Matt Mitchell <[EMAIL PROTECTED]> wrote: > Hi, > > I recently upgraded to Solr 1.2. I've set it up through Tomcat using > context fragment files. I deploy using the tomcat web manager.

Re: update servlet not working

2007-09-06 Thread Tom Hill
I don't use the java client, but when I switched to 1.2, I'd get that message when I forget to add the content type header, as described in CHANGES.txt > 9. The example solrconfig.xml maps /update to XmlUpdateRequestHandler using the new request dispatcher (SOLR-104). This requires posted co

Re: Facet for multiple values field

2007-08-30 Thread Tom Hill
Hi - I wouldn't facet on a "text" field, I tend to use "string" for the reasons you describe. e.g. Use or in your example If I have multiple values, I add them as separate occurrences of the field I am faceting on. If you still need them all in one field for other reasons, use copyField

Re: How to realize index spaces

2007-08-23 Thread Tom Hill
Hi - On 8/23/07, Marc Bechler <[EMAIL PROTECTED]> wrote: > > I was wondering whether or not it is possible to realize different index > spaces with one solr instance. > > Example: imagine, you want to have 2 index spaces that coexist > independently (and wich can be identified, e.g., by a unique i

Synonym questions

2007-08-09 Thread Tom Hill
Hi - Just looking at synonyms, and had a couple of questions. 1) For some of my synonyms, it seems to make senses to simply replace the original word with the other (e.g. "theatre" => "theater", so searches for either will find either). For others, I want to add an alternate term while preserving

Returning errors from request handler

2007-07-26 Thread Tom Hill
Hi - With solr 1.2, when using XmlUpdateRequestHandler , if I post a valid command like "" I get a response like 00 Nice, valid xml. But If I have an error (for example, ) I get an HTML page back. This tends to confuse the client software. Is there a way to get a return like: 1blah, blah

Using request parameters in dismax boost functions

2007-06-04 Thread Tom Hill
Hi - Perhaps I'm missing something obvious, but it there a way to get values from the user's request as arguments to boost functions in dismax? I'm thinking about distance based weighting for search results, which requires the user's x,y. Tom

Optimizing frequently updated index

2007-05-29 Thread Tom Hill
Hi - I have an index that is updated fairly frequently (every few seconds), and I'm replicating to several slave servers. Because of the frequent updates, I'm usually pushing an index that is not optimized. And, as it takes several minutes to optimize, I don't want to do it every time I replicat

Re: takes an hour

2007-05-18 Thread Tom Hill
Hi - What happens if updates occur during the optimize? Thanks, Tom

Re: Index corruptions?

2007-05-07 Thread Tom Hill
Hi Charlie, On 5/3/07, Charlie Jackson <[EMAIL PROTECTED]> wrote: I have a couple of questions regarding index corruptions. 1) Has anyone using Solr in a production environment ever experienced an index corruption? If so, how frequently do they occur? I once had all slaves complain about a

Re: Group results by field?

2007-05-02 Thread Tom Hill
Hi Matthew, You might be able to just get away with just using facets, depending on whether your goal is to provide a clickable list of styles_ids to the user, or if you want to only return one search result for each style_id. For a list of clickable styles, it's basic faceting, and works really

Re: browse a facet without a query?

2007-04-23 Thread Tom Hill
Hi - On 4/23/07, Yonik Seeley <[EMAIL PROTECTED]> wrote: On 4/23/07, Jennifer Seaman <[EMAIL PROTECTED]> wrote: > When there is no "q" Solr complains. How can I browse a facet without > a keyword query? For example, I want to view all document for a given state; > > ?q=&fq=state:California Wit

Re: Embedding Solr vs Lucene, multiple Solr cores?

2007-04-13 Thread Tom Hill
Hi - Of the various approaches that you could take, the one I'd work on first is: deployment constraints imply one webapp instance. In most environments, it's going to cost a lot less to change this, than to try to roll your own, or extensively modify solr. I know I'm sidestepping your state