Some guide about setting up local/geo search at solr
Hey, I spent some times figuring out how to set up local/geo/spatial search at solr. I hope the following description can help given the current status. 1) Download localsolr. I download it from http://developer.k-int.com/m2snapshots/localsolr/localsolr/1.5/ and put jar file (in my case, localsolr-1.5.jar) in your application's WEB_INF/lib directory of application server. 2) Download locallucene. I download it from http://sourceforge.net/projects/locallucene/ and put jar file (in my case, locallucene.jar in locallucene_r2.0/dist/ diectory) in your application's WEB_INF/lib directory of application server. I also need to copy gt2-referencing-2.3.1.jar, geoapi-nogenerics-2.1-M2.jar, and jsr108-0.01.jar under locallucene_r2.0/lib/ directory to WEB_INF/lib. Do not copy lucene-spatial-2.9.1.jar under Lucene codebase. The namespace has been changed from com.pjaol.blah.blah.blah to org.apache.blah blah. 3) Update your solrconfig.xml and schema.xml. I copy it from http://www.gissearch.com/localsolr. 4) Restart application server and try a query /solr/select?&qt=geo&lat=xx.xx&long=yy.yy&q=abc&radius=zz.
Solr date and string search problem
Hi , I have been using solr1.2 for a year and now i m facing a weird problem. Till now i have used only string and number solr types for the search field . and whatever string the users are trying to search will pass it on to the search engine and it will find in appropraite fields and return me the results. But now i have added a another field with type DATE and made it as a search field. so what happens is whatever string that i m giving to solr tries to convert to date and throws me an error. Nov 14, 2009 4:36:05 PM org.apache.solr.core.SolrException log SEVERE: org.apache.solr.core.SolrException: Invalid Date String:'Behavior' at org.apache.solr.schema.DateField.toInternal(DateField.java:108) at org.apache.solr.schema.FieldType$DefaultAnalyzer$1.next(FieldType.java:298) at org.apache.lucene.queryParser.QueryParser.getFieldQuery(QueryParser.java:437) at org.apache.solr.search.SolrQueryParser.getFieldQuery(SolrQueryParser.java:78) at org.apache.solr.util.SolrPluginUtils$DisjunctionMaxQueryParser.getFieldQuery(SolrPluginUtils.java:774) at org.apache.solr.util.SolrPluginUtils$DisjunctionMaxQueryParser.getFieldQuery(SolrPluginUtils.java:762) at org.apache.lucene.queryParser.QueryParser.Term(QueryParser.java:1092) at org.apache.lucene.queryParser.QueryParser.Clause(QueryParser.java:979) at org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:907) at org.apache.lucene.queryParser.QueryParser.TopLevelQuery(QueryParser.java:896) at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:146) at org.apache.solr.request.DisMaxRequestHandler.handleRequestBody(DisMaxRequestHandler.java:238) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:77) at org.apache.solr.core.SolrCore.execute(SolrCore.java:706) at com.tcs.mighty.cb.service.CoreEntitySearcherImpl.solrSearch(CoreEntitySearcherImpl.java:896) at com.tcs.mighty.cb.service.CoreEntitySearcherImpl.search(CoreEntitySearcherImpl.java:342) at com.tcs.mighty.cb.service.CoreEntitySearcherImpl.handleGetSearchResults(CoreEntitySearcherImpl.java:52) at com.tcs.mighty.cb.service.CoreEntitySearcherBase.getSearchResults(CoreEntitySearcherBase.java:122) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:287) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:181) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148) at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:170) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:170) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:176) at $Proxy141.getSearchResults(Unknown Source) at com.tcs.mighty.cb.service.ejb.CoreEntitySearcherBean.getSearchResults(CoreEntitySearcherBean.java:24) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.jboss.ejb.StatelessSessionContainer$ContainerInterceptor.invoke(StatelessSessionContainer.java:683) at org.jboss.resource.connectionmanager.CachedConnectionInterceptor.invoke(CachedConnectionInterceptor.java:186) at org.jboss.ejb.plugins.StatelessSessionInstanceInterceptor.invoke(StatelessSessionInstanceInterceptor.java:72) at org.jboss.ejb.plugins.AbstractTxInterceptor.invokeNext(AbstractTxInterceptor.java:84) at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:315) at org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:148) at org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:111) at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:191) at org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:122) at org.jboss.ejb.StatelessSessionContainer.internal
field collapse using 'adjacent' & 'includeCollapsedDocs' + 'sort' query field
Hi, This almost seems like a bug, but I can't be sure so I'm seeking confirmation. Basically I am building a site that presents search results in reverse chronologically order. I am also leveraging the field collapse feature so that I can group results using 'adjacent' mode and have solr return the collapsed results as well via 'includeCollapsedDocs'. My collapsing field is a custom grouping_id that I have specified. What I'm noticing is that, my search results are coming back in the correct order by descending time (via 'sort' param in the main query) as expected. However, the results returned within the 'collapsedDocs' section via 'includeCollapsedDocs' are not in the same descending time order. My question is, shouldn't the collapsedDocs results also be in the same 'sort' order and key I have specified in the overall query, particularly since 'adjacent' mode is enabled, and that would mean results that are 'adjacent' in the sort order of the results. I'm using Solr 1.4.0 + field collapse patch as of 10/27/2009 Thanks, Michael -- View this message in context: http://old.nabble.com/field-collapse-using-%27adjacent%27---%27includeCollapsedDocs%27-%2B-%27sort%27-query-field-tp26351840p26351840.html Sent from the Solr - User mailing list archive at Nabble.com.
converting over from sphinx
I've been using Sphinx for full text search, but since I want to move my project over to Heroku, need to switch to Solr. Everything's up and running using the acts_as_solr plugin, but I'm curious if I'm using Solr the right way. In particular, I'm doing phrase searching into a corpus of descriptions, such as "I need help with a foo" where I have a bunch of "foo: a foo is a subset of a bar often used to create briznatzes", etc. With Sphinx, I could convert "I need help with a foo" into "*need* *help* *with* *foo*" and get pretty nice matches. With Solr, my understanding is that you can only do wildcard matches on the suffix. In addition, stemming only happens on non-wildcard terms. So, my first thought would be to convert "I need help with a foo" into "need need* help help* with with* foo foo*". Thanks in advance for any help. -- Cory Ondrejka cory.ondre...@gmail.com http://ondrejka.net/
Re: javabin in .NET?
Original code is here: http://bit.ly/hkCbI I just started porting it here: http://bit.ly/37hiOs It needs: tests/debugging, porting NamedList, SolrDocument, SolrDocumentList Thanks for any help! Cheers, Mauricio 2009/11/14 Noble Paul നോബിള് नोब्ळ् > OK. Is there anyone trying it out? where is this code ? I can try to help > .. > > On Fri, Nov 13, 2009 at 8:10 PM, Mauricio Scheffer > wrote: > > I meant the standard IO libraries. They are different enough that the > code > > has to be manually ported. There were some automated tools back when > > Microsoft introduced .Net, but IIRC they never really worked. > > > > Anyway it's not a big deal, it should be a straightforward job. Testing > it > > thoroughly cross-platform is another thing though. > > > > 2009/11/13 Noble Paul നോബിള് नोब्ळ् > > > >> The javabin format does not have many dependencies. it may have 3-4 > >> classes an that is it. > >> > >> On Fri, Nov 13, 2009 at 6:05 PM, Mauricio Scheffer > >> wrote: > >> > Nope. It has to be manually ported. Not so much because of the > language > >> > itself but because of differences in the libraries. > >> > > >> > > >> > 2009/11/13 Noble Paul നോബിള് नोब्ळ् > >> > > >> >> Is there any tool to directly port java to .Net? then we can etxract > >> >> out the client part of the javabin code and convert it. > >> >> > >> >> On Thu, Nov 12, 2009 at 9:56 PM, Erik Hatcher < > erik.hatc...@gmail.com> > >> >> wrote: > >> >> > Has anyone looked into using the javabin response format from .NET > >> >> (instead > >> >> > of SolrJ)? > >> >> > > >> >> > It's mainly a curiosity. > >> >> > > >> >> > How much better could performance/bandwidth/throughput be? How > >> difficult > >> >> > would it be to implement some .NET code (C#, I'd guess being the > best > >> >> > choice) to handle this response format? > >> >> > > >> >> > Thanks, > >> >> >Erik > >> >> > > >> >> > > >> >> > >> >> > >> >> > >> >> -- > >> >> - > >> >> Noble Paul | Principal Engineer| AOL | http://aol.com > >> >> > >> > > >> > >> > >> > >> -- > >> - > >> Noble Paul | Principal Engineer| AOL | http://aol.com > >> > > > > > > -- > - > Noble Paul | Principal Engineer| AOL | http://aol.com >
Re: Solr 1.3 query and index perf tank during optimize
Lance Norskog wrote on 11/13/2009 11:18:42 PM: > The 'maxSegments' feature is new with 1.4. I'm not sure that it will > cause any less disk I/O during optimize. It could still be useful to manage the "too many open files" problem that rears its ugly head on occasion. > The 'mergeFactor=2' idea is not what you think: in this case the index > is always "mostly optimized", so you never need to run optimize. > Indexing is always slower, because you amortize the optimize time into > little continuous chunks during indexing. You never stop indexing. You > should not lose documents. Is the space taken by deleted documents recovered in this case? Jerry
Re: Solr date and string search problem
This line is the key: > SEVERE: org.apache.solr.core.SolrException: Invalid Date String:'Behavior' >at org.apache.solr.schema.DateField.toInternal(DateField.java:108) >at The string 'Behavior' is being parsed as a date, and fails. Your query is attempting to find this as a date. Please post your query. and the configuration that it is used against. On Sat, Nov 14, 2009 at 4:16 AM, ashokcz wrote: > > Hi , > I have been using solr1.2 for a year and now i m facing a weird problem. > Till now i have used only string and number solr types for the search field > . > and whatever string the users are trying to search will pass it on to the > search engine and it will find in appropraite fields and return me the > results. > > But now i have added a another field with type DATE and made it as a search > field. > so what happens is whatever string that i m giving to solr tries to convert > to date and throws me an error. > > Nov 14, 2009 4:36:05 PM org.apache.solr.core.SolrException log > SEVERE: org.apache.solr.core.SolrException: Invalid Date String:'Behavior' > at org.apache.solr.schema.DateField.toInternal(DateField.java:108) > at > org.apache.solr.schema.FieldType$DefaultAnalyzer$1.next(FieldType.java:298) > at > org.apache.lucene.queryParser.QueryParser.getFieldQuery(QueryParser.java:437) > at > org.apache.solr.search.SolrQueryParser.getFieldQuery(SolrQueryParser.java:78) > at > org.apache.solr.util.SolrPluginUtils$DisjunctionMaxQueryParser.getFieldQuery(SolrPluginUtils.java:774) > at > org.apache.solr.util.SolrPluginUtils$DisjunctionMaxQueryParser.getFieldQuery(SolrPluginUtils.java:762) > at > org.apache.lucene.queryParser.QueryParser.Term(QueryParser.java:1092) > at > org.apache.lucene.queryParser.QueryParser.Clause(QueryParser.java:979) > at > org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:907) > at > org.apache.lucene.queryParser.QueryParser.TopLevelQuery(QueryParser.java:896) > at > org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:146) > at > org.apache.solr.request.DisMaxRequestHandler.handleRequestBody(DisMaxRequestHandler.java:238) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:77) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:706) > at > com.tcs.mighty.cb.service.CoreEntitySearcherImpl.solrSearch(CoreEntitySearcherImpl.java:896) > at > com.tcs.mighty.cb.service.CoreEntitySearcherImpl.search(CoreEntitySearcherImpl.java:342) > at > com.tcs.mighty.cb.service.CoreEntitySearcherImpl.handleGetSearchResults(CoreEntitySearcherImpl.java:52) > at > com.tcs.mighty.cb.service.CoreEntitySearcherBase.getSearchResults(CoreEntitySearcherBase.java:122) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:585) > at > org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:287) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:181) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148) > at > org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:97) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:170) > at > org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:170) > at > org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:176) > at $Proxy141.getSearchResults(Unknown Source) > at > com.tcs.mighty.cb.service.ejb.CoreEntitySearcherBean.getSearchResults(CoreEntitySearcherBean.java:24) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:585) > at > org.jboss.ejb.StatelessSessionContainer$ContainerInterceptor.invoke(StatelessSessionContainer.java:683) > at > org.jboss.resource.connectionmanager.CachedConnectionInterceptor.invoke(CachedConnectionInterceptor.java:186) > at > org.jboss.ejb.plugins.StatelessSessionInstanceInterceptor.invoke(StatelessSessionInstanceInterceptor.java:72) > at > org.jboss.ejb.plugins.AbstractTxInterceptor.invoke
Re: Solr 1.3 query and index perf tank during optimize
Good question! The terms in the deleted documents are left behind, and so the relevance behavior will be off. The other space used directly by documents will be reabsorbed. (??) On Sat, Nov 14, 2009 at 1:28 PM, Jerome L Quinn wrote: > > > Lance Norskog wrote on 11/13/2009 11:18:42 PM: > >> The 'maxSegments' feature is new with 1.4. I'm not sure that it will >> cause any less disk I/O during optimize. > > It could still be useful to manage the "too many open files" problem that > rears its ugly head on occasion. > >> The 'mergeFactor=2' idea is not what you think: in this case the index >> is always "mostly optimized", so you never need to run optimize. >> Indexing is always slower, because you amortize the optimize time into >> little continuous chunks during indexing. You never stop indexing. You >> should not lose documents. > > Is the space taken by deleted documents recovered in this case? > > Jerry -- Lance Norskog goks...@gmail.com
Is there a way to skip cache for a query
Hey, I do not want to disable cache completely by changing the setting in solrconfig.xml. I just want to sometimes skip cache for a query for testing purpose. So is there a parameter like skipcache=true to specify in select/?q=hot&version=2.2&start=0&rows=10&skipcache=true to skip cache for the query [hot]. skipcache can by default be false. Thanks.
Newbie Solr questions
Hi, I am new to Solr but fairly advanced with lucene. In the past I have created custom Lucene search engines that indexed objects in a Java application, so my background is coming from this requirement a) Since Solr is built on top of lucene, using SolrJ, can I still directly create custom documents, specify the field specifics etc (indexed, stored etc) and then map POJOs to those documents, simular to just using the straight lucene API? b) I took a quick look at the SolrJ javadocs but did not see anything in there that allowed me to customize if a field is stored, indexed, not indexed etc. How do I do that with SolrJ without having to go directly to the lucene apis? c) The SolrJ beans package. By annotating a POJO with @Field, how exactly does SolrJ treat that field? Indexed/stored, or just indexed? Is there any other way to control this? c) If I create a custom index outside of Solr using straight lucene, is it easy to import a pre-exisiting lucene index into a Solr Server? thanks!
Re: Newbie Solr questions
> > a) Since Solr is built on top of lucene, using SolrJ, can I still directly > create custom documents, specify the field specifics etc (indexed, stored > etc) and then map POJOs to those documents, simular to just using the > straight lucene API? > > b) I took a quick look at the SolrJ javadocs but did not see anything in > there that allowed me to customize if a field is stored, indexed, not > indexed etc. How do I do that with SolrJ without having to go directly to > the lucene apis? > > c) The SolrJ beans package. By annotating a POJO with @Field, how exactly > does SolrJ treat that field? Indexed/stored, or just indexed? Is there any > other way to control this? > The answer to all your questions above is the magical file called schema.xml. For more read here - http://wiki.apache.org/solr/SchemaXml. SolrJ is simply a java client to access (read and update from) the solr server. c) If I create a custom index outside of Solr using straight lucene, is it > easy to import a pre-exisiting lucene index into a Solr Server? > As long as the Lucene index matches the definitions in your schema you can use the same index. The data however needs to copied into a predictable location inside SOLR_HOME. Cheers Avlesh On Sun, Nov 15, 2009 at 9:26 AM, yz5od2 wrote: > Hi, > I am new to Solr but fairly advanced with lucene. > > In the past I have created custom Lucene search engines that indexed > objects in a Java application, so my background is coming from this > requirement > > a) Since Solr is built on top of lucene, using SolrJ, can I still directly > create custom documents, specify the field specifics etc (indexed, stored > etc) and then map POJOs to those documents, simular to just using the > straight lucene API? > > b) I took a quick look at the SolrJ javadocs but did not see anything in > there that allowed me to customize if a field is stored, indexed, not > indexed etc. How do I do that with SolrJ without having to go directly to > the lucene apis? > > c) The SolrJ beans package. By annotating a POJO with @Field, how exactly > does SolrJ treat that field? Indexed/stored, or just indexed? Is there any > other way to control this? > > c) If I create a custom index outside of Solr using straight lucene, is it > easy to import a pre-exisiting lucene index into a Solr Server? > > thanks! >
