RE: Does SOLR supports multiple instances within the same webapplication?
Hi, I am using SOLR 1.2 stable build currently , and this version does not have this feature (Support for Multiple SOLR cores). How do I get this feature? I tried to open the .patch file but I could not understand. I was looking into the link http://svn.apache.org/viewvc/lucene/solr/ for the solr1.3 version which supports this feature. If possible could you please point me to the link/instruction where I can check out this feature or the latest source and build the SOLR binary. Thanks in advance, Dilip -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: Monday, November 12, 2007 9:23 PM To: solr-user@lucene.apache.org Subject: Re: Does SOLR supports multiple instances within the same webapplication? Dilip.TS wrote: > Hello, > > Does SOLR supports multiple instances within the same web application? If > so how is this achieved? > If you want multiple indices, you can run multiple web-apps. If you need multiple indices in the same web-app, check SOLR-350 -- it is still in development, and make sure you *really* need it before going that route. ryan
- I/O exception (java.net.BindException) caught when processing request: Address already in use: connect while updating
Hi all,
i have a Solr instance running under tomcat 5.5. My application uses a java
servlet to handle data. I have a problem with "add-update" process: at the
first run the add process is ok and very fast . At this first run i add
around 2.000 docs. Then i commit and all is ok.
The problem is that if i run the same the "add - update" process again,
sometimes (not all the times) it fails with this error:
13-nov-2007 8.17.32 org.apache.solr.handler.XmlUpdateRequestHandler update
INFO: added id={2524} in 15ms
- I/O exception (java.net.BindException) caught when processing request:
Address already in use: connect
- Retrying request
- I/O exception (java.net.BindException) caught when processing request:
Address already in use: connect
- Retrying request
- I/O exception (java.net.BindException) caught when processing request:
Address already in use: connect
- Retrying request
Fatal transport error: Address already in use: connect
java.net.BindException: Address already in use: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(Unknown Source)
at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.(Unknown Source)
at java.net.Socket.(Unknown Source)
at
org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:79)
at
org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:121)
at
org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706)
at
org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386)
at
org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324)
at
intellisemantic.servlet.solr.DWServlet.sendPostCommand(DWServlet.java:985)
at
intellisemantic.servlet.solr.DWServlet.processExecuteCommand(DWServlet.java:826)
at intellisemantic.servlet.solr.DWServlet.indexing(DWServlet.java:435)
at intellisemantic.servlet.solr.DWServlet.doGet(DWServlet.java:163)
at intellisemantic.servlet.solr.DWServlet.doPost(DWServlet.java:1113)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Unknown Source)
- Servlet.service() for servlet dw threw exception
java.lang.NullPointerException
at
intellisemantic.servlet.solr.DWServlet.sendPostCommand(DWServlet.java:1022)
at
intellisemantic.servlet.solr.DWServlet.processExecuteCommand(DWServlet.java:826)
at intellisemantic.servlet.solr.DWServlet.indexing(DWServlet.java:435)
at intellisemantic.servlet.solr.DWServlet.doGet(DWServlet.java:163)
at intellisemantic.servlet.solr.DWServlet.doPost(DWServlet.java:1113)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at
org.apache.catalina.core.StandardWra
Re: Does SOLR supports multiple instances within the same webapplication?
take a look @: http://lucene.apache.org/solr/version_control.html#Anonymous+Access+%28read-only%29 and http://www.apache.org/dev/version-control.html#anon-svn You may want to use an IDE (eclipse/netbeans/...) to svn there (look for "trunk" dir); this way you could easily download the trunk and compile via build.xml maybe even without touching the keyboard. You can apply the patch via the IDE or via shell with the classic patch command from project main dir (the one that contains build.xml) Walter Dilip.TS wrote: > Hi, > > I am using SOLR 1.2 stable build currently , and this version does not have > this feature (Support for Multiple SOLR cores). > > How do I get this feature? I tried to open the .patch file but I could not > understand. > > I was looking into the link http://svn.apache.org/viewvc/lucene/solr/ for > the solr1.3 version which supports this feature. > > If possible could you please point me to the link/instruction where I can > check out this feature or the latest source and build the SOLR binary. > > Thanks in advance, > Dilip > > > -Original Message- > From: Ryan McKinley [mailto:[EMAIL PROTECTED] > Sent: Monday, November 12, 2007 9:23 PM > To: solr-user@lucene.apache.org > Subject: Re: Does SOLR supports multiple instances within the same > webapplication? > > > Dilip.TS wrote: > >> Hello, >> >> Does SOLR supports multiple instances within the same web application? >> > If > >> so how is this achieved? >> >> > > If you want multiple indices, you can run multiple web-apps. > > If you need multiple indices in the same web-app, check SOLR-350 -- it > is still in development, and make sure you *really* need it before going > that route. > > ryan > > >
Re: - I/O exception (java.net.BindException) caught when processing request: Address already in use: connect while updating
Daniele
>From the stack trace it looks like the java.net.BindException is
happening when you are calling HttpClient code. Searching google for
"java.net.BindException httpclient" finds various hits. Maybe one of
those will help. I don't think this has anything directly to do with
solr or tomcat.
--
Ian.
On Nov 13, 2007 9:16 AM, Daniele Salvatico <[EMAIL PROTECTED]> wrote:
>
>
> Hi all,
>
> i have a Solr instance running under tomcat 5.5. My application uses a java
> servlet to handle data. I have a problem with "add-update" process: at the
> first run the add process is ok and very fast . At this first run i add
> around 2.000 docs. Then i commit and all is ok.
> The problem is that if i run the same the "add - update" process again,
> sometimes (not all the times) it fails with this error:
>
> 13-nov-2007 8.17.32 org.apache.solr.handler.XmlUpdateRequestHandler update
> INFO: added id={2524} in 15ms
> - I/O exception (java.net.BindException) caught when processing request:
> Address already in use: connect
> - Retrying request
> - I/O exception (java.net.BindException) caught when processing request:
> Address already in use: connect
> - Retrying request
> - I/O exception (java.net.BindException) caught when processing request:
> Address already in use: connect
> - Retrying request
> Fatal transport error: Address already in use: connect
> java.net.BindException: Address already in use: connect
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.PlainSocketImpl.doConnect(Unknown Source)
> at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
> at java.net.PlainSocketImpl.connect(Unknown Source)
> at java.net.SocksSocketImpl.connect(Unknown Source)
> at java.net.Socket.connect(Unknown Source)
> at java.net.Socket.connect(Unknown Source)
> at java.net.Socket.(Unknown Source)
> at java.net.Socket.(Unknown Source)
> at
> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:79)
> at
> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:121)
> at
> org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706)
> at
> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386)
> at
> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
> at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
> at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324)
> at
> intellisemantic.servlet.solr.DWServlet.sendPostCommand(DWServlet.java:985)
> at
> intellisemantic.servlet.solr.DWServlet.processExecuteCommand(DWServlet.java:826)
> at intellisemantic.servlet.solr.DWServlet.indexing(DWServlet.java:435)
> at intellisemantic.servlet.solr.DWServlet.doGet(DWServlet.java:163)
> at intellisemantic.servlet.solr.DWServlet.doPost(DWServlet.java:1113)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
> at
> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
> at
> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
> at
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
> at
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
> at java.lang.Thread.run(Unknown Source)
> - Servlet.service() for servlet dw threw exception
> java.lang.NullPointerException
> at
> intellisemantic.servlet.solr.DWServlet.sendPostCommand(DWServlet.java:1022)
> at
> intellisemantic.servlet.solr.DWServlet.processExecuteCommand(DWServlet.java:826)
> at intellisemantic.servlet.solr.DWServlet.indexing(DWServlet.jav
Re: - I/O exception (java.net.BindException) caught when processing request: Address already in use: connect while updating
Thanks Ian,
I'll search for HttpClient related problem.
Daniele
Ian Lea wrote:
>
> Daniele
>
>
> From the stack trace it looks like the java.net.BindException is
> happening when you are calling HttpClient code. Searching google for
> "java.net.BindException httpclient" finds various hits. Maybe one of
> those will help. I don't think this has anything directly to do with
> solr or tomcat.
>
>
> --
> Ian.
>
>
>
>
> On Nov 13, 2007 9:16 AM, Daniele Salvatico <[EMAIL PROTECTED]>
> wrote:
>>
>>
>> Hi all,
>>
>> i have a Solr instance running under tomcat 5.5. My application uses a
>> java
>> servlet to handle data. I have a problem with "add-update" process: at
>> the
>> first run the add process is ok and very fast . At this first run i add
>> around 2.000 docs. Then i commit and all is ok.
>> The problem is that if i run the same the "add - update" process again,
>> sometimes (not all the times) it fails with this error:
>>
>> 13-nov-2007 8.17.32 org.apache.solr.handler.XmlUpdateRequestHandler
>> update
>> INFO: added id={2524} in 15ms
>> - I/O exception (java.net.BindException) caught when processing request:
>> Address already in use: connect
>> - Retrying request
>> - I/O exception (java.net.BindException) caught when processing request:
>> Address already in use: connect
>> - Retrying request
>> - I/O exception (java.net.BindException) caught when processing request:
>> Address already in use: connect
>> - Retrying request
>> Fatal transport error: Address already in use: connect
>> java.net.BindException: Address already in use: connect
>> at java.net.PlainSocketImpl.socketConnect(Native Method)
>> at java.net.PlainSocketImpl.doConnect(Unknown Source)
>> at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
>> at java.net.PlainSocketImpl.connect(Unknown Source)
>> at java.net.SocksSocketImpl.connect(Unknown Source)
>> at java.net.Socket.connect(Unknown Source)
>> at java.net.Socket.connect(Unknown Source)
>> at java.net.Socket.(Unknown Source)
>> at java.net.Socket.(Unknown Source)
>> at
>> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:79)
>> at
>> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:121)
>> at
>> org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706)
>> at
>> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386)
>> at
>> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
>> at
>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
>> at
>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324)
>> at
>> intellisemantic.servlet.solr.DWServlet.sendPostCommand(DWServlet.java:985)
>> at
>> intellisemantic.servlet.solr.DWServlet.processExecuteCommand(DWServlet.java:826)
>> at
>> intellisemantic.servlet.solr.DWServlet.indexing(DWServlet.java:435)
>> at
>> intellisemantic.servlet.solr.DWServlet.doGet(DWServlet.java:163)
>> at
>> intellisemantic.servlet.solr.DWServlet.doPost(DWServlet.java:1113)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
>> at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
>> at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
>> at
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
>> at
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
>> at
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
>> at
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
>> at
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
>> at
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
>> at
>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
>> at
>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
>> at
>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
>> at
>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
>> at
>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
>> at java.lang.Thread.run(Unknown Source)
>> - Servlet.service() for servlet dw threw exception
>> java.lang.NullPointerException
>> at
>> intellisemantic.se
solr internals
Hi All When a word document or excel is sent to solr, I can see my indexes are getting updated [via UpdateRichDocuments patch]. I wish to know what really happens behind the screen. Can anyone explain internals of solr on adding word documents ?. How do we fix fields for word doc which is 5 - 10 pages ?. Any advise will be great Regards Dwarak R This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender&[EMAIL PROTECTED] immediately and delete the original. Any other use of the email by you is prohibited.
Re: PriceJunkie.com using solr!
Hi Mike, I´m checking out www.pricejunkie.com and Í would like to know how do you group the products and find the price range. Is it a batch process ? Are you using MoreLikeThis to do it ? Thanks, William. >ahait is wonderful. >2007/5/24, Mike Austin <[EMAIL PROTECTED]>: >Just one. >-Original Message- >From: James liu [EMAIL PROTECTED] >Sent: Wednesday, May 16, 2007 10:30 PM >To: solr-user@lucene.apache.org >Subject: Re: PriceJunkie.com using solr! >how many solr instance? >2007/5/17, Yonik Seeley <[EMAIL PROTECTED]>: > > Congrats, very nice job! > It's fast too. > > -Yonik > > On 5/16/07, Mike Austin <[EMAIL PROTECTED]> wrote: > > I just wanted to say thanks to everyone for the creation of solr. I've > been > > using it for a while now and I have recently brought one of my side > projects > > online. I have several other projects that will be using solr for it's > > search and facets. > > > > Please check out www.pricejunkie.com and let us know what you think.. > You > > can give feedback and/or sign up on the mailing list for future updates. > > The site is very basic right now and many new and useful features plus > > merchants and product categories will be coming soon! I thought it > would be > > a good idea to at least have a few people use it to get some feedback > early > > and often. > > > > Some of the nice things behind the scenes that we did with solr: > > - created custom request handlers that have category to facet to > attribute > > caching built in > > - category to facet management > > - ability to manage facet groups (attributes within a set facet) > and assign > > them to categories > > - ability to create any category structure and share facet > groups > > > > - facet inheritance for any category (a facet group can be defined on a > > parent category and pushed down to all children) > > - ability to create sub-categories as facets instead of normal sub > > categories > > - simple xml configuration for the final outputted category > configuration > > file > > > > > > I'm sure there are more cool things but that is all for now. Join the > > mailing list to see more improvements in the future. > > > > Also.. how do I get added to the Using Solr wiki page? > > > > > > Thanks, > > Mike Austin >
Solr/bin script - Solaris bash version?
Hi, Is there Solaris bash based script available? The couple of command is not working, and wondering any available scripts I can use before I update it. For ex. snapshooter, snappuller, snapinstaller Thanks, Jae
snapshot files
Hi,
I have successfully built generated the snalshot files but have a question.
Does each snapshot file has all files in the index directory?
Here is the file list in the index
_0.fdt_0.fnm_0.nrm_0.tii_1.fdt
_1.fnm_1.nrm_1.tiisegments.gen
_0.fdx_0.frq_0.prx_0.tis_1.fdx
_1.frq_1.prx_1.tissegments_3
And here is the file list of 2 snapshot files.
snapshot.20071113094936
_0.fdt_0.fdx_0.fnm_0.frq_0.nrm
_0.prx_0.tii_0.tissegments.gen segments_2
snapshot.20071113095508
_0.fdt_0.fnm_0.nrm_0.tii_1.fdt
_1.fnm_1.nrm_1.tiisegments.gen
_0.fdx_0.frq_0.prx_0.tis_1.fdx
_1.frq_1.prx_1.tissegments_3
The later one have all files same as index directory.
I have changed the snapshooter script because the bash in solaris do not
have cp -l option.
#cp -lr ${data_dir}/index ${temp} --> original
mkdir ${temp}
ln ${data_dir}/index/* ${temp}
Thanks,
Jae Joo
two solr instances - index and commit
Hi, I have two solr instance running under different tomcat environment. One solr instance is for indexing and would like to commit to the other solr instance. This is what I tried, but failed. using post.sh (without commit), the docs are indexed in solr-1 instance. After indexed, call commit command with the attribute of solr-2. Can any help me? Jae
LuceneInAction.zip?
I purchased Lucene In Action (really great book by the way, one of the best technical books (if not the best) that I can ever read. Its making me embarrassed about some of the questions I have already posted :) That said, here is another one -- I found LuceneInAction.zip on www.lucenebook.com and downloaded to get the examples in LIA. Is there any copy of this upgraded to Lucene2.2.0 code? While the conversion to 2.2.0 has been fairly obvious for all the cases I have looked at so far, there seems to be allot of use of deprecated methods in external libraries as well. Just wondering if there was a short cut -- and an updated version of this zip that I have not located? Very well done though -- this book and zip is extremely helpful -- even as I tried to get grounded in SOLR as well. Thanks, Dave Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs
Multiple uniqueKey fields
Hello, I just started using Solr and have a question regarding unique keys. Is it somehow possible to specify more than 1 unique key? The problem is, some of my indexed data has one particular unique field, while other data does not have this field at all and uses another field for unique-ness. Any suggestions on how to overcome this? Thanks, David _ Peek-a-boo FREE Tricks & Treats for You! http://www.reallivemoms.com?ocid=TXT_TAGHM&loc=us
Re: LuceneInAction.zip?
On 13-Nov-07, at 9:08 AM, David Neubert wrote: I purchased Lucene In Action (really great book by the way, one of the best technical books (if not the best) that I can ever read. Its making me embarrassed about some of the questions I have already posted :) That said, here is another one -- I found LuceneInAction.zip on www.lucenebook.com and downloaded to get the examples in LIA. Is there any copy of this upgraded to Lucene2.2.0 code? While the conversion to 2.2.0 has been fairly obvious for all the cases I have looked at so far, there seems to be allot of use of deprecated methods in external libraries as well. I don't think so, but I suggest asking this question on java- [EMAIL PROTECTED], which has a much broader lucene-related audience. -Mike
RE: Exception in SOLR when querying for fields of type string
The first question is, what version of Solr are you using? Thanks, Stu -Original Message- From: Kasi Sankaralingam <[EMAIL PROTECTED]> Sent: Tuesday, November 13, 2007 2:27pm To: solr-user@lucene.apache.org Subject: Exception in SOLR when querying for fields of type string Hi, I am running into nullpointerexception on the SOLR side, when I do the following: a) Define a dynamic field in the schema of type string (say title_s) b) Do a query in the SOLR admin tool title_s: photo book I get a null pointer exception when I run a search query on this. If I enclose the search term within double quotes like "photo book", it works fine. Any ideas? Thanks, kasi
Re: Faceting over limited result set
On 12-Nov-07, at 8:03 AM, Chris Hostetter wrote:
if what you are interested in is stats on the first N docs
according to a
specific sort (score or otherwise) then you could write a custom
request
handler that executed a search with a limit of N, got the DocList,
iterated over it to build a DocSet, and then used that DocSet to do
faceting ... but that would probably take even longer then just
using the
full DocSet matching the entire query.
An implementation might look like:
DocList superlist;
int facetDocLimit = params.getInt(DMP.FACET_DOCLIMIT, -1);
if(facetDocLimit > 0 && facetDocLimit != req.getLimit()) {
superlist = s.getDocList(query, restrictions,
SolrPluginUtils.getSort(req),
req.getStart(), facetDocLimit,
flags);
results.docSet = SearcherUtils.getDocSetFromDocList
(superlist, s);
results.docList = superlist.subset(0, req.getLimit());
} else {
Where getDocSetFromDocList() uses DocSetHitCollector to build a DocSet.
To answer the performance question: There is a gain to be had when
doing lots of faceting on huge indices, if N is low (say, 500-1000).
One problem with the implementation above is that it stymies the
query caching in SolrIndexSearcher (since the generated DocList is >
the cache upper bound).
-Mike
RE: Multiple uniqueKey fields
key = sometimesUniqueField + "_" + otherTimesUniqueuField; If one of them is always guaranteed to be unique you should be fine. - will -Original Message- From: Dave C. [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 13, 2007 2:07 PM To: solr-user@lucene.apache.org Subject: Multiple uniqueKey fields Hello, I just started using Solr and have a question regarding unique keys. Is it somehow possible to specify more than 1 unique key? The problem is, some of my indexed data has one particular unique field, while other data does not have this field at all and uses another field for unique-ness. Any suggestions on how to overcome this? Thanks, David _ Peek-a-boo FREE Tricks & Treats for You! http://www.reallivemoms.com?ocid=TXT_TAGHM&loc=us
Index/Query question
Hello all, I am in the process of learning Solr (and the underlying Lucene engine). I have been unable to successfully implement the following scenario, so I was hoping someone on the list might have some insight. - I want to index data values for different data types (fields) that are currently housed in a database. - I want to search across the fields. - I need (I think) to store the string representation (actual value) AND a tokenized/parsed representation -- which is where I fall down. Example: I have an email address ([EMAIL PROTECTED]), a database name (5_user_db), and a username (user). Here is what I would like to see happen (dynamic search as keys are typed) .. 1. I enter 'user' and I get all three above in the search result. I can accomplish this today using the solr.WordDelimiterFilterFactory. 2. I then add '@' so I have 'user@', which I want to return only the email address record. However, this will not happen with solr.WordDelimiterFilterFactory, but I can get this result to work if I only index it as a string. But, then my search results for 'user' will not return the database record. Any thoughts? Let me know if I need to clarify any points. Thanks! Marc
RE: Multiple uniqueKey fields
Thanks, I thought of this shortly after I posted the question! - david > From: [EMAIL PROTECTED] > To: solr-user@lucene.apache.org > Subject: RE: Multiple uniqueKey fields > Date: Tue, 13 Nov 2007 15:37:46 -0500 > > key = sometimesUniqueField + "_" + otherTimesUniqueuField; > > If one of them is always guaranteed to be unique you should be fine. > > - will > > -Original Message- > From: Dave C. [mailto:[EMAIL PROTECTED] > Sent: Tuesday, November 13, 2007 2:07 PM > To: solr-user@lucene.apache.org > Subject: Multiple uniqueKey fields > > Hello, I just started using Solr and have a question regarding unique keys. > > Is it somehow possible to specify more than 1 unique key? > The > problem is, some of my indexed data has one particular unique field, > while other data does not have this field at all and uses another field > for unique-ness. > Any suggestions on how to overcome this? > > Thanks, > David > _ > Peek-a-boo FREE Tricks & Treats for You! > http://www.reallivemoms.com?ocid=TXT_TAGHM&loc=us > _ Help yourself to FREE treats served up daily at the Messenger Café. Stop by today. http://www.cafemessenger.com/info/info_sweetstuff2.html?ocid=TXT_TAGLM_OctWLtagline
Solr java tutorial
Hi everyone, My next question is: is there a beginner's tutorial on how to programmatically query solr? I have seen some users' do things like: SolrServer server = getSolrServer(); SolrQuery solrQuery = new SolrQuery(); solrQuery.setQuery(..); QueryResponse rsp = server.query(solrQuery); But is there some tutorial that outlines the basic classes to use, etc? I know there is a javadoc but it's not helpful if I don't know what to use. Thanks a lot, David _ Help yourself to FREE treats served up daily at the Messenger Café. Stop by today. http://www.cafemessenger.com/info/info_sweetstuff2.html?ocid=TXT_TAGLM_OctWLtagline
RE: Exception in SOLR when querying for fields of type string
I am using the latest version of solr (1.2) -Original Message- From: Stu Hood [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 13, 2007 11:32 AM To: solr-user@lucene.apache.org Subject: RE: Exception in SOLR when querying for fields of type string The first question is, what version of Solr are you using? Thanks, Stu -Original Message- From: Kasi Sankaralingam <[EMAIL PROTECTED]> Sent: Tuesday, November 13, 2007 2:27pm To: solr-user@lucene.apache.org Subject: Exception in SOLR when querying for fields of type string Hi, I am running into nullpointerexception on the SOLR side, when I do the following: a) Define a dynamic field in the schema of type string (say title_s) b) Do a query in the SOLR admin tool title_s: photo book I get a null pointer exception when I run a search query on this. If I enclose the search term within double quotes like "photo book", it works fine. Any ideas? Thanks, kasi
Re: Solr java tutorial
Not yet, but there should be! Currently people learn it from looking at the source and tests. I started to add something to: http://wiki.apache.org/solr/Solrj it (obviously) still needs work. If you are using eclipse (or similar), after typing "solrQuery." you should get a drop down of all the available options - i find this invaluable for learning what is possible. ryan Dave C. wrote: Hi everyone, My next question is: is there a beginner's tutorial on how to programmatically query solr? I have seen some users' do things like: SolrServer server = getSolrServer(); SolrQuery solrQuery = new SolrQuery(); solrQuery.setQuery(..); QueryResponse rsp = server.query(solrQuery); But is there some tutorial that outlines the basic classes to use, etc? I know there is a javadoc but it's not helpful if I don't know what to use. Thanks a lot, David _ Help yourself to FREE treats served up daily at the Messenger Café. Stop by today. http://www.cafemessenger.com/info/info_sweetstuff2.html?ocid=TXT_TAGLM_OctWLtagline
Re: Exception in SOLR when querying for fields of type string
Can you post the full exception? b) Do a query in the SOLR admin tool title_s: photo book Do you have a space after the ':'? q=title_s: photo book I expect that would fail (though null pointer is not a very nice error) q=title_s:"photo book" should work fine: title_s:photo book would match "photo" for field 'title_s' and "book" for the default field. ryan
Re: Multiple uniqueKey fields
I had a similar problem with three sources of keys that have collisions between the values. I prefix a single letter for each source. movies: M12345 people: P12345 and so on. wunder On 11/13/07 12:37 PM, "Will Johnson" <[EMAIL PROTECTED]> wrote: > key = sometimesUniqueField + "_" + otherTimesUniqueuField; > > If one of them is always guaranteed to be unique you should be fine. > > - will > > -Original Message- > From: Dave C. [mailto:[EMAIL PROTECTED] > Sent: Tuesday, November 13, 2007 2:07 PM > To: solr-user@lucene.apache.org > Subject: Multiple uniqueKey fields > > Hello, I just started using Solr and have a question regarding unique keys. > > Is it somehow possible to specify more than 1 unique key? > The > problem is, some of my indexed data has one particular unique field, > while other data does not have this field at all and uses another field > for unique-ness. > Any suggestions on how to overcome this? > > Thanks, > David > _ > Peek-a-boo FREE Tricks & Treats for You! > http://www.reallivemoms.com?ocid=TXT_TAGHM&loc=us >
RE: Exception in SOLR when querying for fields of type string
It is not tokenized, it is a string field, so will it still match "photo" for field 'title_s' and "book" for the default field? Here is the exception stack trace: HTTP Status 500 - null java.lang.NullPointerException at org.apache.solr.search.SolrQueryParser.getFieldQuery(SolrQueryParser.java:73) at org.apache.lucene.queryParser.QueryParser.Term(QueryParser.java:1092) at org.apache.lucene.queryParser.QueryParser.Clause(QueryParser.java:979) at org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:907) at org.apache.lucene.queryParser.QueryParser.TopLevelQuery(QueryParser.java:896) at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:146) at org.apache.solr.search.QueryParsing.parseQuery(QueryParsing.java:94) at org.apache.solr.request.StandardRequestHandler.handleRequestBody(StandardRequestHandler.java:109) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:77) at org.apache.solr.core.SolrCore.execute(SolrCore.java:658) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:191) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:159) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869) at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664) at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527) at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684) at java.lang.Thread.run(Thread.java:595) -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 13, 2007 2:26 PM To: solr-user@lucene.apache.org Subject: Re: Exception in SOLR when querying for fields of type string Can you post the full exception? > > b) Do a query in the SOLR admin tool title_s: photo book > Do you have a space after the ':'? q=title_s: photo book I expect that would fail (though null pointer is not a very nice error) q=title_s:"photo book" should work fine: title_s:photo book would match "photo" for field 'title_s' and "book" for the default field. ryan
RE: Solr java tutorial
I'm really confused...is SolrJ coming in Solr 1.3 (which hasn't been released yet)? It doesn't seem to be found in Solr 1.2 which I am using. please enlighten.. - David > Date: Tue, 13 Nov 2007 17:21:13 -0500 > From: [EMAIL PROTECTED] > To: solr-user@lucene.apache.org > Subject: Re: Solr java tutorial > > > Not yet, but there should be! > > Currently people learn it from looking at the source and tests. I > started to add something to: > > http://wiki.apache.org/solr/Solrj > > it (obviously) still needs work. > > If you are using eclipse (or similar), after typing "solrQuery." you > should get a drop down of all the available options - i find this > invaluable for learning what is possible. > > ryan > > > > Dave C. wrote: > > Hi everyone, > > > > My next question is: is there a beginner's tutorial on how to > > programmatically query solr? > > I have seen some users' do things like: > > > > > > SolrServer server = getSolrServer(); > > > > SolrQuery > > solrQuery = new SolrQuery(); > > > > > > solrQuery.setQuery(..); > > > > > > QueryResponse rsp = server.query(solrQuery); > > > > But is there some tutorial that outlines the basic classes to use, etc? > > I know there is a javadoc but it's not helpful if I don't know what to use. > > > > Thanks a lot, > > David > > > > _ > > Help yourself to FREE treats served up daily at the Messenger Café. Stop by > > today. > > http://www.cafemessenger.com/info/info_sweetstuff2.html?ocid=TXT_TAGLM_OctWLtagline > _ Windows Live Hotmail and Microsoft Office Outlook – together at last. Get it now. http://office.microsoft.com/en-us/outlook/HA102225181033.aspx?pid=CL100626971033
Re: Solr java tutorial
Dave C. wrote: I'm really confused...is SolrJ coming in Solr 1.3 (which hasn't been released yet)? It doesn't seem to be found in Solr 1.2 which I am using. please enlighten.. correct. solrj is in solr-1.3-dev -- to use it you need to use a nightly build. It has not been officially released yet. While it is quite stable, the interfaces are still subject to change until the final release. ryan
RE: Solr java tutorial
Ah... :( Is there a timeline for the 1.3 release? - david > Date: Tue, 13 Nov 2007 18:33:01 -0500 > From: [EMAIL PROTECTED] > To: solr-user@lucene.apache.org > Subject: Re: Solr java tutorial > > Dave C. wrote: > > I'm really confused...is SolrJ coming in Solr 1.3 (which hasn't been > > released yet)? > > It doesn't seem to be found in Solr 1.2 which I am using. > > > > please enlighten.. > > > > correct. solrj is in solr-1.3-dev -- to use it you need to use a nightly > build. > > It has not been officially released yet. While it is quite stable, the > interfaces are still subject to change until the final release. > > ryan _ Boo! Scare away worms, viruses and so much more! Try Windows Live OneCare! http://onecare.live.com/standard/en-us/purchase/trial.aspx?s_cid=wl_hotmailnews
RE: two solr instances - index and commit
This works, the only thing you need to be aware of is the NFS problem if you are running in a distributed environment sharing a NFS partition. a) Index and commit on instance (Typically partitioned as an index server) b) Issue a commit on the search server (like a read only mode) Things to watch out for, you will get stale NFS problem, I replaced lucene core that is shipped with solr to the latest one and it works. -Original Message- From: Jae Joo [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 13, 2007 9:06 AM To: solr-user Subject: two solr instances - index and commit Hi, I have two solr instance running under different tomcat environment. One solr instance is for indexing and would like to commit to the other solr instance. This is what I tried, but failed. using post.sh (without commit), the docs are indexed in solr-1 instance. After indexed, call commit command with the attribute of solr-2. Can any help me? Jae
Re: Faceting over limited result set
On Nov 14, 2007 6:44 AM, Mike Klaas <[EMAIL PROTECTED]> wrote:
>
> An implementation might look like:
>
> DocList superlist;
> int facetDocLimit = params.getInt(DMP.FACET_DOCLIMIT, -1);
> if(facetDocLimit > 0 && facetDocLimit != req.getLimit()) {
>superlist = s.getDocList(query, restrictions,
> SolrPluginUtils.getSort(req),
> req.getStart(), facetDocLimit,
> flags);
>results.docSet = SearcherUtils.getDocSetFromDocList
> (superlist, s);
>results.docList = superlist.subset(0, req.getLimit());
> } else {
>
> Where getDocSetFromDocList() uses DocSetHitCollector to build a DocSet.
>
> To answer the performance question: There is a gain to be had when
> doing lots of faceting on huge indices, if N is low (say, 500-1000).
> One problem with the implementation above is that it stymies the
> query caching in SolrIndexSearcher (since the generated DocList is >
> the cache upper bound).
>
> -Mike
Thanks Mike, that looks like a good place to start. While I really
can't think of any practical use for limiting the size of DocSet other
than simple faceting, the new search component architecture make it a
little more difficult to confine any implementation to only the facet
component (unless there is an efficient way to obtain a subset of a
DocSet, which there doesn't seem to be). I'm also aware of the query
caching issues arising from SolrIndexSearcher however if N is
sufficiently low this (hopefully) shouldn't be too much of a problem.
I can't find either the SearcherUtils class nor any reference to a
getDocSetFromDocList() method in svn trunk, is this deprecated or
custom-build code?
-Piete
Re: Solr java tutorial
Not really--there have been a few threads on this topic recently. Perhaps in a couple months? It may depend on the timing of the lucene release. -MIke On 13-Nov-07, at 3:41 PM, Dave C. wrote: Ah... :( Is there a timeline for the 1.3 release? - david Date: Tue, 13 Nov 2007 18:33:01 -0500 From: [EMAIL PROTECTED] To: solr-user@lucene.apache.org Subject: Re: Solr java tutorial Dave C. wrote: I'm really confused...is SolrJ coming in Solr 1.3 (which hasn't been released yet)? It doesn't seem to be found in Solr 1.2 which I am using. please enlighten.. correct. solrj is in solr-1.3-dev -- to use it you need to use a nightly build. It has not been officially released yet. While it is quite stable, the interfaces are still subject to change until the final release. ryan _ Boo! Scare away worms, viruses and so much more! Try Windows Live OneCare! http://onecare.live.com/standard/en-us/purchase/trial.aspx? s_cid=wl_hotmailnews
Re: Exception in SOLR when querying for fields of type string
On Nov 13, 2007 6:23 PM, Kasi Sankaralingam <[EMAIL PROTECTED]> wrote: > It is not tokenized, it is a string field, so will it still match > "photo" for field 'title_s' and "book" for the default field? Yes, because the query parser splits up things by whitespace before analyzers are even applied. Do you have a default field defined? -Yonik
Embedded Solr
Hello again, This is a horribly newbie question, but what exactly is meant by "embedded solr"? Thanks a lot, David _ Boo! Scare away worms, viruses and so much more! Try Windows Live OneCare! http://onecare.live.com/standard/en-us/purchase/trial.aspx?s_cid=wl_hotmailnews
Re: Embedded Solr
Standard solr is a .war file that you install on your system and run within a servlet container (jetty, resin, tomcat, etc) "embedded" solr refers to running solr without the servlet container. ryan Dave C. wrote: Hello again, This is a horribly newbie question, but what exactly is meant by "embedded solr"? Thanks a lot, David _ Boo! Scare away worms, viruses and so much more! Try Windows Live OneCare! http://onecare.live.com/standard/en-us/purchase/trial.aspx?s_cid=wl_hotmailnews
Re: Query and heap Size
On 12-Nov-07, at 6:02 AM, Jae Joo wrote: In my system, the heap size (old generation) keeps growing up caused by heavy traffic. I have adjusted the size of young generation, but it does not work well. Does anyone have any recommendation regarding this issue? - Solr configuration and/or web.xml ...etc... Perhaps check your cache statistics on the admin gui. Is it possible that you have set the capacity high and they are just filling up? Another thing to look out for is if you tend to sort on many different fields, but rarely. -Mike
Re: Faceting over limited result set
On 13-Nov-07, at 4:44 PM, Pieter Berkel wrote: On Nov 14, 2007 6:44 AM, Mike Klaas <[EMAIL PROTECTED]> wrote: Thanks Mike, that looks like a good place to start. While I really can't think of any practical use for limiting the size of DocSet other than simple faceting, the new search component architecture make it a little more difficult to confine any implementation to only the facet component (unless there is an efficient way to obtain a subset of a DocSet, which there doesn't seem to be). DocSets (so far) are unordered so I don't see how that would work. I'm also aware of the query caching issues arising from SolrIndexSearcher however if N is sufficiently low this (hopefully) shouldn't be too much of a problem. I can't find either the SearcherUtils class nor any reference to a getDocSetFromDocList() method in svn trunk, is this deprecated or custom-build code? Custom. It is a handful of lines that just passes the docs from a DocIterator to DocSetHitCollector. -Mike
solr - other document formats
Hey All I read an article on http://www.xml.com/lpt/a/1668 Its states that "As we've seen, the XML format used by Solr for indexing is quite simple. Extracting the relevant metadata to create these XML documents from the many formats floating around, however, is another story. Fortunately, Lucene users have the same problem and have been working on it for quite a while; the Lucene FAQ lists a number of references to parsers and filters which can be used to extract content and metadata from many common document formats. Solr won't index spreadsheets or other formats out of the box, but that is not its role: you should see Solr as the "search engine" component of a broader "search system," where extraction of content and metadata is handled by other components. This will help to keep your search system maintainable and testable, and it helps the Solr team focus on doing one thing well." Parsing documents like pdf, ms word document, excel to xml will be done other component ? Somebody advise Regards Dwarak R This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender&[EMAIL PROTECTED] immediately and delete the original. Any other use of the email by you is prohibited.
