4.10.4 - nodes up, shard without leader
Hello - i stumbled upon an issue i've never seen earlier, a shard with all
nodes up and running but no leader. This is on 4.10.4. One of the two nodes
emits the following error log entry:
2015-03-08 05:25:49,095 WARN [solr.cloud.ElectionContext] - [Thread-136] - :
cancelElection did not find election node to remove
/overseer_elect/election/93434598784958483-178.21.116.225:8080_solr-n_000246
2015-03-08 05:25:49,121 WARN [solr.cloud.ElectionContext] - [Thread-136] - :
cancelElection did not find election node to remove
/collections/oi/leader_elect/shard3/election/93434598784958483-178.21.116.225:8080_solr_oi_h-n_43
2015-03-08 05:25:49,220 ERROR [solr.update.UpdateLog] - [Thread-136] - : Error
inspecting tlog
tlog{file=/opt/solr/cores/oi_c/data/tlog/tlog.0001394 refcount=2}
java.nio.channels.ClosedChannelException
at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:99)
at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:679)
at
org.apache.solr.update.ChannelFastInputStream.readWrappedStream(TransactionLog.java:784)
at
org.apache.solr.common.util.FastInputStream.refill(FastInputStream.java:89)
at
org.apache.solr.common.util.FastInputStream.read(FastInputStream.java:125)
at java.io.InputStream.read(InputStream.java:101)
at
org.apache.solr.update.TransactionLog.endsWithCommit(TransactionLog.java:218)
at org.apache.solr.update.UpdateLog.recoverFromLog(UpdateLog.java:800)
at org.apache.solr.cloud.ZkController.register(ZkController.java:841)
at org.apache.solr.cloud.ZkController$1.command(ZkController.java:277)
at
org.apache.solr.common.cloud.ConnectionManager$1$1.run(ConnectionManager.java:166)
2015-03-08 05:25:49,225 ERROR [solr.update.UpdateLog] - [Thread-136] - : Error
inspecting tlog
tlog{file=/opt/solr/cores/oi_c/data/tlog/tlog.0001471 refcount=2}
java.nio.channels.ClosedChannelException
at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:99)
at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:679)
at
org.apache.solr.update.ChannelFastInputStream.readWrappedStream(TransactionLog.java:784)
at
org.apache.solr.common.util.FastInputStream.refill(FastInputStream.java:89)
at
org.apache.solr.common.util.FastInputStream.read(FastInputStream.java:125)
at java.io.InputStream.read(InputStream.java:101)
at
org.apache.solr.update.TransactionLog.endsWithCommit(TransactionLog.java:218)
at org.apache.solr.update.UpdateLog.recoverFromLog(UpdateLog.java:800)
at org.apache.solr.cloud.ZkController.register(ZkController.java:841)
at org.apache.solr.cloud.ZkController$1.command(ZkController.java:277)
at
org.apache.solr.common.cloud.ConnectionManager$1$1.run(ConnectionManager.java:166)
2015-03-08 12:21:04,438 WARN [solr.cloud.RecoveryStrategy] -
[zkCallback-2-thread-28] - : Stopping recovery for core=oi_h
coreNodeName=178.21.116.225:8080_solr_oi_h
The other node makes a mess in the logs:
2015-03-08 05:25:46,020 WARN [solr.cloud.RecoveryStrategy] -
[zkCallback-2-thread-20] - : Stopping recovery for core=oi_c
coreNodeName=194.145.201.190:
8080_solr_oi_c
2015-03-08 05:26:08,670 ERROR [solr.cloud.ShardLeaderElectionContext] -
[zkCallback-2-thread-19] - : There was a problem trying to register as the
leader:org.
apache.solr.common.SolrException: Could not register as the leader because
creating the ephemeral registration node in ZooKeeper failed
at
org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:146)
at
org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:317)
at
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
at
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
at
org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
at
org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:210)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.common.SolrException:
org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
NodeExists for /collections/openi
ndex/leaders/shard3
at
org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:40)
at
org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:134)
... 11 more
Caused by: org.apac
Re: Frequency of Suggestion are varying from original Frequency in index
Hi Wang,
I am using SolrCloud. Is suggestion not working properly in
that?
On Fri, Mar 6, 2015 at 2:36 PM, gaohang wang wrote:
> do you use solrcloud?maybe your suggestion is not support distribute
>
> 2015-03-04 22:39 GMT+08:00 Nitin Solanki :
>
> > Hi..
> >I have a term("who") where original frequency of "who" is 191 but
> > when I get suggestion of "who" it gives me 90. Why?
> >
> > Example :
> >
> > *Original Frequency* comes like:
> >
> > "spellcheck":{
> > "suggestions":[
> > "who",{
> > "numFound":1,
> > "startOffset":1,
> > "endOffset":4,
> > "origFreq":*191*,
> > "correctlySpelled",false]}}
> >
> > While In *Suggestion*, it gives like:
> >
> > "spellcheck":{
> > "suggestions":[
> > "whs",{
> > "numFound":1,
> > "startOffset":1,
> > "endOffset":4,
> > "origFreq":0,
> > "suggestion":[{
> > "word":"who",
> > "freq":*90*}]},
> > "correctlySpelled",false]}}
> >
> >
> >
> > Why it is so?
> >
> > I am using StandardTokenizerFactory with ShingleFilterFactory in
> > Schema.xml..
> >
>
Re: Frequency of Suggestion are varying from original Frequency in index
Hi ale42, I am not using /solr.IndexBasedSpellChecker/. I used solr.DirectSpellChecker. Is there anyway to solve my issue? On Fri, Mar 6, 2015 at 6:27 PM, ale42 < alexandre.faye...@etu.esisar.grenoble-inp.fr> wrote: > I think these frequencies are not the frequence of the term in the same > index > : > > - original frequency represents the number of results that you have in > lucene index when you query "who". > > - suggestion frequency is the number of results of this term in the > spellcheck dictionnary. > > I guess you're using /solr.IndexBasedSpellChecker/ ! > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Frequency-of-Suggestion-are-varying-from-original-Frequency-in-index-tp4190927p4191397.html > Sent from the Solr - User mailing list archive at Nabble.com. >
Solr TCP layer
Dear Solr Contributors, I want to start working on adding a TCP layer for client to node and inter-node communication. I am not up to date on recent changes happening to Solr. So before I start looking into code, I would like to know if there is already some work done in this direction, which I can reuse. Are there any know challenges/complexities? I would appreciate any help to kick start this effort. Also, what would be the best way to discuss and get feedback on design from contributors? Open a JIRA?? Regards, Saumitra -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-TCP-layer-tp4191715.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: SpellCheck component query
Hi, AFAIK solr currently not providing this feature. Suppose a scenario, the user is trying to search for "chai" (hindi meaning of tea). And in your index you have more documents containing the keyword "chat" as compared to the the keyword "chai". chat => 501 chai => 29 and the maxQueryFrequency is 25. So solr will suggest you chat as this term is present in more documents and if you want from solr to search internally for the suggestion then you will be displaying he results of chat to the user when the user intended to search for chai. So I suppose it is good to show wrong suggestion instead of providing the inappropriate results to the user. In this case you could the show the chat and chai as suggestion to the user and then he could select the appropriate suggestion. With Regards Aman Tandon On Sat, Mar 7, 2015 at 7:57 PM, Ashish Mukherjee wrote: > Hello, > > I have enabled the Spellcheck component in Solr, which gives me spelling > suggestions. However, I would like those suggestions to be applied in the > same select request handler to retrieve additional results based on the > suggestions. How can this be achieved with Solr? > > Regards, > Ashish >
Re: 4.10.4 - nodes up, shard without leader
Interesting bug.
First there is the already closed transaction log. That by itself deserves
a look. I'm not even positive we should be replaying the log we
reconnecting from ZK disconnect, but even if we do, this should never
happen.
Beyond that there seems to be some race. Because of the log trouble, we try
and cancel the election - but we don't find the ephemeral election node yet
for some reason and so just assume it's fine, no node there to remove
(well, we WARN, because it is a little unexpected). Then that ephemeral
node materializes I guess, and the new leader doesn't register because the
old leader won't give up the thrown. We don't try and force the new leader
because that may just hide bugs and cause data loss, we no leader is
elected.
I'd guess there are two JIRA issues to resolve here.
- Mark
On Sun, Mar 8, 2015 at 8:37 AM Markus Jelsma
wrote:
> Hello - i stumbled upon an issue i've never seen earlier, a shard with all
> nodes up and running but no leader. This is on 4.10.4. One of the two nodes
> emits the following error log entry:
>
> 2015-03-08 05:25:49,095 WARN [solr.cloud.ElectionContext] - [Thread-136] -
> : cancelElection did not find election node to remove
> /overseer_elect/election/93434598784958483-178.21.116.
> 225:8080_solr-n_000246
> 2015-03-08 05:25:49,121 WARN [solr.cloud.ElectionContext] - [Thread-136] -
> : cancelElection did not find election node to remove
> /collections/oi/leader_elect/shard3/election/93434598784958483-178.21.116.
> 225:8080_solr_oi_h-n_43
> 2015-03-08 05:25:49,220 ERROR [solr.update.UpdateLog] - [Thread-136] - :
> Error inspecting tlog
> tlog{file=/opt/solr/cores/oi_c/data/tlog/tlog.0001394
> refcount=2}
> java.nio.channels.ClosedChannelException
> at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:99)
> at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:679)
> at org.apache.solr.update.ChannelFastInputStream.
> readWrappedStream(TransactionLog.java:784)
> at org.apache.solr.common.util.FastInputStream.refill(
> FastInputStream.java:89)
> at org.apache.solr.common.util.FastInputStream.read(
> FastInputStream.java:125)
> at java.io.InputStream.read(InputStream.java:101)
> at org.apache.solr.update.TransactionLog.endsWithCommit(
> TransactionLog.java:218)
> at org.apache.solr.update.UpdateLog.recoverFromLog(
> UpdateLog.java:800)
> at org.apache.solr.cloud.ZkController.register(
> ZkController.java:841)
> at org.apache.solr.cloud.ZkController$1.command(
> ZkController.java:277)
> at org.apache.solr.common.cloud.ConnectionManager$1$1.run(
> ConnectionManager.java:166)
> 2015-03-08 05:25:49,225 ERROR [solr.update.UpdateLog] - [Thread-136] - :
> Error inspecting tlog
> tlog{file=/opt/solr/cores/oi_c/data/tlog/tlog.0001471
> refcount=2}
> java.nio.channels.ClosedChannelException
> at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:99)
> at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:679)
> at org.apache.solr.update.ChannelFastInputStream.
> readWrappedStream(TransactionLog.java:784)
> at org.apache.solr.common.util.FastInputStream.refill(
> FastInputStream.java:89)
> at org.apache.solr.common.util.FastInputStream.read(
> FastInputStream.java:125)
> at java.io.InputStream.read(InputStream.java:101)
> at org.apache.solr.update.TransactionLog.endsWithCommit(
> TransactionLog.java:218)
> at org.apache.solr.update.UpdateLog.recoverFromLog(
> UpdateLog.java:800)
> at org.apache.solr.cloud.ZkController.register(
> ZkController.java:841)
> at org.apache.solr.cloud.ZkController$1.command(
> ZkController.java:277)
> at org.apache.solr.common.cloud.ConnectionManager$1$1.run(
> ConnectionManager.java:166)
> 2015-03-08 12:21:04,438 WARN [solr.cloud.RecoveryStrategy] -
> [zkCallback-2-thread-28] - : Stopping recovery for core=oi_h coreNodeName=
> 178.21.116.225:8080_solr_oi_h
>
> The other node makes a mess in the logs:
>
> 2015-03-08 05:25:46,020 WARN [solr.cloud.RecoveryStrategy] -
> [zkCallback-2-thread-20] - : Stopping recovery for core=oi_c coreNodeName=
> 194.145.201.190:
> 8080_solr_oi_c
> 2015-03-08 05:26:08,670 ERROR [solr.cloud.ShardLeaderElectionContext] -
> [zkCallback-2-thread-19] - : There was a problem trying to register as the
> leader:org.
> apache.solr.common.SolrException: Could not register as the leader
> because creating the ephemeral registration node in ZooKeeper failed
> at org.apache.solr.cloud.ShardLeaderElectionContextBase
> .runLeaderProcess(ElectionContext.java:146)
> at org.apache.solr.cloud.ShardLeaderElectionContext.
> runLeaderProcess(ElectionContext.java:317)
> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(
> LeaderElector.java:163)
> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(
> LeaderElector.java:125)
> at org.apache.solr
Re: Solr TCP layer
On 3/8/2015 2:05 PM, Saumitra Srivastav wrote: > I want to start working on adding a TCP layer for client to node and > inter-node communication. > > I am not up to date on recent changes happening to Solr. So before I start > looking into code, I would like to know if there is already some work done > in this direction, which I can reuse. Are there any know > challenges/complexities? > > I would appreciate any help to kick start this effort. Also, what would be > the best way to discuss and get feedback on design from contributors? Open a > JIRA?? What follows is my mostly my opinion, interspersed with a few things that might be loosely called facts: I personally do not mind at all that Solr uses HTTP. Some people view it as an inefficient protocol, but the overhead is low on all but the slowest connections. On a LAN, it is probably not even worth discussing. Most of the time, there will be a LAN connection between the Solr client and the Solr server. I think there are two primary advantages to HTTP: *) Testing can be carried out in a browser with hand-typed URLs. *) We don't have to worry about writing the code to handle the underlying protocol. The first point makes testing throughout the Solr deployment process VERY easy. Expanding on the second point: There are well-tested and mostly bug-free HTTP client and server libraries available for us to use. We don't have to figure out how to write them, troubleshoot them, or optimize them. There are hundreds or thousands of other people using those libraries that can find and report bugs and inefficiencies, often including the fix in the bug report. We might be able to make Solr a little more efficient if we use our own TCP protocol instead of HTTP, but there are drawbacks. It takes a lot of experience to write a network protocol from scratch, and we are bound to make mistakes, mistakes that will cause users a LOT of problems. The authors of those libraries that I mentioned have already gone through that pain, often many times. Solr must be a standalone application to realistically use a custom TCP protocol directly. Although there are already loose plans in place to pull the HTTP and network layers into Solr and make it a standalone application, we are not there yet. If you want to work on a new protocol, feel free. If there is anything I can do to help in between my other duties, I will. I do not know of any existing work that you can re-use, although it's possible there might be something. You might look at the Zookeeper project for an existing implementation of a custom network protocol in Java. I suspect that it will take a very long time before any such work is as stable as HttpClient and the Servlet API (implemented by Jetty). Unless you can demonstrate that stability, it will not become the default protocol. Thanks, Shawn
Re: Solr TCP layer
I agree. I have some servers that use a TCP socket protocol and they are beastly hard to monitor, load-balance, all that stuff. HTTP rules. I need a really big advantage to recommend a non-HTTP server. Understand that I helped design at least one socket protocol, HP JetDirect. This was designed before HTTP, so I have an excuse. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) On Mar 8, 2015, at 8:15 PM, Shawn Heisey wrote: > On 3/8/2015 2:05 PM, Saumitra Srivastav wrote: >> I want to start working on adding a TCP layer for client to node and >> inter-node communication. >> >> I am not up to date on recent changes happening to Solr. So before I start >> looking into code, I would like to know if there is already some work done >> in this direction, which I can reuse. Are there any know >> challenges/complexities? >> >> I would appreciate any help to kick start this effort. Also, what would be >> the best way to discuss and get feedback on design from contributors? Open a >> JIRA?? > > > What follows is my mostly my opinion, interspersed with a few things > that might be loosely called facts: > > I personally do not mind at all that Solr uses HTTP. > > Some people view it as an inefficient protocol, but the overhead is low > on all but the slowest connections. On a LAN, it is probably not even > worth discussing. Most of the time, there will be a LAN connection > between the Solr client and the Solr server. > > I think there are two primary advantages to HTTP: > > *) Testing can be carried out in a browser with hand-typed URLs. > *) We don't have to worry about writing the code to handle the > underlying protocol. > > The first point makes testing throughout the Solr deployment process > VERY easy. > > Expanding on the second point: There are well-tested and mostly > bug-free HTTP client and server libraries available for us to use. We > don't have to figure out how to write them, troubleshoot them, or > optimize them. There are hundreds or thousands of other people using > those libraries that can find and report bugs and inefficiencies, often > including the fix in the bug report. > > We might be able to make Solr a little more efficient if we use our own > TCP protocol instead of HTTP, but there are drawbacks. It takes a lot > of experience to write a network protocol from scratch, and we are bound > to make mistakes, mistakes that will cause users a LOT of problems. The > authors of those libraries that I mentioned have already gone through > that pain, often many times. > > Solr must be a standalone application to realistically use a custom TCP > protocol directly. Although there are already loose plans in place to > pull the HTTP and network layers into Solr and make it a standalone > application, we are not there yet. > > If you want to work on a new protocol, feel free. If there is anything > I can do to help in between my other duties, I will. I do not know of > any existing work that you can re-use, although it's possible there > might be something. You might look at the Zookeeper project for an > existing implementation of a custom network protocol in Java. > > I suspect that it will take a very long time before any such work is as > stable as HttpClient and the Servlet API (implemented by Jetty). Unless > you can demonstrate that stability, it will not become the default protocol. > > Thanks, > Shawn >
Re: SpellCheck component query
Hi Aman, Thanks for your response. Taking your example further to elaborate what I am looking to do - if user types 'chai' and suggestion is 'chat' , then I would like to see all the values which 'chat' in them as suggestions, such as 'text chat', 'video chat', 'audio chat' etc. without making another search request for 'chat'. Can this be accomplished? Regards, Ashish On Mon, Mar 9, 2015 at 2:50 AM, Aman Tandon wrote: > Hi, > > AFAIK solr currently not providing this feature. > > Suppose a scenario, the user is trying to search for "chai" (hindi meaning > of tea). And in your index you have more documents containing the keyword > "chat" as compared to the the keyword "chai". > > chat => 501 > chai => 29 > > and the maxQueryFrequency is 25. > > So solr will suggest you chat as this term is present in more documents and > if you want from solr to search internally for the suggestion then you will > be displaying he results of chat to the user when the user intended to > search for chai. > > So I suppose it is good to show wrong suggestion instead of providing the > inappropriate results to the user. > > In this case you could the show the chat and chai as suggestion to the user > and then he could select the appropriate suggestion. > > With Regards > Aman Tandon > > On Sat, Mar 7, 2015 at 7:57 PM, Ashish Mukherjee < > ashish.mukher...@gmail.com > > wrote: > > > Hello, > > > > I have enabled the Spellcheck component in Solr, which gives me spelling > > suggestions. However, I would like those suggestions to be applied in the > > same select request handler to retrieve additional results based on the > > suggestions. How can this be achieved with Solr? > > > > Regards, > > Ashish > > >
Re: how to change configurations in solrcloud setup
Please help. With Regards Aman Tandon On Sat, Mar 7, 2015 at 9:58 PM, Aman Tandon wrote: > Hi, > > Please tell me what is best way to apply configuration changes in solr > cloud and how to do that. > > Thanks in advance. > > With Regards > Aman Tandon >
