Solr Replication during Tomcat shutdown causes shutdown to hang/fail
Hello,
I have solr 4.10 running on tomcat 7. I'm doing replication from one master
to about 10 slaves, with standard configuration:
${enable.master:false}
commit
startup
schema.xml,stopwords.txt
${enable.slave:false}
http://master:8080/solr/mycore
00:00:60
It appears that if tomcat gets shutdown while solr is replicating, it
prevents tomcat from shutting down fully. Immediately after receiving the
shutdown command, a thread dump is logged into catalina.out (this may have
been turned on by some configuration someone else on my team made). I
removed some threads that didn't look related, mostly about tomcat session
replication, or with names like "http-bio-8080-exec-10".
62252 [http-bio-8080-exec-1] INFO org.apache.solr.core.SolrCore –
[mycore] webapp=/solr path=/replication
params={command=details&_=1412014928648&wt=json} status=0 QTime=6
63310 [http-bio-8080-exec-1] INFO org.apache.solr.core.SolrCore –
[mycore] webapp=/solr path=/replication
params={command=details&_=1412014929699&wt=json} status=0 QTime=6
2014-09-29 14:22:10
Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.65-b04 mixed
mode):
"fsyncService-12-thread-1" prio=10 tid=0x7f3bd4002000 nid=0x203d
waiting on condition [0x7f3c271f]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0007e1ff4458> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
"explicit-fetchindex-cmd" daemon prio=10 tid=0x7f3c0413e800
nid=0x203c runnable [0x7f3c272f1000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:152)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.read(AbstractSessionInputBuffer.java:198)
at
org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:174)
at
org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:137)
at
org.apache.solr.common.util.FastInputStream.readWrappedStream(FastInputStream.java:80)
at
org.apache.solr.common.util.FastInputStream.read(FastInputStream.java:114)
at
org.apache.solr.common.util.FastInputStream.readFully(FastInputStream.java:152)
at
org.apache.solr.handler.SnapPuller$DirectoryFileFetcher.fetchPackets(SnapPuller.java:1239)
at
org.apache.solr.handler.SnapPuller$DirectoryFileFetcher.fetchFile(SnapPuller.java:1187)
at
org.apache.solr.handler.SnapPuller.downloadIndexFiles(SnapPuller.java:774)
at
org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:424)
at
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:337)
at
org.apache.solr.handler.ReplicationHandler$1.run(ReplicationHandler.java:227)
"process reaper" daemon prio=10 tid=0x7f3c0409c000 nid=0x203b
waiting on condition [0x7f3c984e9000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0007dfbfd890> (a
java.util.concurrent.SynchronousQueue$TransferStack)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
"process reaper" daemon prio=10 tid=0x7f3c0409a000 nid=0x2039
waiting on condition [0x7f3cac04]
java.lang.Thread.
Stemming breaks highlighting?
Hi all, I changed some of my fields from text_general to text_en, hoping to take advantage of stemming and some other improvements, but unfortunately the change has broken highlighting. It seems that it only wants to highlight non-stemmed words (i.e. words whose stemmed version is the same as the word itself, like "child"). I'm using the default fieldType definition: And enabling highlighting with hl.fl=title&hl=true in my query. This is also a faceted search, if that matters. In this case, as I said, only unstemmed words like "child" are highlighted. If I remove the stemming filter from the index analyzer (only; the query analyzer seems to have no effect) in the text_en definition, all matched words except stopwords are highlighted. Furthermore, if I change text_en to use the EnglishMinimalStemFilterFactory, more words are highlighted, which I assume is because they are stemmed by the Porter stemmer but not by this one. An example of such a word is "strides". Does anyone know what's going on? Thanks,Nick
Re: Stemming breaks highlighting?
Hey Ahmet, Thanks for the quick response. Yes, I reindexed; to be sure I just wiped out the whole data directory and made a fresh index again. Nick On Fri, Oct 10, 2014 at 3:20 PM, Ahmet Arslan wrote: > Hi Nick, > > Did you re-index after schema change? > > Ahmet > > > > On Friday, October 10, 2014 10:19 PM, Nicholas Violi < > nvi...@globalgiving.org> wrote: > Hi all, > > I changed some of my fields from text_general to text_en, hoping to take > advantage of stemming and some other improvements, but unfortunately the > change has broken highlighting. It seems that it only wants to highlight > non-stemmed words (i.e. words whose stemmed version is the same as the word > itself, like "child"). > I'm using the default fieldType definition: > positionIncrementGap="100"> class="solr.StandardTokenizerFactory"/> class="solr.StopFilterFactory" ignoreCase="true" > words="lang/stopwords_en.txt" /> class="solr.LowerCaseFilterFactory"/> class="solr.EnglishPossessiveFilterFactory"/> class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/> >type="query"> > ignoreCase="true" expand="true"/> class="solr.StopFilterFactory" ignoreCase="true" > words="lang/stopwords_en.txt" /> class="solr.LowerCaseFilterFactory"/> class="solr.EnglishPossessiveFilterFactory"/> class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/> > > And enabling highlighting with hl.fl=title&hl=true > in my query. This is also a faceted search, if that matters. > In this case, as I said, only unstemmed words like "child" are highlighted. > If I remove the stemming filter from the index analyzer (only; the query > analyzer seems to have no effect) in the text_en definition, all matched > words except stopwords are highlighted. Furthermore, if I change text_en to > use the EnglishMinimalStemFilterFactory, more words are highlighted, which > I assume is because they are stemmed by the Porter stemmer but not by this > one. An example of such a word is "strides". > Does anyone know what's going on? > Thanks,Nick > >
