Re: "Classic" 4.2 master-slave replication not completing
Okay, I have done this (updated to 4.3.1 across master and four slaves; one of these is my own PC for experiments, it is not being accessed by clients). Just had a minor replication this morning, and all three slaves are "stuck" again. Replication supposedly started at 8:40, ended 30 seconds later or so (on my local PC, set up identically to the other three slaves). The three slaves will NOT complete the roll-over to the new index. All three index folders have a write.lock and latest files are dated 8:40am (now it is 8:54am, with no further activity in the index folders). There exists an "index.2013062708461" (or some variation thereof) in all three slaves' data folder. The seemingly-relevant thread dump of a "snappuller" thread on each of these slaves: - sun.misc.Unsafe.park(Native Method) - java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281) - java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218) - java.util.concurrent.FutureTask.get(FutureTask.java:83) - org.apache.solr.handler.SnapPuller.openNewWriterAndSearcher(SnapPuller.java:631) - org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:446) - org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:317) - org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:223) - java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) - java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) - java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) - java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) - java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) - java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) - java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) - java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) - java.lang.Thread.run(Thread.java:662) Here they sit. My local PC "slave" replicated very quickly, switched over to the new generation (206) immediately. I am not sure why the three slaves are dragging on this. If there's any configuration elements or other details you need, please let me know. I can manually "kick" them by reloading the core from the admin pages, but obviously I would like this to be a hands-off process. Any help is greatly appreciated; this has been bugging me for some time now. On Mon, Jun 24, 2013 at 9:34 AM, Shalin Shekhar Mangar < shalinman...@gmail.com> wrote: > A bunch of replication related issues were fixed in 4.2.1 so you're > better off upgrading to 4.2.1 or later (4.3.1 is the latest release). > > On Mon, Jun 24, 2013 at 6:55 PM, Neal Ensor wrote: > > As a bit of background, we run a setup (coming from 3.6.1 to 4.2 > relatively > > recently) with a single master receiving updates with three slaves > pulling > > changes in. Our index is around 5 million documents, around 26GB in size > > total. > > > > The situation I'm seeing is this: occasionally we update the master, and > > replication begins on the three slaves, seems to proceed normally until > it > > hits the end. At that point, it "sticks"; there's no messages going on > in > > the logs, nothing on the admin page seems to be happening. I sit there > for > > sometimes upwards of 30 minutes, seeing no further activity in the index > > folder(s). After a while, I go to the core admin page and manually > reload > > the core, which "catches it up". It seems like the index readers / > writers > > are not releasing the index otherwise? The configuration is set to > reopen; > > very occasionally this situation actually fixes itself after a longish > > period of time, but it seems very annoying. > > > > I had at first suspected this to be due to our underlying shared (SAN) > > storage, so we installed SSDs in all three slave machines, and moved the > > entire indexes to those. It did not seem to affect this issue at all > > (additionally, I didn't really see the expected performance boost, but > > that's a separate issue entirely). > > > > Any ideas? Any configuration details I might share/reconfigure? Any > > suggestions are appreciated. I could also upgrade to the later 4.3+ > > versions, if that might help. > > > > Thanks! > > > > Neal Ensor > > nen...@gmail.com > > > > -- > Regards, > Shalin Shekhar Mangar. >
Re: "Classic" 4.2 master-slave replication not completing
is it conceivable that there's too much traffic, causing Solr to stall re-opening the searcher (thus releasing to the new index)? I'm grasping at straws, and this is beginning to bug me a lot. The traffic logs wouldn't seem to support this (apart from periodic health-check pings, the load is distributed fairly evenly across 3 slaves by a load-balancer tool). After 35+ minutes this morning, none of the three successfully "unstuck", and had to be manually core-reloaded. Is there perhaps a configuration element I'm overlooking that might make solr a bit less "friendly" about it, and just dump the searchers/reopen when replication completes? As a side note, I'm getting really frustrated with trying to get log4j logging on 4.3.1 set up; my tomcat container persists in complaining that it cannot find log4j.properties, when I've put it in the WEB-INF/classes of the war file, have SLF4j libraries AND log4j at the shared container "lib" level, and log4j.debug turned on. I can't find any excuses why it cannot seem to locate the configuration. Any suggestions or pointers would be greatly appreciated. Thanks! On Thu, Jun 27, 2013 at 10:35 AM, Mark Miller wrote: > Odd - looks like it's stuck waiting to be notified that a new searcher is > ready. > > - Mark > > On Jun 27, 2013, at 8:58 AM, Neal Ensor wrote: > > > Okay, I have done this (updated to 4.3.1 across master and four slaves; > one > > of these is my own PC for experiments, it is not being accessed by > clients). > > > > Just had a minor replication this morning, and all three slaves are > "stuck" > > again. Replication supposedly started at 8:40, ended 30 seconds later or > > so (on my local PC, set up identically to the other three slaves). The > > three slaves will NOT complete the roll-over to the new index. All three > > index folders have a write.lock and latest files are dated 8:40am (now it > > is 8:54am, with no further activity in the index folders). There exists > an > > "index.2013062708461" (or some variation thereof) in all three > slaves' > > data folder. > > > > The seemingly-relevant thread dump of a "snappuller" thread on each of > > these slaves: > > > > - sun.misc.Unsafe.park(Native Method) > > - java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) > > - > > > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) > > - > > > java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969) > > - > > > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281) > > - java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218) > > - java.util.concurrent.FutureTask.get(FutureTask.java:83) > > - > > > org.apache.solr.handler.SnapPuller.openNewWriterAndSearcher(SnapPuller.java:631) > > - > > > org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:446) > > - > > > org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:317) > > - org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:223) > > - > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) > > - > > > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) > > - java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) > > - > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) > > - > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) > > - > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) > > - > > > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) > > - > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) > > - java.lang.Thread.run(Thread.java:662) > > > > > > Here they sit. My local PC "slave" replicated very quickly, switched > over > > to the new generation (206) immediately. I am not sure why the three > > slaves are dragging on this. If there's any configuration elements or > > other details you need, please let me know. I can manually "kick" them > by > > reloading the core from the admin pages, but obviously I would like this > to > &g
Re: "Classic" 4.2 master-slave replication not completing
Posted the solr config up as http://apaste.info/4eKC (hope that works). Note that this is largely a hold-over from upgrades of previous solr versions, there may be lots of cruft left over. If it's advisable to do so, I would certainly be open to starting from scratch with a 4.3+ example configuration. I will take up the logging portions you listed there, to see where I might have missed a library or configuration point, thanks. On Mon, Jul 1, 2013 at 4:04 PM, Shawn Heisey wrote: > On 7/1/2013 1:07 PM, Neal Ensor wrote: > >> is it conceivable that there's too much traffic, causing Solr to stall >> re-opening the searcher (thus releasing to the new index)? I'm grasping >> at >> straws, and this is beginning to bug me a lot. The traffic logs wouldn't >> seem to support this (apart from periodic health-check pings, the load is >> distributed fairly evenly across 3 slaves by a load-balancer tool). After >> 35+ minutes this morning, none of the three successfully "unstuck", and >> had >> to be manually core-reloaded. >> >> Is there perhaps a configuration element I'm overlooking that might make >> solr a bit less "friendly" about it, and just dump the searchers/reopen >> when replication completes? >> > > Can you share your solrconfig.xml file, someplace like http://apaste.info? > Please be sure to choose the correct file type ... on that website it is > (X)HTML for an XML file. > > > As a side note, I'm getting really frustrated with trying to get log4j >> logging on 4.3.1 set up; my tomcat container persists in complaining that >> it cannot find log4j.properties, when I've put it in the WEB-INF/classes >> of >> the war file, have SLF4j libraries AND log4j at the shared container "lib" >> level, and log4j.debug turned on. I can't find any excuses why it cannot >> seem to locate the configuration. >> > > The wiki is still down for maintenance, so below is a relevant section of > the SolrLogging wiki page extracted from Google Cache. When it comes back > up, you can find it at this URL: > > http://wiki.apache.org/solr/**SolrLogging#Switching_from_** > Log4J_back_to_JUL_.28java.**util.logging.29<http://wiki.apache.org/solr/SolrLogging#Switching_from_Log4J_back_to_JUL_.28java.util.logging.29> > > = > The example logging setup takes over the configuration of Solr logging, > which prevents the container from controlling where logs go. Users of > containers other than the included Jetty (Tomcat in particular) may be > accustomed to doing the logging configuration in the container. If you want > to switch back to java.util.logging so this is once again possible, here's > what to do. These steps apply to the example/lib/ext directory in the Solr > example, or to your container's lib directory as mentioned in the previous > section. These steps also assume that the slf4j version is 1.6.6, which > comes with Solr4.3. Newer versions may use a different slf4j version. As of > May 2013, you can use a newer SLF4J version with no trouble, but be aware > that all slf4j components in your classpath must be the same version. > > Download slf4j version 1.6.6 (the version used in Solr4.3.x). > http://www.slf4j.org/dist/**slf4j-1.6.6.zip<http://www.slf4j.org/dist/slf4j-1.6.6.zip> > Unpack the slf4j archive. > Delete these JARs from your lib folder: slf4j-log4j12-1.6.6.jar, > jul-to-slf4j-1.6.6.jar, log4j-1.2.16.jar > Add these JARs to your lib folder (from slf4j zip): > slf4j-jdk14-1.6.6.jar, log4j-over-slf4j-1.6.6.jar > Use your old logging.properties > = > > Thanks, > Shawn > >
Re: "Classic" 4.2 master-slave replication not completing
I'll give that a shot, thanks! On Wed, Jul 3, 2013 at 12:28 PM, Shawn Heisey wrote: > On 7/3/2013 9:29 AM, Neal Ensor wrote: > >> Posted the solr config up as http://apaste.info/4eKC (hope that works). >> Note that this is largely a hold-over from upgrades of previous solr >> versions, there may be lots of cruft left over. If it's advisable to do >> so, I would certainly be open to starting from scratch with a 4.3+ example >> configuration. >> >> I will take up the logging portions you listed there, to see where I might >> have missed a library or configuration point, thanks. >> > > You have a LOT of auto warming in your cache configuration. Unless your > filter queries are really super simple, a more realistic autowarmCount for > the filterCache is low double digits or even single digit. If they are > super simple, a number like 64 might be OK. For the queryResultCache, I > don't think I would go above a few hundred at the absolute most. You > probably don't need overall cache sizes that large, but that shouldn't > cause any problems, unless it is pushing the limits of your java heap. > > The high numbers that you have mean that every commit with openSearcher > set to true (the default) will result in Solr running 4608 queries against > the new index before it can go online. If both master and slave servers > have similar config, it will happen on both. As you might imagine, this > many queries can take a very long time, which can lead to a whole cascade > of other problems, and I wouldn't be surprised to learn that replication is > affected. > > Try reducing the autowarmCount values drastically and see what happens. > > Thanks, > Shawn > >
Question about filter query: "half" of my index is slower than the other?
So, I have an oddball question I have been battling with in the last day or two. I have an 8 million document solr index, roughly divided down the middle by an identifying "product" value, one of two distinct values. The documents in both "sides" are very similar, with stored text fields, etc. I have two nearly identical request handlers, one for each "side". When I perform very similar queries on either "side" for random phrases, requesting 500 rows with highlighting on titles and summaries, I get very different results. One "side" consistently returns results in around 1-2 seconds, whereas the other one consistently returns in 6-10 seconds. I don't see any reason why it's worse; each run of queries is deliberately randomized to avoid caches getting in the way. Each test query returns the full first 500 in most cases. My filter query cache configuration looks like: (desperately trying to increase it, hoping this would help). The other caches are quite small; the use cases the customer is dealing with don't involve much in the way of paging, just returning a large initial set with highlighting in the shortest time. I'm trying to optimize this down so the disparity between the two "halves" is not so dramatic. Is there any optimizations or things I should be looking for to tune? Is it just the "way it is"? I've tried to argue to decrease the return set size, turn off highlighting, etc., but these seem to be out of the question. I would at least like some concrete reason why one filter query would be so relatively out of whack than the other, given the document ranges are very nearly half (3.8 million vs. 4.0 million in the slower side). Any pointers or suggestions would be appreciated. Thanks in advance. Neal Ensor nen...@gmail.com
Re: Question about filter query: "half" of my index is slower than the other?
It seems (from observation only) that most of the documents on both sides of this equation have the same "weights". I don't see any wide swaths of unpopulated fields on the "good" side. Just wondering if there's some caching involved that I'm missing here, or something I can balance out better... On Fri, Aug 9, 2013 at 11:39 AM, Raymond Wiker wrote: > On Aug 9, 2013, at 17:36 , Neal Ensor wrote: > > So, I have an oddball question I have been battling with in the last day > or > > two. > > > > I have an 8 million document solr index, roughly divided down the middle > by > > an identifying "product" value, one of two distinct values. The > documents > > in both "sides" are very similar, with stored text fields, etc. I have > two > > nearly identical request handlers, one for each "side". > > > > When I perform very similar queries on either "side" for random phrases, > > requesting 500 rows with highlighting on titles and summaries, I get very > > different results. One "side" consistently returns results in around 1-2 > > seconds, whereas the other one consistently returns in 6-10 seconds. I > > don't see any reason why it's worse; each run of queries is deliberately > > randomized to avoid caches getting in the way. Each test query returns > the > > full first 500 in most cases. > > > > My filter query cache configuration looks like: > > > > > size="75" > > initialSize="1" > > autowarmCount="0"/> > > > > (desperately trying to increase it, hoping this would help). The other > > caches are quite small; the use cases the customer is dealing with don't > > involve much in the way of paging, just returning a large initial set > with > > highlighting in the shortest time. > > > > I'm trying to optimize this down so the disparity between the two > "halves" > > is not so dramatic. Is there any optimizations or things I should be > > looking for to tune? Is it just the "way it is"? I've tried to argue to > > decrease the return set size, turn off highlighting, etc., but these seem > > to be out of the question. I would at least like some concrete reason > why > > one filter query would be so relatively out of whack than the other, > given > > the document ranges are very nearly half (3.8 million vs. 4.0 million in > > the slower side). > > > > Any pointers or suggestions would be appreciated. Thanks in advance. > > > > Neal Ensor > > nen...@gmail.com > > Does one side have mcuh more data in one of the fields that is being > returned?
SOLR 4.2, slaves replicating reporting higher version number than master
Having a strange intermittent issue with my 1 master, 3 slave solr 4.2 setup. On occasion, after indexing the master and replicating across the three slaves, each slave will start reporting they are one generation ahead (525 vs. 524 on the master) and thus out of sync. Replication runs appear to do nothing, and it seems to not really be affecting performance, it's just tickling my admin nerves. Any suggestions of what to look at? Just upgrade solr perhaps? 4.2 might be getting rather old...
Re: SOLR 4.2, slaves replicating reporting higher version number than master
Looks much like what I'm encountering. Guessing that will go away once I update solr, just wanted to make sure it wasn't a real bug. Entirely possible we are getting some "empty commits" given the nature of the index maintenance. Thanks for the pointer! On Mon, Sep 16, 2013 at 2:00 PM, Chris Hostetter wrote: > > Sounds like perhaps you are getting confused by this... > > https://issues.apache.org/jira/browse/SOLR-4661 > > ...if that is the situation then it's not a bug you need to worry about, > just a confusion in how the ReplicaitonHandler reports it's stats -- the > newer UI makes it more clear what numbers you are looking at. > > If that doesn't looke like the problem you are seeing, then more detail on > how to reproduce what you are seeing would be helpful (replicaiton > configs, logs from amster & slave, etc...) > > > -Hoss >
"Classic" 4.2 master-slave replication not completing
As a bit of background, we run a setup (coming from 3.6.1 to 4.2 relatively recently) with a single master receiving updates with three slaves pulling changes in. Our index is around 5 million documents, around 26GB in size total. The situation I'm seeing is this: occasionally we update the master, and replication begins on the three slaves, seems to proceed normally until it hits the end. At that point, it "sticks"; there's no messages going on in the logs, nothing on the admin page seems to be happening. I sit there for sometimes upwards of 30 minutes, seeing no further activity in the index folder(s). After a while, I go to the core admin page and manually reload the core, which "catches it up". It seems like the index readers / writers are not releasing the index otherwise? The configuration is set to reopen; very occasionally this situation actually fixes itself after a longish period of time, but it seems very annoying. I had at first suspected this to be due to our underlying shared (SAN) storage, so we installed SSDs in all three slave machines, and moved the entire indexes to those. It did not seem to affect this issue at all (additionally, I didn't really see the expected performance boost, but that's a separate issue entirely). Any ideas? Any configuration details I might share/reconfigure? Any suggestions are appreciated. I could also upgrade to the later 4.3+ versions, if that might help. Thanks! Neal Ensor nen...@gmail.com
SOLR 4.1 and Tomcat 6 class loader issue (older lucene implementation in common lib)
So, I have had great success as a SOLR user so far, love its ease of set up. However, I have run into this problem (and this may be more of a Tomcat error; if so, please point me in that direction). My tomcat application containers have an older lucene library in the common library "lib" folder (2.9) to support some deployed legacy searching applications. The SOLR I'm trying to install in this container has in its .war file WEB-INF/lib all the 4.1 lucene libraries, which if I read the class loader documentation correctly, should override the shared library in tomcat. In fact, when I deploy it, this is the case: the dashboard shows using 4.1.0 lucene implementation. However, if I stop and start the application while the container is running, it CHANGES to 2.9! This mysterious switcheroo caused my slaves to fail after replicating mysteriously over the weekend, as they had been bounced (and thus 2.9) and had received a 4.1 lucene index from the master, and promptly failed hard. Watching one interactively showed it replicating most of an index, then getting an "invalid master" error about 75% in, then going down. Any clues as to why this might be? Should Tomcat be ignoring delegation here, or is it in error? My solution so far has been to move my solr installations off onto "pristine" containers without the lucene library anywhere except the .war file itself. This presents slight problems in our server farm arrangements, and I'd like to not have to "special case" it if possible. Any pointers would be great (even if its to point me at Tomcat's mailing list. :) ) Thanks! Neal Ensor nen...@gmail.com
SOLR 4.2 SNAPSHOT "There exists no core with name x"
Just pulled down the latest SVN and built it in place to try and upgrade from 4.1 (having replication issues). However, bringing it up with my current core (copied) presents me with the admin page, but an empty drop-down to select specific cores. If I go to "Core Admin" I can see the core, I can issue queries against it, but I cannot go to its statistics, interactive query page, etc. at all. Attempting to access the "old" core admin stuff via direct URL results in the pop-up stating "There exists no core with name myname", even though I can clearly see it there. I even created a brand new core via the core admin interface, and it cannot be seen either. Restarted the container, examined logs, etc. to no avail. Am I missing something obvious here? Any serious deviations between 4.1 and 4.2 (trunk) configuration I need to update? Using tomcat 6.0.36 as a container, freshly-built solr-4.2-SNAPSHOT.war hot off the compiler. Neal Ensor nen...@gmail.com
Re: SOLR 4.2 SNAPSHOT "There exists no core with name x"
Using Chrome (latest) on Mac OSX 10.8.2. In 4.1.0, I accessed cores via something like: http://machine:port/solr/#/corename and got to the Ping, Query, Schema, etc. I attempted a similar URL with my local installation and got the error I mentioned. I have one core locally named "master", and made a new one named "testcore" via the Cores Admin. Neither could be accessed directly (but could be searched via the select query URL) yet came up on the Cores Admin page fine. The drop-down under the main menu remains empty. Now that you mentioned browser versions, I JUST tried it using Safari, and it actually came up; there might be some java script thing that's giving Chrome indigestion perhaps. Looking at the Javascript Console in Chrome I don't see any errors. The apparently relevant portion of the source on that page looks like empty. If I can help with any other information, please let me know. Again, it appears to work on Safari fine hitting the same container, so must be something Chrome-specific (perhaps something I have disabled?) On Fri, Mar 1, 2013 at 1:25 PM, Stefan Matheis wrote: > Hey Neal > > We changed the Navigation after 4.1 was released > (https://issues.apache.org/jira/browse/SOLR-4284) But you're the first one > reporting problems with this change (if it's related). > > 1) What Browser on which OS are you using? > 2) And what is/are the Name of your core(s)? > 3) When you're talking about the "old" core admin, what is the url you're > trying to access? > > Perhaps you can as well provide Screenshots of the Core-Admin and the > Drop-Down (left side in the Navigation) while it's expanded? > > Stefan > > > On Friday, March 1, 2013 at 6:57 PM, Neal Ensor wrote: > >> Just pulled down the latest SVN and built it in place to try and >> upgrade from 4.1 (having replication issues). However, bringing it up >> with my current core (copied) presents me with the admin page, but an >> empty drop-down to select specific cores. If I go to "Core Admin" I >> can see the core, I can issue queries against it, but I cannot go to >> its statistics, interactive query page, etc. at all. Attempting to >> access the "old" core admin stuff via direct URL results in the pop-up >> stating "There exists no core with name myname", even though I can >> clearly see it there. >> >> I even created a brand new core via the core admin interface, and it >> cannot be seen either. Restarted the container, examined logs, etc. >> to no avail. Am I missing something obvious here? Any serious >> deviations between 4.1 and 4.2 (trunk) configuration I need to update? >> Using tomcat 6.0.36 as a container, freshly-built >> solr-4.2-SNAPSHOT.war hot off the compiler. >> >> Neal Ensor >> nen...@gmail.com (mailto:nen...@gmail.com) > > >
Re: SOLR 4.2 SNAPSHOT "There exists no core with name x"
Actually, just updated Chrome this morning, and it all appears to work. Flushed cache as well, so could be part of that. All's well that ends well I suppose. neal On Mon, Mar 4, 2013 at 4:44 AM, Jens Grivolla wrote: > On 03/01/2013 07:46 PM, Neal Ensor wrote: >> >> Again, it appears to work on Safari fine hitting the same container, >> so must be something Chrome-specific (perhaps something I have >> disabled?) > > > This sounds like it might just be a browser cache issue (if you used Chrome > to access the same URL previously with the old Solr version installed). It > might just not be refreshing everything. > > Jens >
Re: SOLR 4.2 SNAPSHOT "There exists no core with name x"
Confirmed it was indeed caching, as I just updated my live master from 4.1 to the 4.2 snap shot and got the empty drop-down; cleared cache and reloaded, and working. On Mon, Mar 4, 2013 at 10:03 AM, Stefan Matheis wrote: > Thanks Jens! Didn't think about caching .. :/ > > Perhaps we should change the requests in favor of > https://issues.apache.org/jira/browse/SOLR-4311 to avoid any caching at the > UI? Results maybe in a few more (real) requests but i guess that would be > okay? > > Stefan > > > On Monday, March 4, 2013 at 2:21 PM, Neal Ensor wrote: > >> Actually, just updated Chrome this morning, and it all appears to >> work. Flushed cache as well, so could be part of that. All's well >> that ends well I suppose. >> >> neal >> >> On Mon, Mar 4, 2013 at 4:44 AM, Jens Grivolla > (mailto:j+...@grivolla.net)> wrote: >> > On 03/01/2013 07:46 PM, Neal Ensor wrote: >> > > >> > > Again, it appears to work on Safari fine hitting the same container, >> > > so must be something Chrome-specific (perhaps something I have >> > > disabled?) >> > >> > >> > >> > >> > This sounds like it might just be a browser cache issue (if you used Chrome >> > to access the same URL previously with the old Solr version installed). It >> > might just not be refreshing everything. >> > >> > Jens > >
SOLR 4.2 (snapshot, March 1, 2013): replication not "finishing"?
I'm getting intermittent issues with replication in my current arrangement: one master, 3 slaves; all the same SOLR version/war file deployment. I update the master, which kicks off replication across the other three; however, they never seem to "finish". In the data/ folders I get an empty index.timestamp folder, the admin page for replication shows it "stuck" pulling a file (no progress shown, just constant refresh). The index never changes over (always claiming to be out-of-date). Abort messages are ignored, both from the admin console and through a curl "abortfetch" request to the slaves. The searcher is still responsive, I can query, but the master's changes are of course not there. If I kill my container (tomcat 6) and start it back up, magically the replication has "finished" and the slave is up-to-date. This sort of leads me to believe something isn't finalizing the change over and opening a new searcher on the index (looks like the "main" index is actually being updated, but a close/open, or reopen, is not happening?) The relevant thread dump when in that state seems to be: snapPuller-7-thread-1 (12) java.util.concurrent.FutureTask$Sync@55ccfb48 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt (AbstractQueuedSynchronizer.java:811) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly (AbstractQueuedSynchronizer.java:969) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly (AbstractQueuedSynchronizer.java:1281) java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218) java.util.concurrent.FutureTask.get(FutureTask.java:83) org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:655) org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:466) org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:281) org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:223) java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101 (ScheduledThreadPoolExecutor.java:98) java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic (ScheduledThreadPoolExecutor.java:180) java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run (ScheduledThreadPoolExecutor.java:204) java.util.concurrent.ThreadPoolExecutor$Worker.runTask (ThreadPoolExecutor.java:895) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) java.lang.Thread.run(Thread.java:662) Tomcat containers are all the same; each of the slaves is running entirely alone on its own container, separate machines. SOLR reported versions: Versions solr-spec 4.2.0.2013.03.01.10.10.50 solr-impl 4.2-SNAPSHOT 1451604 - ensorn - 2013-03-01 10:10:50 lucene-spec 4.2-SNAPSHOT lucene-impl 4.2-SNAPSHOT 1451604 - ensorn - 2013-03-01 10:02:53 Any help would be appreciated. This is getting very frustrating. To make things worse, I have set up a new "slave" on my work PC (Mac), and it replicates FLAWLESSLY on the same set up; only difference is the slaves on the servers are on a SAN array (not sure if locking is causing the heartburn?) Any pointers would be great. This is obviously becoming a pain to work with, even with fairly infrequent replications. Thanks! Neal Ensor nen...@gmail.com
