Solr 5 upgrade

[richardg]

Ubuntu 14.04.02
Trying to install solr 5 following this:
https://cwiki.apache.org/confluence/display/solr/Upgrading+a+Solr+4.x+Cluster+to+Solr+5.0

I keep getting "this script requires extracting a war file with either the
jar or unzip utility, please install these utilities or contact your
administrator for assistance." after running install_solr_service.sh.  It
says "Service solr installed." but when I try to run the service I get the
above error.  Not sure the resolution.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-5-upgrade-tp4192127.html
Sent from the Solr - User mailing list archive at Nabble.com.

SOLR URL

[richardg]

I'm in the process of upgrading to Solr 5 from Solr 4* using Tomcat w/
multiple webapps.  When setting up a SOLR 5 Node is it possible to change
the URL From localhost:8080/solr to localhost:8080/whateverIwant ?  

Another issue to consider w/ this I will have multiple SOLR instances.

The way I envision it working is:

localhost:8080/solr1

localhost:8081/solr2

Or is there a better way to have multiple SOLR instances on the same server. 
Like I said before under Tomcat it was just multiple webapps.

Thanks,

Rich



--
View this message in context: 
http://lucene.472066.n3.nabble.com/SOLR-URL-tp4194218.html
Sent from the Solr - User mailing list archive at Nabble.com.

Solr 7* Sorry, no dataimport-handler defined

[richardg]

I see where this was an issue w/ 6.4 and fixed.  I keep getting this error w/
7.0.1 and 7.1.0.  Works fine up until 6.6.2.  Could this issue have been
reintroduced?  Is there somewhere to check what might be going on?  I don't
see anything in the error logs.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr 7* Sorry, no dataimport-handler defined

[richardg]

Yes I am referring to the dataimport tab in the admin UI and issue
SOLR-10035.  My previous setup w/ 6.3 did not show this error.  I then
upgraded to 7.1.0 and the error shows.  I upgraded(downgraded) to versions
6.5.0 and 6.6.2 and I do not see the error.  Version 7.0.1 also shows the
error for me.  I am currently using version 6.6.2 and have been successfully
able to run a data import from the admin UI. 

In my config directory we have 

solcore.properties
solrconfig.xml which defines the dataimport handler (data-config.xml)
schema.xml
dataimport.properties
data-config.xml
some admin-extra*.html files

We copy all the config files over to the slave instances and they do no show
this behavior on 7.1.0, dataimport tab loads fine.  The only thing I notice
is on the slaves I see entries like this in the log:

2017-11-07 13:36:11.200 INFO  (qtp2053591126-35) [  
x:solr_aggregate_production] o.a.s.c.S.Request [solr_aggregate_production] 
webapp=/solr path=/admin/mbeans params={cat=QUERY&wt=json&_=1510061783971}
status=0 QTime=0

vs on the master that shows the error.

2017-11-07 13:29:14.131 INFO  (qtp1839206329-36) [  
x:solr_aggregate_production] o.a.s.c.S.Request [solr_aggregate_production] 
webapp=/solr path=/admin/mbeans
params={cat=QUERYHANDLER&wt=json&_=1510061366718} status=0 QTime=2

I see just "QUERY" in the slave that is working and "QUERYHANDLER" in the
master that isn't.  This is why I referenced the issue w/ 6.4 (SOLR-10035). 
Other than that I do not see anything in the log showing and error for the
dataimport handler.

Thanks



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Solr 4.3 and above core swap

[richardg]

Since upgrading to solr 4.3 we get the following errors on our slaves when we
swap cores on our master and is still an issue on 4.4:

Solr index directory
'/usr/local/solr_aggregate/solr_aggregate/data/index.20130513152644966' is
locked.  Throwing exception

SEVERE: Unable to reload core: production
org.apache.solr.common.SolrException: Index locked for write for core
production

SEVERE: Could not reload core 
org.apache.solr.common.SolrException: Unable to reload core: production

On older solr versions it would create a new index.* directory and use it,
it hasn't been the case w/ 4.3+.  The new core seems to replicate fine and
the new index files are in the original index.* directory so I'm not sure
what is happening.




--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-4-3-and-above-core-swap-tp4084794.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr 4.3 and above core swap

[richardg]

Thanks for the reply, no nothing else would be writing to the index, I'm sure
it is a solrconfig setting but not sure which.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-4-3-and-above-core-swap-tp4084794p4085091.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr 4.3 and above core swap

[richardg]

  

I commented out the lockType, so it should be using the default of native
according to the documentation.  

There is nothing special about our file systems.

Thanks



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-4-3-and-above-core-swap-tp4084794p4085456.html
Sent from the Solr - User mailing list archive at Nabble.com.

4.3 logging setup

[richardg]

On all prior index version I setup my log via the logging.properties file in
/usr/local/tomcat/conf, it looked like this:

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

handlers = 1catalina.org.apache.juli.FileHandler,
2localhost.org.apache.juli.FileHandler,
3manager.org.apache.juli.FileHandler,
4host-manager.org.apache.juli.FileHandler, java.util.logging.ConsoleHandler

.handlers = 1catalina.org.apache.juli.FileHandler,
java.util.logging.ConsoleHandler


# Handler specific properties.
# Describes specific configuration info for Handlers.


1catalina.org.apache.juli.FileHandler.level = WARNING
1catalina.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
1catalina.org.apache.juli.FileHandler.prefix = catalina.

2localhost.org.apache.juli.FileHandler.level = FINE
2localhost.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
2localhost.org.apache.juli.FileHandler.prefix = localhost.

3manager.org.apache.juli.FileHandler.level = FINE
3manager.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
3manager.org.apache.juli.FileHandler.prefix = manager.

4host-manager.org.apache.juli.FileHandler.level = FINE
4host-manager.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
4host-manager.org.apache.juli.FileHandler.prefix = host-manager.

java.util.logging.ConsoleHandler.level = FINE
java.util.logging.ConsoleHandler.formatter =
java.util.logging.SimpleFormatter



# Facility specific properties.
# Provides extra control for each logger.


org.apache.catalina.core.ContainerBase.[Catalina].[localhost].level = INFO
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].handlers =
2localhost.org.apache.juli.FileHandler

org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/manager].level
= INFO
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/manager].handlers
= 3manager.org.apache.juli.FileHandler

org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/host-manager].level
= INFO
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/host-manager].handlers
= 4host-manager.org.apache.juli.FileHandler

# For example, set the org.apache.catalina.util.LifecycleBase logger to log
# each component that extends LifecycleBase changing state:
#org.apache.catalina.util.LifecycleBase.level = FINE

# To see debug messages in TldLocationsCache, uncomment the following line:
#org.apache.jasper.compiler.TldLocationsCache.level = FINE

After upgrading to 4.3 today the files defined aren't being logged to.  I
know things have changed for logging w/ 4.3 but how can I get it setup like
it was before?



--
View this message in context: 
http://lucene.472066.n3.nabble.com/4-3-logging-setup-tp4061875.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: 4.3 logging setup

[richardg]

Thanks for responding.  My issue is I've never changed anything w/ logging, I
have always used the built in Juli.  I've never messed w/ any jar files,
just had edit the logging.properties file.  I don't know where I would get
the jars for juli or where to put them, if that is what is needed.  I had
read what you posted before I just can't make any sense of it.

Thanks



--
View this message in context: 
http://lucene.472066.n3.nabble.com/4-3-logging-setup-tp4061875p4061901.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: 4.3 logging setup

[richardg]

I had already copied those jars over and gotten the app to start(it wouldn't
without them).  I was able configure solf4j/log4j logging using the
log4j.properties in the /lib folder to start logging but I don't want to
switch.  I have alerts set on the wording that the juli logging puts out but
everything I've tried to get it to work has failed.  I have older
indexes(4.2 and under) running on the server that are still able to log
correctly, it is just 4.3.  I am obviously missing something.

Thanks



--
View this message in context: 
http://lucene.472066.n3.nabble.com/4-3-logging-setup-tp4061875p4061907.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: 4.3 logging setup

[richardg]

These are the files I have in my /lib folder:

slf4j-api-1.6.6
log4j-1.2.16
jul-to-slf4j-1.6.6
jcl-over-slf4j-1.6.6
slf4j-jdk14-1.6.6
log4j-over-slf4j-1.6.6

Currently everything seems to be logging like before.  After I followed the
instructions in Jan's post replacing slf4j-log4j12-1.6.6.jar with this
slf4j-jdk14-1.6.6.jar it all started working.  Shawn I then removed
everything as you instructed and put in just  log4j-over-slf4j-1.6.6.jar and
slf4j-jdk14-1.6.6.jar but the index showed an error and wouldn't start.  So
that is why I have those 6 files in there now, I'm not sure if
log4j-over-slf4j-1.6.6.jar this file is needed or not.  Let me know if you
need me to test anything else.

Thanks



--
View this message in context: 
http://lucene.472066.n3.nabble.com/4-3-logging-setup-tp4061875p4061922.html
Sent from the Solr - User mailing list archive at Nabble.com.

Solr 4.3 core swap

[richardg]

Since upgrading to solr 4.3 we get the following errors on our slaves when we
swap cores on our master:

Solr index directory
'/usr/local/solr_aggregate/solr_aggregate/data/index.20130513152644966' is
locked.  Throwing exception

SEVERE: Unable to reload core: production
org.apache.solr.common.SolrException: Index locked for write for core
production

SEVERE: Could not reload core 
org.apache.solr.common.SolrException: Unable to reload core: production

On older solr versions it would create a new index.* directory and use it,
it hasn't been the case w/ 4.3.  The new core seems to replicate fine and
the new index files are in the original index.* directory so I'm not sure
what is happening.

Thanks



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-4-3-core-swap-tp4063065.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: High Slave CPU Intermittently After Replication

[richardg]

 
 
 
 
 
 
 
 

This is from the last time we had issues around 12:00PM yesterday, I have
since added in some cache warming and haven't had the issue since but have
gone some time before w/out any issue before.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/High-Slave-CPU-Intermittently-After-Replication-tp4020520p4021363.html
Sent from the Solr - User mailing list archive at Nabble.com.

Solr 4 Admin UI Dashboard Not Populating

[richardg]

Our Admin UI Dashboard is not populating on one of our servers, not sure if
it is a permission issue or what.  We have three others that it is working
on.


 




--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-4-Admin-UI-Dashboard-Not-Populating-tp4021602.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr 4 Admin UI Dashboard Not Populating

[richardg]

I was able to figure it out, I ran solr/admin/system?wt=xml and noticed that
the host entry was blank.  Our servers are Linux so I looked at /etc/hosts
file and noticed it was messed up.  I made the change and everything is
populating now.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-4-Admin-UI-Dashboard-Not-Populating-tp4021602p4021676.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: High Slave CPU Intermittently After Replication

[richardg]

We started having high load again today a few times today, each time looking
at SPM monitor our filter cache starts having high lookups and low hit rate,
this is our filtercache setting:



would it be possibly too slow?

here is the graph when it happens:

 



--
View this message in context: 
http://lucene.472066.n3.nabble.com/High-Slave-CPU-Intermittently-After-Replication-tp4020520p4022464.html
Sent from the Solr - User mailing list archive at Nabble.com.

Excluding caching of queryresult

[richardg]

We are aware of adding cache=false to our queries but everything we see seems
to reference filterCache.  We weren't sure if this parameter would work the
same way with the queryResultsCache.  Here is an example of a query we don't
want to cache(NOTE: I'm just the administrator, I'm not so familiar w/ query
construction):

Currently it is constructed as: &fq=doc_type:%22Cast%22&q=id:48880

We want to rewrite as: &q=*:*&fq={!cache=false}(doc_type:"Cast" AND
id:"48880")  

As such I believe it would not cache the filter, would this also not cache
the queryResult or would it need to be written another way:


&q={!cache=false}(*:*)&fq={!cache=false}(doc_type:"Cast" AND id:"48880")  

Or is it not possible to exclude thing from the queryCache?




--
View this message in context: 
http://lucene.472066.n3.nabble.com/Excluding-caching-of-queryresult-tp4023105.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Excluding caching of queryresult

[richardg]

Thanks Erick, I just found this ticket implying that is able to be used by
the main query also:

https://issues.apache.org/jira/browse/SOLR-2429
  

As you stated queryResultCache is cheap, I guess it would nice to get a
definitive answer and example as it would be used for other caches.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Excluding-caching-of-queryresult-tp4023105p4023233.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Excluding caching of queryresult

[richardg]

Using cache=false seems to Not be caching the query result. I ran queries
against our master server that doesn't get web traffic with and without the
parameter and would only notice inserts when the parameter wasn't included.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Excluding-caching-of-queryresult-tp4023105p4023241.html
Sent from the Solr - User mailing list archive at Nabble.com.

Solr 4.0 to Solr 4.1 upgrade

[richardg]

I upgrade one of my slaves by replacing the solr.war, all other slaves and
master were still 4.0.  When I started to monitor it w/ SPM I noticed that
the request rate was way up while the request count was way down.  I've
since put back the solr.war for 4.0 and the slave has returned to normal.  

I'm concerned if I upgraded my whole setup to 4.1 performance will suffer.

 



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-4-0-to-Solr-4-1-upgrade-tp4044990.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr 4.0 to Solr 4.1 upgrade

[richardg]

Otis,

I noticed this in my logs repeatedly during that time period:

Mar 5, 2013 1:28:00 PM org.apache.solr.core.CachingDirectoryFactory close
INFO: Releasing
directory:/usr/local/solr_aggregate/solr_aggregate/data/index

It wasn't in my logs any other time.

I found this:

https://issues.apache.org/jira/browse/SOLR-4200

Not sure what would cause this or if that is causing my issues, other
metrics during that time seem normal.

Thanks



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-4-0-to-Solr-4-1-upgrade-tp4044990p4045222.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr 4.0 to Solr 4.1 upgrade

[richardg]

This ended up being a SPM issue.  I noticed the same issue w/ 4.2 and decided
to upgrade to monitor version 1.9.0 and it is now showing correct data.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-4-0-to-Solr-4-1-upgrade-tp4044990p4046631.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr 4.1 monitoring with /solr/replication?command=details - indexVersion?

[richardg]

I believe this is the same issue as described, I'm running 4.2 and as you can
see my slave is a couple versions ahead of the master (all three slaves show
the same behavior).  This was never the case until I upgraded from 4.0 to
4.2.

Master: 
1363272681951
93
1,022.31 MB
Slave:  
1363273274085
95
1,022.31 MB



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-4-1-monitoring-with-solr-replication-command-details-indexVersion-tp4047329p4047380.html
Sent from the Solr - User mailing list archive at Nabble.com.

High Slave CPU Intermittently After Replication

[richardg]

Here is our setup:

Solr 4.0
Master replicates to three slaves after optimize

We have a problem were every so often after replication the CPU load on the
Slave servers maxes out and request come to a crawl.  

We do a dataimport every 10 minutes and depending on the number of updates
since the last optimize we run an update command with either
optimize=true&maxsegements=4 or just optimize=true (more than 1500 updates
since last full optimize).   We had this issue more often until we put the
optimize updates statements into our process.

Everything had been running great for a week or so until today after
replication everything maxed out on all three slaves, it isn't that things
get progressively worse, right after the replication the issue occurs.  The
only way to recover from it is to do an optimize=true update and once it
replicates out things return to normal where there isn't much load on the
slaves at all.

There isn't anyway to predict this issue and so far I haven't seen anything
in the logs that would offer any clues.  Any ideas?





--
View this message in context: 
http://lucene.472066.n3.nabble.com/High-Slave-CPU-Intermittently-After-Replication-tp4020520.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: High Slave CPU Intermittently After Replication

[richardg]

We tried using MergeFactor setting but out CPU Load/Slow Query time issues
were more widespread, optimizing the index always alleviated the issue that
is why we are using it now.  Our index is 2 GB when optimized and would
balloon to over 4 GB so we thought the issue was it was getting too big.

I notice a small spike in CPU load after every replication but then after a
couple of seconds load returns to normal (which is less that 25%) it is just
sometimes (once in the last week) that it would spike and stay high (10
minutes) until I optimized the index.  Before I would optimize the index
after every commit the issue would occur more often.

We would like to not optimize and use the built in Merging but we had before
and the issue would occur more often.  We were thinking of trying a
mergefactor of 2 again but I'm afraid this issues will return.

I installed SPM and am monitoring it to see if it tells me anything, I can
post the results on Monday and hopefully it will tell us something.

At this time we aren't warming and caches, we weren't sure if this was an
issue because our slowdowns weren't happening every time. Also, we are using
the join functionality of Solr 4 if that helps.

Thanks for your help 



--
View this message in context: 
http://lucene.472066.n3.nabble.com/High-Slave-CPU-Intermittently-After-Replication-tp4020520p4020743.html
Sent from the Solr - User mailing list archive at Nabble.com.