Actually I was considering using logstash4solr, but it didn't seem mature enough. does it work fine? any known bugs?
are you collecting the logs in the same solr cluster you use for the production systems? if so, what will you do if for some reason solr is down and you would like to analyze the logs to see what happend? btw, i started a new solr cluster with 7 shards, replicationfactor=3 and run indexing job of 400K docs, it got stuck on 150K because I used Socketappender directly to write to logstash and logstash disk got full. that's why I moved to using AsyncAppender, and I plan on moving to using rabbit. but this is also why I wanted to filter some of the logs. indexing 150K docs prodcued 50GB of logs. this seemed too much. Tim Potter wrote > I'm using logstash4solr (http://logstash4solr.org) for something similar > ... > > I setup my Solr to use Log4J by passing the following on the command-line > when starting Solr: > -Dlog4j.configuration=file:///$SCRIPT_DIR/log4j.properties > > Then I use a custom Log4J appender that writes to RabbitMQ: > > https://github.com/plant42/rabbitmq-log4j-appender > > You can then configure a RabbitMQ input for logstash - > http://logstash.net/docs/1.3.2/inputs/rabbitmq > > This decouples the log writes from log indexing in logstash4solr, which > scales better for active Solr installations. > > Btw ... I just log everything from Solr using this approach but you can > use standard Log4J configuration settings to limit which classes / log > levels to send to the RabbitMQ appender. > > Cheers, > > Timothy Potter > Sr. Software Engineer, LucidWorks > www.lucidworks.com > > ________________________________________ > From: adfel70 < > adfel70@ > > > Sent: Monday, December 30, 2013 8:15 AM > To: > solr-user@.apache > Subject: monitoring solr logs > > hi > i'm trying to figure out which solr and zookeeper logs i should monitor > and > collect. > All the logs will be written to a file but I want to collect some of them > with logstash in order to be able to analyze them efficiently. > any inputs on logs of which classes i should collect? > > thanks. > > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/monitoring-solr-logs-tp4108721.html > Sent from the Solr - User mailing list archive at Nabble.com. -- View this message in context: http://lucene.472066.n3.nabble.com/monitoring-solr-logs-tp4108721p4108737.html Sent from the Solr - User mailing list archive at Nabble.com.
