Interesting stuff; Solr as a syslog store.

[Antonio Lobato]

Hey everyone, I don't actually have a question, but I just thought I'd 
share something really cool that I did with Solr for our company.

We run a good amount of servers, well into the several hundreds, and 
naturally we need a way to centralize all of the system logs.  For a 
while we used a commercial solution to centralize and search our logs, 
but they wanted to charge us tens of thousands of dollars for just one 
gigabyte/day more of indexed data.  So I said forget it, I'll write my 
own solution!

We already use Solr for some of our other backend searching systems, so 
I came up with an idea to index all of our logs to Solr.  I wrote a 
daemon in perl that listens on the syslog port, and pointed every single 
system's syslog to forward to this single server.  From there, this 
daemon will write to a Solr indexing server after parsing them into 
fields, such as date/time, host, program, pid, text, etc.  I then wrote 
a cool javascript/ajax web front end for Solr searching, and bam.  Real 
time searching of all of our syslogs from a web interface, for no cost!

Just thought this would be a neat story to share with you all.  I've 
really grown to love Solr, it's something else!

Thanks,
-Antonio