Re: How to keep a maintained index with crawled data

2011-01-20 Thread Erlend Garåsen
requests to Solr Cell. -- Jack Krupansky -Original Message- From: Erlend Garåsen Sent: Wednesday, January 19, 2011 4:29 AM To: solr-user@lucene.apache.org Subject: How to keep a maintained index with crawled data We need a crawler for all web pages outside our CMS, but one crucial future

Re: How to keep a maintained index with crawled data

2011-01-19 Thread Jack Krupansky
repositories. It has an output connector that sends documents and delete requests to Solr Cell. -- Jack Krupansky -Original Message- From: Erlend Garåsen Sent: Wednesday, January 19, 2011 4:29 AM To: solr-user@lucene.apache.org Subject: How to keep a maintained index with crawled data We need a

How to keep a maintained index with crawled data

2011-01-19 Thread Erlend Garåsen
We need a crawler for all web pages outside our CMS, but one crucial future seems to be missing in many of them - a way to detect changes in these documents. Say that you have run a daily crawler job for two months looking for new web pages to crawl in order to keep the Solr index updated. Bu