Hi All,
We are trying implement solr for our newspapers site search.
To build out the index with all the articles published so far, we are
running script which send the request to dataimport handler with different
dates.
What we are seeing is the request is dispatched to solr server,but its not
being processed.
Just wanted to check if its some kind of threading issues, and whats the
best approach to achieve this.
We are sleeping for 75 secs between the requests,
while (($date+=86400) < $now) {
$curdate=strftime("%D", localtime($date));
print "Updating index for $curdate\n";
$curdate=uri_escape($curdate);
my $url =
'http://test.solr.ddtc.cmgdigital.com:8080/solr/npmetrosearch_statesman/dataimport?command=full-import&entity=initialLoad&clean=false&commit=true&forDate='
. $curdate .
'&numArticles=-1&server=app5&site=statesman&articleTypes=story,slideshow,video,poll,specialArticle,list';
print "Sending: $url\n";
#if (system("wget -q -O - \'$url\' | egrep -q \'$regex_pat\'")) {
if (system("curl -s \'$url\' | egrep -q \'$regex_pat\'")) {
print "Failed to match expected regex reply: \"$regex_pat\"\n";
exit 1;
}
sleep 75;
}
This is what we are seeing on the server logs
2010-06-14 12:51:01,328 INFO [org.apache.solr.core.SolrCore]
(http-0.0.0.0-8080-1) [npmetrosearch_statesman] webapp=/solr
path=/dataimport
params={site=statesman&forDate=03/24/10&articleTypes=story,slideshow,video,poll,specialArticle,list&clean=false&commit=true&entity=initialLoad&command=full-import&numArticles=-1&server=app5}
status=0 QTime=0
2010-06-14 12:51:01,329 INFO
[org.apache.solr.handler.dataimport.DataImporter] (Thread-378) Starting Full
Import
2010-06-14 12:51:01,332 INFO
[org.apache.solr.handler.dataimport.SolrWriter] (Thread-378) Read
dataimport.properties
2010-06-14 12:51:01,425 INFO
[org.apache.solr.handler.dataimport.DocBuilder] (Thread-378) Time taken =
0:0:0.93
2010-06-14 12:51:16,338 INFO [org.apache.solr.core.SolrCore]
(http-0.0.0.0-8080-1) [npmetrosearch_statesman] webapp=/solr
path=/dataimport
params={site=statesman&forDate=03/25/10&articleTypes=story,slideshow,video,poll,specialArticle,list&clean=false&commit=true&entity=initialLoad&command=full-import&numArticles=-1&server=app5}
status=0 QTime=0
2010-06-14 12:51:16,338 INFO
[org.apache.solr.handler.dataimport.DataImporter] (Thread-379) Starting Full
Import
2010-06-14 12:51:16,338 INFO
[org.apache.solr.handler.dataimport.SolrWriter] (Thread-379) Read
dataimport.properties
2010-06-14 12:51:16,465 INFO
[org.apache.solr.handler.dataimport.DocBuilder] (Thread-379) Time taken =
0:0:0.126
Appreciate any thoughts on this.
Thanks
Indrani
--
View this message in context:
http://lucene.472066.n3.nabble.com/DatImportHandler-and-cron-issue-tp897698p897698.html
Sent from the Solr - User mailing list archive at Nabble.com.
