How did you verify it was not processed? Did you
1. Query for docs - with no results
2. Use Solr Admin tool?
3. Bypass data import handler and see if the doc post/commit works.
On Tue, Jun 15, 2010 at 10:29 PM, iboppana
<indrani.bopp...@cmgdigital.com>wrote:
>
> Hi All,
>
> We are trying implement solr for our newspapers site search.
> To build out the index with all the articles published so far, we are
> running script which send the request to dataimport handler with different
> dates.
> What we are seeing is the request is dispatched to solr server,but its not
> being processed.
> Just wanted to check if its some kind of threading issues, and whats the
> best approach to achieve this.
>
> We are sleeping for 75 secs between the requests,
>
>
> while (($date+=86400) < $now) {
> $curdate=strftime("%D", localtime($date));
>
> print "Updating index for $curdate\n";
>
> $curdate=uri_escape($curdate);
>
> my $url =
> '
> http://test.solr.ddtc.cmgdigital.com:8080/solr/npmetrosearch_statesman/dataimport?command=full-import&entity=initialLoad&clean=false&commit=true&forDate=
> '
> . $curdate .
>
> '&numArticles=-1&server=app5&site=statesman&articleTypes=story,slideshow,video,poll,specialArticle,list';
>
> print "Sending: $url\n";
>
>
> #if (system("wget -q -O - \'$url\' | egrep -q \'$regex_pat\'")) {
> if (system("curl -s \'$url\' | egrep -q \'$regex_pat\'")) {
> print "Failed to match expected regex reply: \"$regex_pat\"\n";
> exit 1;
> }
>
> sleep 75;
> }
>
>
>
>
> This is what we are seeing on the server logs
> 2010-06-14 12:51:01,328 INFO [org.apache.solr.core.SolrCore]
> (http-0.0.0.0-8080-1) [npmetrosearch_statesman] webapp=/solr
> path=/dataimport
>
> params={site=statesman&forDate=03/24/10&articleTypes=story,slideshow,video,poll,specialArticle,list&clean=false&commit=true&entity=initialLoad&command=full-import&numArticles=-1&server=app5}
> status=0 QTime=0
> 2010-06-14 12:51:01,329 INFO
> [org.apache.solr.handler.dataimport.DataImporter] (Thread-378) Starting
> Full
> Import
> 2010-06-14 12:51:01,332 INFO
> [org.apache.solr.handler.dataimport.SolrWriter] (Thread-378) Read
> dataimport.properties
> 2010-06-14 12:51:01,425 INFO
> [org.apache.solr.handler.dataimport.DocBuilder] (Thread-378) Time taken =
> 0:0:0.93
> 2010-06-14 12:51:16,338 INFO [org.apache.solr.core.SolrCore]
> (http-0.0.0.0-8080-1) [npmetrosearch_statesman] webapp=/solr
> path=/dataimport
>
> params={site=statesman&forDate=03/25/10&articleTypes=story,slideshow,video,poll,specialArticle,list&clean=false&commit=true&entity=initialLoad&command=full-import&numArticles=-1&server=app5}
> status=0 QTime=0
> 2010-06-14 12:51:16,338 INFO
> [org.apache.solr.handler.dataimport.DataImporter] (Thread-379) Starting
> Full
> Import
> 2010-06-14 12:51:16,338 INFO
> [org.apache.solr.handler.dataimport.SolrWriter] (Thread-379) Read
> dataimport.properties
> 2010-06-14 12:51:16,465 INFO
> [org.apache.solr.handler.dataimport.DocBuilder] (Thread-379) Time taken =
> 0:0:0.126
>
> Appreciate any thoughts on this.
>
> Thanks
> Indrani
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/DatImportHandler-and-cron-issue-tp897698p897698.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
