Re: DIH - Example of using $nextUrl and $hasMore

2009-02-04 Thread Noble Paul നോബിള്‍ नोब्ळ्
currently the initial counter is not set , so the value becomes an empty string http://subdomain.site.com/boards.rss?page=${blogs.n} becomes http://subdomain.site.com/boards.rss?page= we need to fix this. Unfortunately the transformer is invoked only after the first chunk is fetched. the best bet

Re: DIH - Example of using $nextUrl and $hasMore

2009-02-02 Thread Jon Baer
See I think Im just misunderstanding how this entity is suppose to be setup ... for example, using the patch on 1.3 I ended up in a loop where .n is never set ... Feb 2, 2009 1:31:02 PM org.apache.solr.handler.dataimport.HttpDataSource getData INFO: Created URL to: http://subdomain.site.com/feed.r

Re: DIH - Example of using $nextUrl and $hasMore

2009-02-02 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Mon, Feb 2, 2009 at 11:01 PM, Jon Baer wrote: > Yes I think what Jared mentions in the JIRA is what I was thinking about > when it is recommended to always return true for $hasMore ... > > "The transformer must know somehow when $hasMore should be true. If the > transformer always give $hasMore

Re: DIH - Example of using $nextUrl and $hasMore

2009-02-02 Thread Jon Baer
Yes I think what Jared mentions in the JIRA is what I was thinking about when it is recommended to always return true for $hasMore ... "The transformer must know somehow when $hasMore should be true. If the transformer always give $hasMore a value "true", will there be infinite requests made or wi

Re: DIH - Example of using $nextUrl and $hasMore

2009-02-02 Thread Shalin Shekhar Mangar
On Mon, Feb 2, 2009 at 9:20 PM, Jon Baer wrote: > Hi, > > Sorry I know this exists ... > > "If an API supports chunking (when the dataset is too large) multiple calls > need to be made to complete the process. XPathEntityprocessor supports this > with a transformer. If transformer returns a row w

Re: DIH - Example of using $nextUrl and $hasMore

2009-02-02 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Mon, Feb 2, 2009 at 9:20 PM, Jon Baer wrote: > Hi, > > Sorry I know this exists ... > > "If an API supports chunking (when the dataset is too large) multiple calls > need to be made to complete the process. XPathEntityprocessor supports this > with a transformer. If transformer returns a row wh

DIH - Example of using $nextUrl and $hasMore

2009-02-02 Thread Jon Baer
Hi, Sorry I know this exists ... "If an API supports chunking (when the dataset is too large) multiple calls need to be made to complete the process. XPathEntityprocessor supports this with a transformer. If transformer returns a row which contains a field * $hasMore* with a the value "true" the