Re: DIH: URLDataSource and incremental indexing

2009-07-09 Thread Noble Paul നോബിള്‍ नोब्ळ्
hi Erik, It is designed to achieve this using a Transformer. I am assuming that your API gives delta "deleted/modified/added" documents. Always run a full-import with clean=false. Depending on the values returned by the API your transformer can use $deleteById for deletes etc. $nextUrl and $hasM

DIH: URLDataSource and incremental indexing

2009-07-09 Thread Erik Hatcher
I'm exploring other ways of getting data into Solr via DataImportHandler than through a relational database, particularly the URLDataSource. I see the special commands for deleting by id and query as well as the $hasMore/$nextUrl techniques, but I'm unclear on exactly how one would go abo