This will do what you describe: https://lucene.apache.org/solr/guide/8_1/stream-source-reference.html#topic
Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Sep 9, 2019 at 4:18 PM Mikhail Khludnev <m...@apache.org> wrote: > Isn't _version_ a timestamp of insertion by default? > > On Mon, Sep 9, 2019 at 9:47 PM Vidit Asthana <vidit.astha...@gmail.com> > wrote: > > > Hi, > > > > I am building a service where I have to continously read data from a Solr > > collection and insert it into another database. Collection will receive > > daily updates. Initial size of collection is very large. After I have > > indexed whole data(through cursor mark), on daily basis I want to only do > > incremental inserts. > > > > My documents don't have anything like timestamp which I can use to fetch > > "only newly added" documents after a certain point. Is there any internal > > field which I can use to create this checkpoint and then later use that > to > > fetch "only incremental updates" from that point onwards? > > > > I initially tried to sort the document by ID and use last fetched cursor > > mark, but my unique-ID field is a string and there is NO guarantee that > > newly added document's ID will be in sorted order. > > > > Solr version is 8.2.0. > > > > Regards, > > Vidit > > > > > -- > Sincerely yours > Mikhail Khludnev >
