Hey Mikhail, Isn't _version_ a timestamp of insertion by default?
I think yes. From a similar query on SE <https://stackoverflow.com/questions/45671144/how-to-get-last-document-insert-in-solr> : You can sort by _version_ field in descending order. AFAIK, _version_ field > is a epoch timestamp (of when the document was indexed into Solr) in > milliseconds multiplied by 2^20. However, I cannot find any official documentation of Solr about this (willing to know more about this). Also, if the user wants to do Date Math <https://lucene.apache.org/solr/guide/6_6/working-with-dates.html#WorkingwithDates-DateMath> with the indexing time, I prefer the solution given by Toke: <field name="index_time" type="date" default="NOW" /> PS: Vidit, never do sorting or querying on _docid_ - I remember querying for a set of documents using _docid by additionally using start and rows. The query never returned the result but I remember that it crashed the Solr server by shooting up the load to over 10x! I think that's because docid -> document thing is (logically) not indexed by the way we are querying. On Fri, 13 Sep 2019 at 06:43, Joel Bernstein <joels...@gmail.com> wrote: > This will do what you describe: > > https://lucene.apache.org/solr/guide/8_1/stream-source-reference.html#topic > > Joel Bernstein > http://joelsolr.blogspot.com/ > > > On Mon, Sep 9, 2019 at 4:18 PM Mikhail Khludnev <m...@apache.org> wrote: > > > Isn't _version_ a timestamp of insertion by default? > > > > On Mon, Sep 9, 2019 at 9:47 PM Vidit Asthana <vidit.astha...@gmail.com> > > wrote: > > > > > Hi, > > > > > > I am building a service where I have to continously read data from a > Solr > > > collection and insert it into another database. Collection will receive > > > daily updates. Initial size of collection is very large. After I have > > > indexed whole data(through cursor mark), on daily basis I want to only > do > > > incremental inserts. > > > > > > My documents don't have anything like timestamp which I can use to > fetch > > > "only newly added" documents after a certain point. Is there any > internal > > > field which I can use to create this checkpoint and then later use that > > to > > > fetch "only incremental updates" from that point onwards? > > > > > > I initially tried to sort the document by ID and use last fetched > cursor > > > mark, but my unique-ID field is a string and there is NO guarantee that > > > newly added document's ID will be in sorted order. > > > > > > Solr version is 8.2.0. > > > > > > Regards, > > > Vidit > > > > > > > > > -- > > Sincerely yours > > Mikhail Khludnev > > > -- -- Regards, *Paras Lehana* [65871] Software Programmer, Auto-Suggest, IndiaMART Intermesh Ltd. 8th Floor, Tower A, Advant-Navis Business Park, Sector 142, Noida, UP, IN - 201303 Mob.: +91-9560911996 Work: 01203916600 | Extn: *8173* -- IMPORTANT: NEVER share your IndiaMART OTP/ Password with anyone.
