Re: Ensuring stable timestamp ordering

2010-11-02 Thread Dennis Gearon
from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036' EARTH has a Right To Life, otherwise we all die. - Original Message From: Toke Eskildsen To: "solr-user@lucene.apache.org" Sent: Mon, November 1, 2010 11:45:34 PM Subject: RE: Ensuring stable times

RE: Ensuring stable timestamp ordering

2010-11-01 Thread Toke Eskildsen
Dennis Gearon [gear...@sbcglobal.net] wrote: > how about a timrstamp with either a GUID appended on the end of it? Since long (8 bytes) is the largest atomic type supported by Java, this would have to be represented as a String (or rather BytesRef) and would take up 4 + 32 bytes + 2 * 4 bytes f

RE: Ensuring stable timestamp ordering

2010-11-01 Thread Dennis Gearon
;http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036' EARTH has a Right To Life, otherwise we all die. --- On Sun, 10/31/10, Toke Eskildsen wrote: > From: Toke Eskildsen > Subject: RE: Ensuring stable timestamp ordering > To: "solr-user@lucene.apache.org"

RE: Ensuring stable timestamp ordering

2010-10-31 Thread Toke Eskildsen
Dennis Gearon [gear...@sbcglobal.net] wrote: > Even microseconds may not be enough on some really good, fast machine. True, especially since the timer might not provide microsecond granularity although the returned value is in microseconds. However, an unique timestamp generator should keep trac

RE: Ensuring stable timestamp ordering

2010-10-31 Thread Dennis Gearon
;http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036' EARTH has a Right To Life, otherwise we all die. --- On Sun, 10/31/10, Toke Eskildsen wrote: > From: Toke Eskildsen > Subject: RE: Ensuring stable timestamp ordering > To: "solr-user@lucene.apache.org"

Re: Ensuring stable timestamp ordering

2010-10-31 Thread Michael Sokolov
Hmm - personally, I wouldn't want to rely on timestamps as a unique-id generation scheme. Might we not one day want to have distributed parallel indexing that merges lazily? Keeping timestamps unique and in sync across multiple nodes would be a tough requirement. I would be happy simply havin

RE: Ensuring stable timestamp ordering

2010-10-31 Thread Toke Eskildsen
Lance Norskog [goks...@gmail.com] wrote: > It would be handy to have an auto-incrementing date field, so that > each document would get a unique number and the timestamp would then > be the unique ID of the document. If someone want to implement this, I'll just note that the granilarity of Solr d

Re: Ensuring stable timestamp ordering

2010-10-31 Thread Erick Erickson
O, I didn't realize that, thanks! Erick On Sat, Oct 30, 2010 at 10:27 PM, Lance Norskog wrote: > Hi- > > NOW does not get re-run for each document. If you give a large upload > batch, the same NOW is given to each document. > > It would be handy to have an auto-incrementing date field, so t

Re: Ensuring stable timestamp ordering

2010-10-30 Thread Lance Norskog
Hi- NOW does not get re-run for each document. If you give a large upload batch, the same NOW is given to each document. It would be handy to have an auto-incrementing date field, so that each document would get a unique number and the timestamp would then be the unique ID of the document. On Sa

Re: Ensuring stable timestamp ordering

2010-10-30 Thread Erick Erickson
What are the actual values in your index? I'm wondering if they all get the same values somehow, perhaps due to the granularity of your dates? And (and I'm really grasping at straws here) your is causing enough delay to have time intervals be greater than your granularity. Unfortunately, that do

RE: Ensuring stable timestamp ordering

2010-10-28 Thread Michael Sokolov
(Sorry - fumble finger sent too soon.) My confusion stems from the fact that in my test I insert a number of documents, and then retrieve them ordered by timestamp, and they don't come back in the same order they were inserted (the order seems random), unless I commit after each insert. Is that