: A while back Hoss described Solr queuing behavior:
:
: > searches can go on happily while commits/adds are happening, and
: > multiple adds can happen in parallel, ... but all adds block while a
: > commit is taking place. i just give all of clients that update the
: > index a really large timeout value (ie: 60 seconds or so) and don't
: > worry about queing up indexing requests.
:
: I am worried about those adds queuing up behind an ongoing commit before being
queueing isn't really the approriate word. HTTP requests to add documents
are handled by threads that your servlet container (ie: Jetty) creates to
run the appropriate code for dealing with those adds. that code involves
making an addDocument() call on the index, which will block when a segment
merge is happening (during a "commit" or during some adds if they trigger
a merge).
my last sentence there ("i just ... don't worry about queing up
indexing requests.") was in regards to me using long timeouts with retry
to avoid needing to write any sort of queuing code in the clients that
send add or commit messages to Solr.
(Note: some of this blocking should go away when the
ConcurrentMergeSchedule becomes usabel in Solr 1.3)
: processed in parallel. What if a document is added multiple times and each
: time it is added it has different field values? You'd want the newest, last
: queued version of that document to win, i.e. to be the version that represents
: the document in the index. But processing in parallel suggests that the time
: order of the adds of that document could be lost. Does Solr time stamp the
if you have two parallel requests coming in at teh same time to add
two documents with the same uniqueKey field then there is no garuntee
which one will make it in -- regardless of wether a segment merge or a
commit is happening. The JVM decides when to switch between threads, so
even if a client#1 connects to the server 1 second before client#2 the
servlet container might run the thread corrisponding with client#2's
request first, and then client#1's copy of that document will overwrite
it.
-Hoss
