mikemccand commented on a change in pull request #1397: LUCENE-9304: Refactor
DWPTPool to pool DWPT directly
URL: https://github.com/apache/lucene-solr/pull/1397#discussion_r403701483
##########
File path: lucene/core/src/java/org/apache/lucene/index/DocumentsWriter.java
##########
@@ -59,21 +58,25 @@
* Threads:
*
* Multiple threads are allowed into addDocument at once.
- * There is an initial synchronized call to getThreadState
- * which allocates a ThreadState for this thread. The same
- * thread will get the same ThreadState over time (thread
- * affinity) so that if there are consistent patterns (for
- * example each thread is indexing a different content
- * source) then we make better use of RAM. Then
- * processDocument is called on that ThreadState without
+ * There is an initial synchronized call to
+ * {@link DocumentsWriterFlushControl#obtainAndLock()}
+ * which allocates a DWPT for this indexing thread. The same
+ * thread will not necessarily get the same DWPT over time.
+ * Then updateDocuments is called on that DWPT without
* synchronization (most of the "heavy lifting" is in this
- * call). Finally the synchronized "finishDocument" is
- * called to flush changes to the directory.
+ * call). Once a DWPT fills up enough RAM or hold enough
+ * documents in memory the DWPT is checked out for flush
+ * and all changes are written to the directory. Each DWPT
+ * corresponds to one segment being written.
*
- * When flush is called by IndexWriter we forcefully idle
- * all threads and flush only once they are all idle. This
- * means you can call flush with a given thread even while
- * other threads are actively adding/deleting documents.
+ * When flush is called by IndexWriter we check out all DWPTs
+ * that are associated with the current {@link DocumentsWriterDeleteQueue}
+ * out of the {@link DocumentsWriterPerThreadPool} and write
+ * them to disk. The flush process can piggy-back on incoming
+ * indexing threads or event block them from adding documents
Review comment:
s/`event`/`even`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
