[
https://issues.apache.org/jira/browse/BOOKKEEPER-572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584465#comment-13584465
]
Ivan Kelly commented on BOOKKEEPER-572:
---------------------------------------
I did a quick bench with the new patch.
TPT 1000 ledgers: 25177 ops/sec
Bookie Latency: 1.2008332
Bookie TPT: 20938
Compare to 4.2.1
TPT 1000 ledgers: 25014 ops/sec
Bookie Latency: 1.3229621999999999
Bookie TPT: 18960
So performance is fine. I also ran for 30 minutes, with a profiler attached.
Memory usage is also fine. I can share the yourkit snapshot if anyone is
interested enough.
The patch modifies two tests, SlowBookieTest and LedgerCacheTest. For
SlowBookieTest, there was a bug in the test, where we were closing the ledger
handle before we allowed the handle to handlebookiefailure properly. This patch
changes timing, so it triggered this bug to hit. For LedgerCacheTest, it
assumed that Bookie#addEntry wrote to the indices synchronously. This is not
the case anymore (it writes to the indices in the write callback).
Also, I added a bound on the journal's queue, and an option to modify it.
Before we were throttled by the LedgerCacheImpl's file accesses. This is no
longer the case.
> Make the journal a write ahead log
> ----------------------------------
>
> Key: BOOKKEEPER-572
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-572
> Project: Bookkeeper
> Issue Type: Bug
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: 4.3.0
>
> Attachments:
> 0003-BOOKKEEPER-572-Write-to-the-journal-before-writing-t.patch
>
>
> A bookie adds to the LedgerStorage before writing to the journal. This is the
> fundamental problem behind BOOKKEEPER-447 and blocks a nice solution to
> BOOKKEEPER-530. By writing to the memory state before the journal, we exposed
> ourselves to bugs if the bookie crashed before we wrote to the journal. The
> entry may exist in index, but not in the entrylog, a situation which cannot
> be distinguished from an I/O error. The comments on BOOKKEEPER-447 goes into
> more details.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
