Re: Ticket CASSANDRA-3578 - Multithreaded CommitLog

2011-12-10 Thread Jonathan Ellis
2011/12/8 Piotr Kołaczkowski : > Can someone tell me what is the use pattern of the CommitLog#add method? I > mean, is it possible, that a single thread calls add many times, remembers > the returned Future objects and *then* waits on all / some of them? Or is it > always like: add, then wait (unti

Re: Ticket CASSANDRA-3578 - Multithreaded CommitLog

2011-12-08 Thread Piotr Kołaczkowski
W dniu 2011-12-08 08:40, Jonathan Ellis pisze: 2011/12/8 Piotr Kołaczkowski: Right, this would be the best option to have an ability to write into multiple log files, put on multiple disks. I'm not sure if it is part of that ticket, though. It's not. I don't think anyone needs more than 80MB/

Re: Ticket CASSANDRA-3578 - Multithreaded CommitLog

2011-12-07 Thread Jonathan Ellis
2011/12/8 Piotr Kołaczkowski : > Right, this would be the best option to have an ability to write into > multiple log files, put on multiple disks. I'm not sure if it is part of > that ticket, though. It's not. I don't think anyone needs more than 80MB/s or so of commitlog bandwidth for a while.

Re: Ticket CASSANDRA-3578 - Multithreaded CommitLog

2011-12-07 Thread Piotr Kołaczkowski
Right, this would be the best option to have an ability to write into multiple log files, put on multiple disks. I'm not sure if it is part of that ticket, though. Maybe we should split it into two things: parallel serialization / CRC and parallel writes to multiple logfiles (as another ticke

Re: Ticket CASSANDRA-3578 - Multithreaded CommitLog

2011-12-07 Thread Jeremiah Jordan
Another option is to have multiple threads reading from the queue and writing to their own commit log files. If you have multiple commit log directories with each having its own task writing to it, you can keep the "only sequential writes" optimization. Multiple writers to one disk only makes

Ticket CASSANDRA-3578 - Multithreaded CommitLog

2011-12-07 Thread Piotr Kołaczkowski
Hello everyone, As an interview task I've got to make CommitLog multithreaded. I'm new to Cassandra project and therefore, before I start modifying code, I have to make sure I understand what is going on there correctly. Feel free to correct anything I got wrong or partially wrong. 1. The Co