Build failed in Jenkins: Cassandra-quick #162

2011-12-07 Thread Apache Jenkins Server 
See 

Changes:

[jbellis] update CHANGES

--
[...truncated 1694 lines...]
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.833 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.locator.TokenMetadataTest
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.454 sec
[junit] 
[junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceCounterTest
[junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.904 sec
[junit] 
[junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceCounterTest):
 Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:484)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:461)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceCounterTest 
FAILED
[junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceStandardTest
[junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.704 sec
[junit] 
[junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceStandardTest):
Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:484)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:461)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceStandardTest 
FAILED
[junit] Testsuite: org.apache.cassandra.service.CassandraServerTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.57 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.service.ConsistencyLevelTest
[junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 1.04 sec
[junit] 
[junit] - Standard Error -
[junit] ERROR 15:55:18,376 unknown endpoint /127.0.0.2
[junit] ERROR 15:55:18,382 unknown endpoint /127.0.0.2
[junit] -  ---
[junit] Testcase: 
testReadWriteConsistencyChecks(org.apache.cassandra.service.ConsistencyLevelTest):
FAILED
[junit] 
[junit] junit.framework.AssertionFailedError: 
[junit] at 
org.apache.cassandra.service.ConsistencyLevelTest.testReadWriteConsistencyChecks(ConsistencyLevelTest.java:165)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.ConsistencyLevelTest FAILED
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
[junit] 
[junit] Testcase: 
org.apache.cassandra.service.EmbeddedCassandraServiceTest:BeforeFirstTest:  
  Caused an ERROR
[junit] Forked Java VM exited abnormally. Please note the time in the 
report does not reflect the time until the VM exit.
[junit] junit.framework.AssertionFailedError: Forked Java VM exited 
abnormally. Please note the time in the report does not reflect the time until 
the VM exit.
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.EmbeddedCassandraServiceTest 
FAILED (crashed)
[junit] Testsuite: org.apache.cassandra.service.InitClientTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0.459 sec
[junit] 
[junit] Testcase: 
testInitClient

Re: Build failed in Jenkins: Cassandra-quick #162

2011-12-07 Thread Ted Crossman 

Hi all,

I was just wondering if this mailing list only sees the failures,
 while successful builds are happening and the list is not being 
alerted to them?


Seems kind of pointless to have the e-mail being sent if no one does 
anything about them :)


Just curious,

tedo

On 12/7/11 7:57 AM, Apache Jenkins Server wrote:

See

Changes:

[jbellis] update CHANGES

--
[...truncated 1694 lines...]
 [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.833 sec
 [junit]
 [junit] Testsuite: org.apache.cassandra.locator.TokenMetadataTest
 [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.454 sec
 [junit]
 [junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceCounterTest
 [junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.904 sec
 [junit]
 [junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceCounterTest):
Caused an ERROR
 [junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
 [junit] org.apache.cassandra.config.ConfigurationException: 
/127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
 [junit]at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
 [junit]at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
 [junit]at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:484)
 [junit]at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:461)
 [junit]at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
 [junit]
 [junit]
 [junit] Test org.apache.cassandra.service.AntiEntropyServiceCounterTest 
FAILED
 [junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceStandardTest
 [junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.704 sec
 [junit]
 [junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceStandardTest):
   Caused an ERROR
 [junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
 [junit] org.apache.cassandra.config.ConfigurationException: 
/127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
 [junit]at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
 [junit]at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
 [junit]at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:484)
 [junit]at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:461)
 [junit]at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
 [junit]
 [junit]
 [junit] Test org.apache.cassandra.service.AntiEntropyServiceStandardTest 
FAILED
 [junit] Testsuite: org.apache.cassandra.service.CassandraServerTest
 [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.57 sec
 [junit]
 [junit] Testsuite: org.apache.cassandra.service.ConsistencyLevelTest
 [junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 1.04 sec
 [junit]
 [junit] - Standard Error -
 [junit] ERROR 15:55:18,376 unknown endpoint /127.0.0.2
 [junit] ERROR 15:55:18,382 unknown endpoint /127.0.0.2
 [junit] -  ---
 [junit] Testcase: 
testReadWriteConsistencyChecks(org.apache.cassandra.service.ConsistencyLevelTest):
   FAILED
 [junit]
 [junit] junit.framework.AssertionFailedError:
 [junit]at 
org.apache.cassandra.service.ConsistencyLevelTest.testReadWriteConsistencyChecks(ConsistencyLevelTest.java:165)
 [junit]
 [junit]
 [junit] Test org.apache.cassandra.service.ConsistencyLevelTest FAILED
 [junit] Testsuite: 
org.apache.cassandra.service.EmbeddedCassandraServiceTest
 [junit] Testsuite: 
org.apache.cassandra.service.EmbeddedCassandraServiceTest
 [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
 [junit]
 [junit] Testcase: 
org.apache.cassandra.service.EmbeddedCassandraServiceTest:BeforeFirstTest:  
 Caused an ERROR
 [junit] Forked Java VM exited abnormally. Please note the time in the 
report does not reflect the time until the VM exit.
 [junit] junit.framework.AssertionFailedError: Forked Java VM exited 
abnormally. Please note the time in th

Build failed in Jenkins: Cassandra-quick #163

2011-12-07 Thread Apache Jenkins Server 
See 

Changes:

[slebresne] merge from 1.0

--
[...truncated 1712 lines...]
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.837 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.locator.TokenMetadataTest
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.489 sec
[junit] 
[junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceCounterTest
[junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.765 sec
[junit] 
[junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceCounterTest):
 Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:484)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:461)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceCounterTest 
FAILED
[junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceStandardTest
[junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.968 sec
[junit] 
[junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceStandardTest):
Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:484)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:461)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceStandardTest 
FAILED
[junit] Testsuite: org.apache.cassandra.service.CassandraServerTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.575 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.service.ConsistencyLevelTest
[junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 0.981 sec
[junit] 
[junit] - Standard Error -
[junit] ERROR 16:24:53,714 unknown endpoint /127.0.0.2
[junit] ERROR 16:24:53,720 unknown endpoint /127.0.0.2
[junit] -  ---
[junit] Testcase: 
testReadWriteConsistencyChecks(org.apache.cassandra.service.ConsistencyLevelTest):
FAILED
[junit] 
[junit] junit.framework.AssertionFailedError: 
[junit] at 
org.apache.cassandra.service.ConsistencyLevelTest.testReadWriteConsistencyChecks(ConsistencyLevelTest.java:165)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.ConsistencyLevelTest FAILED
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
[junit] 
[junit] Testcase: 
org.apache.cassandra.service.EmbeddedCassandraServiceTest:BeforeFirstTest:  
  Caused an ERROR
[junit] Forked Java VM exited abnormally. Please note the time in the 
report does not reflect the time until the VM exit.
[junit] junit.framework.AssertionFailedError: Forked Java VM exited 
abnormally. Please note the time in the report does not reflect the time until 
the VM exit.
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.EmbeddedCassandraServiceTest 
FAILED (crashed)
[junit] Testsuite: org.apache.cassandra.service.InitClientTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0.462 sec
[junit] 
[junit] Testcase: 
testInitCl

Build failed in Jenkins: Cassandra #1242

2011-12-07 Thread Apache Jenkins Server 
See 

Changes:

[slebresne] merge from 1.0

[jbellis] update CHANGES

--
[...truncated 2382 lines...]
[junit] Testsuite: org.apache.cassandra.locator.TokenMetadataTest
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.459 sec
[junit] 
[junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceCounterTest
[junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.732 sec
[junit] 
[junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceCounterTest):
 Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:484)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:461)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceCounterTest 
FAILED
[junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceStandardTest
[junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.657 sec
[junit] 
[junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceStandardTest):
Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:484)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:461)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceStandardTest 
FAILED
[junit] Testsuite: org.apache.cassandra.service.CassandraServerTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.575 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.service.ConsistencyLevelTest
[junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 1.058 sec
[junit] 
[junit] - Standard Error -
[junit] ERROR 16:36:45,614 unknown endpoint /127.0.0.2
[junit] ERROR 16:36:45,620 unknown endpoint /127.0.0.2
[junit] -  ---
[junit] Testcase: 
testReadWriteConsistencyChecks(org.apache.cassandra.service.ConsistencyLevelTest):
FAILED
[junit] 
[junit] junit.framework.AssertionFailedError: 
[junit] at 
org.apache.cassandra.service.ConsistencyLevelTest.testReadWriteConsistencyChecks(ConsistencyLevelTest.java:165)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.ConsistencyLevelTest FAILED
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
[junit] 
[junit] Testcase: 
org.apache.cassandra.service.EmbeddedCassandraServiceTest:BeforeFirstTest:  
  Caused an ERROR
[junit] Forked Java VM exited abnormally. Please note the time in the 
report does not reflect the time until the VM exit.
[junit] junit.framework.AssertionFailedError: Forked Java VM exited 
abnormally. Please note the time in the report does not reflect the time until 
the VM exit.
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.EmbeddedCassandraServiceTest 
FAILED (crashed)
[junit] Testsuite: org.apache.cassandra.service.InitClientTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0.458 sec
[junit] 
[junit] Testcase: 
testInitClientStartup(org.apache.cassandra.service.InitClientTest):   
C

Re: Build failed in Jenkins: Cassandra-quick #162

2011-12-07 Thread Jonathan Ellis 
Unfortunately Jenkins tries to run all the Cassandra branches at once
and gets port conflicts like this.  Maybe we should restrict Jenkins
to either current-stable or to trunk.

On Wed, Dec 7, 2011 at 10:21 AM, Ted Crossman  wrote:
> Hi all,
>
> I was just wondering if this mailing list only sees the failures,
>  while successful builds are happening and the list is not being alerted to
> them?
>
> Seems kind of pointless to have the e-mail being sent if no one does
> anything about them :)
>
> Just curious,
>
> tedo
>
> On 12/7/11 7:57 AM, Apache Jenkins Server wrote:
>>
>> See
>>
>> Changes:
>>
>> [jbellis] update CHANGES
>>
>> --
>> [...truncated 1694 lines...]
>>     [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.833 sec
>>     [junit]
>>     [junit] Testsuite: org.apache.cassandra.locator.TokenMetadataTest
>>     [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.454 sec
>>     [junit]
>>     [junit] Testsuite:
>> org.apache.cassandra.service.AntiEntropyServiceCounterTest
>>     [junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.904 sec
>>     [junit]
>>     [junit] Testcase:
>> testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceCounterTest):
>>        Caused an ERROR
>>     [junit] /127.0.0.1:7010 is in use by another process.  Change
>> listen_address:storage_port in cassandra.yaml to values that do not conflict
>> with other services
>>     [junit] org.apache.cassandra.config.ConfigurationException:
>> /127.0.0.1:7010 is in use by another process.  Change
>> listen_address:storage_port in cassandra.yaml to values that do not conflict
>> with other services
>>     [junit]    at
>> org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
>>     [junit]    at
>> org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
>>     [junit]    at
>> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:484)
>>     [junit]    at
>> org.apache.cassandra.service.StorageService.initServer(StorageService.java:461)
>>     [junit]    at
>> org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
>>     [junit]
>>     [junit]
>>     [junit] Test
>> org.apache.cassandra.service.AntiEntropyServiceCounterTest FAILED
>>     [junit] Testsuite:
>> org.apache.cassandra.service.AntiEntropyServiceStandardTest
>>     [junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.704 sec
>>     [junit]
>>     [junit] Testcase:
>> testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceStandardTest):
>>       Caused an ERROR
>>     [junit] /127.0.0.1:7010 is in use by another process.  Change
>> listen_address:storage_port in cassandra.yaml to values that do not conflict
>> with other services
>>     [junit] org.apache.cassandra.config.ConfigurationException:
>> /127.0.0.1:7010 is in use by another process.  Change
>> listen_address:storage_port in cassandra.yaml to values that do not conflict
>> with other services
>>     [junit]    at
>> org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
>>     [junit]    at
>> org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
>>     [junit]    at
>> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:484)
>>     [junit]    at
>> org.apache.cassandra.service.StorageService.initServer(StorageService.java:461)
>>     [junit]    at
>> org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
>>     [junit]
>>     [junit]
>>     [junit] Test
>> org.apache.cassandra.service.AntiEntropyServiceStandardTest FAILED
>>     [junit] Testsuite: org.apache.cassandra.service.CassandraServerTest
>>     [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.57 sec
>>     [junit]
>>     [junit] Testsuite: org.apache.cassandra.service.ConsistencyLevelTest
>>     [junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 1.04 sec
>>     [junit]
>>     [junit] - Standard Error -
>>     [junit] ERROR 15:55:18,376 unknown endpoint /127.0.0.2
>>     [junit] ERROR 15:55:18,382 unknown endpoint /127.0.0.2
>>     [junit] -  ---
>>     [junit] Testcase:
>> testReadWriteConsistencyChecks(org.apache.cassandra.service.ConsistencyLevelTest):
>>       FAILED
>>     [junit]
>>     [junit] junit.framework.AssertionFailedError:
>>     [junit]    at
>> org.apache.cassandra.service.ConsistencyLevelTest.testReadWriteConsistencyChecks(ConsistencyLevelTest.java:165)
>>     [junit]
>>     [junit]
>>     [junit] Test org.apache.cassandra.service.ConsistencyLevelTest FAILED
>>     [junit] Testsuite:
>> org.apache.cassandra.service.EmbeddedCassandraServiceTest
>>     [junit] Testsuite:
>> org.apache.cassandra.service.EmbeddedCassandraServiceTest
>>

Build failed in Jenkins: Cassandra-quick #164

2011-12-07 Thread Apache Jenkins Server 
See 

Changes:

[eevans] Update to support TimeUUIDType (CASSANDRA-3541)

Patch by eevans; reviewed by Pavel Yaskevich for CASSANDRA-2268

[eevans] optional CQL query support

Patch by eevans; reviewed by Pavel Yaskevich for CASSANDRA-2268

[eevans] CASSANDRA-2268 add stress source folder (eclipse)

Patch by eevans; reviewed by Pavel Yaskevich for CASSANDRA-2268

--
[...truncated 1725 lines...]
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.743 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.locator.TokenMetadataTest
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.365 sec
[junit] 
[junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceCounterTest
[junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.144 sec
[junit] 
[junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceCounterTest):
 Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:484)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:461)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceCounterTest 
FAILED
[junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceStandardTest
[junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.366 sec
[junit] 
[junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceStandardTest):
Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:484)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:461)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceStandardTest 
FAILED
[junit] Testsuite: org.apache.cassandra.service.CassandraServerTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.491 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.service.ConsistencyLevelTest
[junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 0.881 sec
[junit] 
[junit] - Standard Error -
[junit] ERROR 16:52:04,747 unknown endpoint /127.0.0.2
[junit] ERROR 16:52:04,751 unknown endpoint /127.0.0.2
[junit] -  ---
[junit] Testcase: 
testReadWriteConsistencyChecks(org.apache.cassandra.service.ConsistencyLevelTest):
FAILED
[junit] 
[junit] junit.framework.AssertionFailedError: 
[junit] at 
org.apache.cassandra.service.ConsistencyLevelTest.testReadWriteConsistencyChecks(ConsistencyLevelTest.java:165)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.ConsistencyLevelTest FAILED
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
[junit] 
[junit] Testcase: 
org.apache.cassandra.service.EmbeddedCassandraServiceTest:BeforeFirstTest:  
  Caused an ERROR
[junit] Forked Java VM exited abnormally. Please note the time in the 
report does not reflect the time until the VM exit.
[junit] junit.framework.AssertionFailedError: Forked Java VM exited 
abnormally. Please note the time in the report does not reflect the time unti

Ticket CASSANDRA-3578 - Multithreaded CommitLog

2011-12-07 Thread Piotr Kołaczkowski 


Hello everyone,

As an interview task I've got to make CommitLog multithreaded. I'm new 
to Cassandra project and therefore, before I start modifying code, I 
have to make sure I understand what is going on there correctly.

Feel free to correct anything I got wrong or partially wrong.

1. The CommitLog singleton object is responsible for receiving 
RowMutation objects by its add method. The add method is thread-safe and 
is aimed to be called by many threads adding their RowMutations 
independently.


2. Each invocation of CommitLog#add  puts a new task onto the queue. 
This task is represented by LogRecordAdder callable object, which is 
responsible for actually calling the CommitLogSegment#write method for 
doing all the "hard work" of serializing the RowMutation object, 
calculating CRC and writing that to the memory mapped CommitLogSegment 
file buffer. The add method immediately returns a Future object, which 
can be waited for (if needed) - it will block until the row mutation is 
saved to the log file and (optionally) synced.


3. The queued tasks are processed one-by-one, sequentially by the 
appropriate ICommitLogExecutorService. This service also controls 
syncing the active memory mapped segments. There are two sync strategies 
available: periodic and batched. The periodic simply calls sync 
periodically by asynchronously putting appropriate sync task into the 
queue, inbetween the LogRecordAdder tasks. The LogRecordAdder tasks are 
"done" as soon as they are written to the log, so the caller *won't 
wait* for the sync. On the other hand, the batched strategy 
(BatchCommitLogExecutorService), performs the tasks in batches, each 
batch finished with an sync operation. The tasks are marked as done 
*after* the sync operation is finished. This deferred task marking  is 
achieved thanks to CheaterFutureTask class - allowing to run the task 
without immediately marking FutureTask as done. Nice. :)


4. The serialized size of the RowMutation object is calculated twice: 
once before submitting to the ExecutorService - to detect if it is not 
larger than the segment size, and then after being taken from the queue 
for execution - to check if it fits into the active CommitLogSegment, 
and if it doesn't, to activate a new CommitLogSegment. Looks to me like 
a point needing optimisation. I couldn't find any code for caching the 
serialized size to avoid doing it twice.


5. The serialization, CRC calculation and actual commit log writes are 
happening sequentially. The aim of this ticket is to make it parallel.


Questions:
1. What happens to the recovery, if the power goes off before the log 
has been synced, and it has been written partially (e.g. it is truncated 
in the middle of the RowMutation data)? Are incomplete RowMutation 
writes detected only by means of CRC (CommitLog around lines 237-240), 
or is there some other mechanism for it?


2. Is the CommitLog#add method allowed to do some heavier computations? 
What is the contract for it? Does it have to return immediately or can I 
move some code into it?


Solutions I consider (please comment):

1. Moving the serialized size calculation, serialization and CRC 
calculation totally before the executor service queue, so that these 
operations would be parallel, and performed once per RowMutation object. 
The calculated size / data array / CRC value would be appended to the 
task and put into the queue. Then copying that into the commit log would 
proceed sequentially - the task would contain only code for log writing. 
This is the safest and easiest solution, but also the least performant, 
because copying is still sequential and still might be a bottleneck. The 
logic of allocating new commit log segments and syncing remains unchanged.


2. Moving the serialized size calculation, serialization, CRC 
calculation *and commit log writing* before the executor service queue. 
This raises immediately some problems / questions:
a) The code for segment allocation needs to be changed, as it becomes 
multithreaded. It can be done using AtomicInteger.compareAndSet, so that 
each RowMutation gets its own, non-overlapping piece of commit log to 
write into.
b) What happens if there is not enough free space in the current active 
segment? Do we allow more active segments at once? Or do we restrict the 
parallelism to writing just into a single active segment (I don't like 
it, as it would be for certain less performant, because we would have to 
wait for finishing the current active segement, before we can start a 
new one)?
c) Is the recovery method ready for reading partially written (invalid) 
RowMutation, that is not the last mutation in the commit log? If we 
allow writing several row mutations parallel, it has to be.
d) The tasks are sent to the queue only for wait-for-sync functionality 
- they would not contain any code to execute, because everything would 
be already done.


3. Everything just as 2., but with an addition, that the serialization 
code

Build failed in Jenkins: Cassandra #1243

2011-12-07 Thread Apache Jenkins Server 
See 

Changes:

[eevans] Update to support TimeUUIDType (CASSANDRA-3541)

Patch by eevans; reviewed by Pavel Yaskevich for CASSANDRA-2268

[eevans] optional CQL query support

Patch by eevans; reviewed by Pavel Yaskevich for CASSANDRA-2268

[eevans] CASSANDRA-2268 add stress source folder (eclipse)

Patch by eevans; reviewed by Pavel Yaskevich for CASSANDRA-2268

--
[...truncated 2374 lines...]
[junit] 
[junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceCounterTest
[junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.867 sec
[junit] 
[junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceCounterTest):
 Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:484)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:461)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceCounterTest 
FAILED
[junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceStandardTest
[junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.964 sec
[junit] 
[junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceStandardTest):
Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:484)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:461)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceStandardTest 
FAILED
[junit] Testsuite: org.apache.cassandra.service.CassandraServerTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.579 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.service.ConsistencyLevelTest
[junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 1.08 sec
[junit] 
[junit] - Standard Error -
[junit] ERROR 17:21:54,178 unknown endpoint /127.0.0.2
[junit] ERROR 17:21:54,183 unknown endpoint /127.0.0.2
[junit] -  ---
[junit] Testcase: 
testReadWriteConsistencyChecks(org.apache.cassandra.service.ConsistencyLevelTest):
FAILED
[junit] 
[junit] junit.framework.AssertionFailedError: 
[junit] at 
org.apache.cassandra.service.ConsistencyLevelTest.testReadWriteConsistencyChecks(ConsistencyLevelTest.java:165)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.ConsistencyLevelTest FAILED
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
[junit] 
[junit] Testcase: 
org.apache.cassandra.service.EmbeddedCassandraServiceTest:BeforeFirstTest:  
  Caused an ERROR
[junit] Forked Java VM exited abnormally. Please note the time in the 
report does not reflect the time until the VM exit.
[junit] junit.framework.AssertionFailedError: Forked Java VM exited 
abnormally. Please note the time in the report does not reflect the time until 
the VM exit.
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.EmbeddedCassandraServiceTest 
FAILED (crashed)
[junit] Testsuite: org.apache.cassandra.service.InitClientTest
[junit] Tests run: 1, Failures:

Build failed in Jenkins: Cassandra-0.8 #411

2011-12-07 Thread Apache Jenkins Server 
See 

Changes:

[slebresne] Detect misuses of CounterColumnType
patch by slebresne; reviewed by jbellis for CASSANDRA-3422

--
[...truncated 2005 lines...]
[junit] Testsuite: org.apache.cassandra.locator.NetworkTopologyStrategyTest
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.465 sec
[junit] 
[junit] Testsuite: 
org.apache.cassandra.locator.OldNetworkTopologyStrategyTest
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.425 sec
[junit] 
[junit] Testsuite: 
org.apache.cassandra.locator.ReplicationStrategyEndpointCacheTest
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.616 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.locator.SimpleStrategyTest
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.66 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.locator.TokenMetadataTest
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.432 sec
[junit] 
[junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceCounterTest
[junit] Tests run: 7, Failures: 0, Errors: 1, Time elapsed: 2.68 sec
[junit] 
[junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceCounterTest):
 Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:237)
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:210)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:459)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:436)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceCounterTest 
FAILED
[junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceStandardTest
[junit] Tests run: 7, Failures: 0, Errors: 1, Time elapsed: 2.773 sec
[junit] 
[junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceStandardTest):
Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:237)
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:210)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:459)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:436)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceStandardTest 
FAILED
[junit] Testsuite: org.apache.cassandra.service.CassandraServerTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.419 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.service.ConsistencyLevelTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.907 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
[junit] 
[junit] Testcase: 
org.apache.cassandra.service.EmbeddedCassandraServiceTest:BeforeFirstTest:  
  Caused an ERROR
[junit] Forked Java VM exited abnormally. Please note the time in the 
report does not reflect the time until the VM exit.
[junit] junit.framework.AssertionFailedError: Forked Java VM exited 
abnormally. Please note the time in the report does not reflect the time until 
the VM exit.
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.EmbeddedCassandraServiceTest 
FAILED (crashed)
[junit] Testsuite: org.apache.cassandra.service.InitClientTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0.441 sec
[junit] 
[junit] Testcase: 
t

Build failed in Jenkins: Cassandra-quick #165

2011-12-07 Thread Apache Jenkins Server 
See 

Changes:

[slebresne] merge from 1.0

--
[...truncated 1729 lines...]
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.86 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.locator.TokenMetadataTest
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.456 sec
[junit] 
[junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceCounterTest
[junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.871 sec
[junit] 
[junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceCounterTest):
 Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:484)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:461)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceCounterTest 
FAILED
[junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceStandardTest
[junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.656 sec
[junit] 
[junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceStandardTest):
Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:484)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:461)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceStandardTest 
FAILED
[junit] Testsuite: org.apache.cassandra.service.CassandraServerTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.571 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.service.ConsistencyLevelTest
[junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 1.074 sec
[junit] 
[junit] - Standard Error -
[junit] ERROR 17:58:39,894 unknown endpoint /127.0.0.2
[junit] ERROR 17:58:39,899 unknown endpoint /127.0.0.2
[junit] -  ---
[junit] Testcase: 
testReadWriteConsistencyChecks(org.apache.cassandra.service.ConsistencyLevelTest):
FAILED
[junit] 
[junit] junit.framework.AssertionFailedError: 
[junit] at 
org.apache.cassandra.service.ConsistencyLevelTest.testReadWriteConsistencyChecks(ConsistencyLevelTest.java:165)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.ConsistencyLevelTest FAILED
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
[junit] 
[junit] Testcase: 
org.apache.cassandra.service.EmbeddedCassandraServiceTest:BeforeFirstTest:  
  Caused an ERROR
[junit] Forked Java VM exited abnormally. Please note the time in the 
report does not reflect the time until the VM exit.
[junit] junit.framework.AssertionFailedError: Forked Java VM exited 
abnormally. Please note the time in the report does not reflect the time until 
the VM exit.
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.EmbeddedCassandraServiceTest 
FAILED (crashed)
[junit] Testsuite: org.apache.cassandra.service.InitClientTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0.455 sec
[junit] 
[junit] Testcase: 
testInitCli

Build failed in Jenkins: Cassandra #1244

2011-12-07 Thread Apache Jenkins Server 
See 

Changes:

[slebresne] merge from 1.0

--
[...truncated 2409 lines...]
[junit] Testsuite: org.apache.cassandra.locator.TokenMetadataTest
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.487 sec
[junit] 
[junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceCounterTest
[junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.733 sec
[junit] 
[junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceCounterTest):
 Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:484)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:461)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceCounterTest 
FAILED
[junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceStandardTest
[junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.922 sec
[junit] 
[junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceStandardTest):
Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:484)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:461)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceStandardTest 
FAILED
[junit] Testsuite: org.apache.cassandra.service.CassandraServerTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.578 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.service.ConsistencyLevelTest
[junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 1.085 sec
[junit] 
[junit] - Standard Error -
[junit] ERROR 18:24:34,197 unknown endpoint /127.0.0.2
[junit] ERROR 18:24:34,202 unknown endpoint /127.0.0.2
[junit] -  ---
[junit] Testcase: 
testReadWriteConsistencyChecks(org.apache.cassandra.service.ConsistencyLevelTest):
FAILED
[junit] 
[junit] junit.framework.AssertionFailedError: 
[junit] at 
org.apache.cassandra.service.ConsistencyLevelTest.testReadWriteConsistencyChecks(ConsistencyLevelTest.java:165)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.ConsistencyLevelTest FAILED
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
[junit] 
[junit] Testcase: 
org.apache.cassandra.service.EmbeddedCassandraServiceTest:BeforeFirstTest:  
  Caused an ERROR
[junit] Forked Java VM exited abnormally. Please note the time in the 
report does not reflect the time until the VM exit.
[junit] junit.framework.AssertionFailedError: Forked Java VM exited 
abnormally. Please note the time in the report does not reflect the time until 
the VM exit.
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.EmbeddedCassandraServiceTest 
FAILED (crashed)
[junit] Testsuite: org.apache.cassandra.service.InitClientTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0.46 sec
[junit] 
[junit] Testcase: 
testInitClientStartup(org.apache.cassandra.service.InitClientTest):   
Caused an ERROR
[junit]

Re: Ticket CASSANDRA-3578 - Multithreaded CommitLog

2011-12-07 Thread Jeremiah Jordan 
Another option is to have multiple threads reading from the queue and 
writing to their own commit log files.  If you have multiple commit log 
directories with each having its own task writing to it, you can keep 
the "only sequential writes" optimization.  Multiple writers to one disk 
only makes sense if you are using a SSD for storage, other wise you 
don't only have have sequential writes, which would slow down the writing.


On 12/07/2011 10:56 AM, Piotr Kołaczkowski wrote:


Hello everyone,

As an interview task I've got to make CommitLog multithreaded. I'm new 
to Cassandra project and therefore, before I start modifying code, I 
have to make sure I understand what is going on there correctly.

Feel free to correct anything I got wrong or partially wrong.

1. The CommitLog singleton object is responsible for receiving 
RowMutation objects by its add method. The add method is thread-safe 
and is aimed to be called by many threads adding their RowMutations 
independently.


2. Each invocation of CommitLog#add  puts a new task onto the queue. 
This task is represented by LogRecordAdder callable object, which is 
responsible for actually calling the CommitLogSegment#write method for 
doing all the "hard work" of serializing the RowMutation object, 
calculating CRC and writing that to the memory mapped CommitLogSegment 
file buffer. The add method immediately returns a Future object, which 
can be waited for (if needed) - it will block until the row mutation 
is saved to the log file and (optionally) synced.


3. The queued tasks are processed one-by-one, sequentially by the 
appropriate ICommitLogExecutorService. This service also controls 
syncing the active memory mapped segments. There are two sync 
strategies available: periodic and batched. The periodic simply calls 
sync periodically by asynchronously putting appropriate sync task into 
the queue, inbetween the LogRecordAdder tasks. The LogRecordAdder 
tasks are "done" as soon as they are written to the log, so the caller 
*won't wait* for the sync. On the other hand, the batched strategy 
(BatchCommitLogExecutorService), performs the tasks in batches, each 
batch finished with an sync operation. The tasks are marked as done 
*after* the sync operation is finished. This deferred task marking  is 
achieved thanks to CheaterFutureTask class - allowing to run the task 
without immediately marking FutureTask as done. Nice. :)


4. The serialized size of the RowMutation object is calculated twice: 
once before submitting to the ExecutorService - to detect if it is not 
larger than the segment size, and then after being taken from the 
queue for execution - to check if it fits into the active 
CommitLogSegment, and if it doesn't, to activate a new 
CommitLogSegment. Looks to me like a point needing optimisation. I 
couldn't find any code for caching the serialized size to avoid doing 
it twice.


5. The serialization, CRC calculation and actual commit log writes are 
happening sequentially. The aim of this ticket is to make it parallel.


Questions:
1. What happens to the recovery, if the power goes off before the log 
has been synced, and it has been written partially (e.g. it is 
truncated in the middle of the RowMutation data)? Are incomplete 
RowMutation writes detected only by means of CRC (CommitLog around 
lines 237-240), or is there some other mechanism for it?


2. Is the CommitLog#add method allowed to do some heavier 
computations? What is the contract for it? Does it have to return 
immediately or can I move some code into it?


Solutions I consider (please comment):

1. Moving the serialized size calculation, serialization and CRC 
calculation totally before the executor service queue, so that these 
operations would be parallel, and performed once per RowMutation 
object. The calculated size / data array / CRC value would be appended 
to the task and put into the queue. Then copying that into the commit 
log would proceed sequentially - the task would contain only code for 
log writing. This is the safest and easiest solution, but also the 
least performant, because copying is still sequential and still might 
be a bottleneck. The logic of allocating new commit log segments and 
syncing remains unchanged.


2. Moving the serialized size calculation, serialization, CRC 
calculation *and commit log writing* before the executor service 
queue. This raises immediately some problems / questions:
a) The code for segment allocation needs to be changed, as it becomes 
multithreaded. It can be done using AtomicInteger.compareAndSet, so 
that each RowMutation gets its own, non-overlapping piece of commit 
log to write into.
b) What happens if there is not enough free space in the current 
active segment? Do we allow more active segments at once? Or do we 
restrict the parallelism to writing just into a single active segment 
(I don't like it, as it would be for certain less performant, because 
we would have to wait for finishing the current active

Build failed in Jenkins: Cassandra-quick #166

2011-12-07 Thread Apache Jenkins Server 
See 

Changes:

[xedin] merge from 1.0

--
[...truncated 1740 lines...]
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.845 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.locator.TokenMetadataTest
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.458 sec
[junit] 
[junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceCounterTest
[junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.773 sec
[junit] 
[junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceCounterTest):
 Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:487)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:464)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceCounterTest 
FAILED
[junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceStandardTest
[junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.94 sec
[junit] 
[junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceStandardTest):
Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:487)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:464)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceStandardTest 
FAILED
[junit] Testsuite: org.apache.cassandra.service.CassandraServerTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.577 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.service.ConsistencyLevelTest
[junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 1.066 sec
[junit] 
[junit] - Standard Error -
[junit] ERROR 21:36:38,971 unknown endpoint /127.0.0.2
[junit] ERROR 21:36:38,976 unknown endpoint /127.0.0.2
[junit] -  ---
[junit] Testcase: 
testReadWriteConsistencyChecks(org.apache.cassandra.service.ConsistencyLevelTest):
FAILED
[junit] 
[junit] junit.framework.AssertionFailedError: 
[junit] at 
org.apache.cassandra.service.ConsistencyLevelTest.testReadWriteConsistencyChecks(ConsistencyLevelTest.java:165)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.ConsistencyLevelTest FAILED
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
[junit] 
[junit] Testcase: 
org.apache.cassandra.service.EmbeddedCassandraServiceTest:BeforeFirstTest:  
  Caused an ERROR
[junit] Forked Java VM exited abnormally. Please note the time in the 
report does not reflect the time until the VM exit.
[junit] junit.framework.AssertionFailedError: Forked Java VM exited 
abnormally. Please note the time in the report does not reflect the time until 
the VM exit.
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.EmbeddedCassandraServiceTest 
FAILED (crashed)
[junit] Testsuite: org.apache.cassandra.service.InitClientTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0.472 sec
[junit] 
[junit] Testcase: 
testInitClientS

Build failed in Jenkins: Cassandra #1245

2011-12-07 Thread Apache Jenkins Server 
See 

Changes:

[xedin] merge from 1.0

--
[...truncated 2386 lines...]
[junit] Testsuite: org.apache.cassandra.locator.TokenMetadataTest
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.355 sec
[junit] 
[junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceCounterTest
[junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.035 sec
[junit] 
[junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceCounterTest):
 Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:487)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:464)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceCounterTest 
FAILED
[junit] Testsuite: 
org.apache.cassandra.service.AntiEntropyServiceStandardTest
[junit] Tests run: 6, Failures: 0, Errors: 1, Time elapsed: 2.133 sec
[junit] 
[junit] Testcase: 
testValidatorPrepare(org.apache.cassandra.service.AntiEntropyServiceStandardTest):
Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:281)
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:251)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:487)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:464)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTestAbstract.prepare(AntiEntropyServiceTestAbstract.java:80)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceStandardTest 
FAILED
[junit] Testsuite: org.apache.cassandra.service.CassandraServerTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.493 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.service.ConsistencyLevelTest
[junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 1.006 sec
[junit] 
[junit] - Standard Error -
[junit] ERROR 22:20:06,610 unknown endpoint /127.0.0.2
[junit] ERROR 22:20:06,617 unknown endpoint /127.0.0.2
[junit] -  ---
[junit] Testcase: 
testReadWriteConsistencyChecks(org.apache.cassandra.service.ConsistencyLevelTest):
FAILED
[junit] 
[junit] junit.framework.AssertionFailedError: 
[junit] at 
org.apache.cassandra.service.ConsistencyLevelTest.testReadWriteConsistencyChecks(ConsistencyLevelTest.java:165)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.ConsistencyLevelTest FAILED
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
[junit] 
[junit] Testcase: 
org.apache.cassandra.service.EmbeddedCassandraServiceTest:BeforeFirstTest:  
  Caused an ERROR
[junit] Forked Java VM exited abnormally. Please note the time in the 
report does not reflect the time until the VM exit.
[junit] junit.framework.AssertionFailedError: Forked Java VM exited 
abnormally. Please note the time in the report does not reflect the time until 
the VM exit.
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.EmbeddedCassandraServiceTest 
FAILED (crashed)
[junit] Testsuite: org.apache.cassandra.service.InitClientTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0.367 sec
[junit] 
[junit] Testcase: 
testInitClientStartup(org.apache.cassandra.service.InitClientTest):   
Caused an ERROR
[junit] /12

Build failed in Jenkins: Cassandra-0.7 #556

2011-12-07 Thread Apache Jenkins Server 
See 

Changes:

[jbellis] excise mention of 2189 from CHANGES since it didn't actually get fixed

--
[...truncated 1776 lines...]
[junit] 
[junit] Test org.apache.cassandra.locator.DynamicEndpointSnitchTest FAILED
[junit] Testsuite: org.apache.cassandra.locator.EC2SnitchTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.449 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.locator.NetworkTopologyStrategyTest
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.443 sec
[junit] 
[junit] Testsuite: 
org.apache.cassandra.locator.OldNetworkTopologyStrategyTest
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.467 sec
[junit] 
[junit] Testsuite: 
org.apache.cassandra.locator.ReplicationStrategyEndpointCacheTest
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.622 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.locator.SimpleStrategyTest
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.643 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.locator.TokenMetadataTest
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.43 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.service.AntiEntropyServiceTest
[junit] Tests run: 0, Failures: 0, Errors: 1, Time elapsed: 0.558 sec
[junit] 
[junit] Testcase: org.apache.cassandra.service.AntiEntropyServiceTest:  
Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:174)
[junit] at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:420)
[junit] at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:400)
[junit] at 
org.apache.cassandra.service.AntiEntropyServiceTest.prepareClass(AntiEntropyServiceTest.java:71)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.AntiEntropyServiceTest FAILED
[junit] Testsuite: org.apache.cassandra.service.CassandraServerTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.437 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.service.ConsistencyLevelTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.652 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Testsuite: org.apache.cassandra.service.EmbeddedCassandraServiceTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
[junit] 
[junit] Testcase: 
org.apache.cassandra.service.EmbeddedCassandraServiceTest:BeforeFirstTest:  
  Caused an ERROR
[junit] Forked Java VM exited abnormally. Please note the time in the 
report does not reflect the time until the VM exit.
[junit] junit.framework.AssertionFailedError: Forked Java VM exited 
abnormally. Please note the time in the report does not reflect the time until 
the VM exit.
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.EmbeddedCassandraServiceTest 
FAILED (crashed)
[junit] Testsuite: org.apache.cassandra.service.InitClientTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0.454 sec
[junit] 
[junit] Testcase: 
testInitClientStartup(org.apache.cassandra.service.InitClientTest):   
Caused an ERROR
[junit] /127.0.0.1:7010 is in use by another process.  Change 
listen_address:storage_port in cassandra.yaml to values that do not conflict 
with other services
[junit] org.apache.cassandra.config.ConfigurationException: /127.0.0.1:7010 
is in use by another process.  Change listen_address:storage_port in 
cassandra.yaml to values that do not conflict with other services
[junit] at 
org.apache.cassandra.net.MessagingService.listen(MessagingService.java:174)
[junit] at 
org.apache.cassandra.service.StorageService.initClient(StorageService.java:344)
[junit] at 
org.apache.cassandra.service.InitClientTest.testInitClientStartup(InitClientTest.java:33)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.service.InitClientTest FAILED
[junit] Testsuite: org.apache.cassandra.service.MoveTest
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 1.151 sec
[junit] 
[junit] - Standard Error -
[junit]  WARN 00:41:18,872 Node /127.0.0.3 'leaving' token mismatch. Long 
network partition?
[junit] -  ---
[junit] Testsuite: org.

Build failed in Jenkins: Cassandra-0.8 #412

2011-12-07 Thread Apache Jenkins Server 
See 

Changes:

[jbellis] merge from 0.7

--
[...truncated 747 lines...]
AUNEWS.txt
AUbuild.xml
AUNOTICE.txt
A tools
A tools/stress
A tools/stress/src
A tools/stress/src/org
A tools/stress/src/org/apache
A tools/stress/src/org/apache/cassandra
A tools/stress/src/org/apache/cassandra/contrib
A tools/stress/src/org/apache/cassandra/contrib/stress
A tools/stress/src/org/apache/cassandra/contrib/stress/operations
A tools/stress/src/org/apache/cassandra/contrib/stress/util
A tools/stress/src/org/apache/cassandra/stress
A tools/stress/src/org/apache/cassandra/stress/StressAction.java
A tools/stress/src/org/apache/cassandra/stress/Session.java
A tools/stress/src/org/apache/cassandra/stress/Stress.java
A tools/stress/src/org/apache/cassandra/stress/operations
A 
tools/stress/src/org/apache/cassandra/stress/operations/CounterAdder.java
A 
tools/stress/src/org/apache/cassandra/stress/operations/IndexedRangeSlicer.java
A tools/stress/src/org/apache/cassandra/stress/operations/Reader.java
A 
tools/stress/src/org/apache/cassandra/stress/operations/MultiGetter.java
A 
tools/stress/src/org/apache/cassandra/stress/operations/CounterGetter.java
A tools/stress/src/org/apache/cassandra/stress/operations/Inserter.java
A 
tools/stress/src/org/apache/cassandra/stress/operations/RangeSlicer.java
A tools/stress/src/org/apache/cassandra/stress/server
AUtools/stress/src/org/apache/cassandra/stress/server/StressThread.java
AUtools/stress/src/org/apache/cassandra/stress/StressServer.java
A tools/stress/src/org/apache/cassandra/stress/util
AUtools/stress/src/org/apache/cassandra/stress/util/Operation.java
A tools/stress/bin
AUtools/stress/bin/stress
AUtools/stress/bin/stressd
A tools/stress/README.txt
A tools/stress/build.xml
A tools/py_stress
AUtools/py_stress/stress.py
AUtools/py_stress/README.txt
AULICENSE.txt
A contrib
A contrib/pig
A contrib/pig/src
A contrib/pig/src/java
A contrib/pig/src/java/org
A contrib/pig/src/java/org/apache
A contrib/pig/src/java/org/apache/cassandra
A contrib/pig/src/java/org/apache/cassandra/hadoop
A contrib/pig/src/java/org/apache/cassandra/hadoop/pig
AU
contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
A contrib/pig/bin
AUcontrib/pig/bin/pig_cassandra
AUcontrib/pig/README.txt
A contrib/pig/example-script.pig
AUcontrib/pig/build.xml
A contrib/javautils
A contrib/javautils/src
A contrib/javautils/src/test
A contrib/javautils/src/test/java
A contrib/javautils/src/test/java/org
A contrib/javautils/src/test/java/org/apache
A contrib/javautils/src/test/java/org/apache/cassandra
A contrib/javautils/src/test/java/org/apache/cassandra/contrib
A contrib/javautils/src/test/java/org/apache/cassandra/contrib/utils
A 
contrib/javautils/src/test/java/org/apache/cassandra/contrib/utils/service
A 
contrib/javautils/src/test/java/org/apache/cassandra/contrib/utils/service/CassandraServiceTest.java
A contrib/javautils/src/main
A contrib/javautils/src/main/java
A contrib/javautils/src/main/java/org
A contrib/javautils/src/main/java/org/apache
A contrib/javautils/src/main/java/org/apache/cassandra
A contrib/javautils/src/main/java/org/apache/cassandra/contrib
A contrib/javautils/src/main/java/org/apache/cassandra/contrib/utils
A 
contrib/javautils/src/main/java/org/apache/cassandra/contrib/utils/service
A 
contrib/javautils/src/main/java/org/apache/cassandra/contrib/utils/service/CassandraServiceDataCleaner.java
A contrib/javautils/README
A contrib/javautils/build.xml
A interface
A interface/thrift
A interface/thrift/gen-java
A interface/thrift/gen-java/org
A interface/thrift/gen-java/org/apache
A interface/thrift/gen-java/org/apache/cassandra
A interface/thrift/gen-java/org/apache/cassandra/thrift
AUinterface/thrift/gen-java/org/apache/cassandra/thrift/Mutation.java
AU
interface/thrift/gen-java/org/apache/cassandra/thrift/UnavailableException.java
A interface/thrift/gen-java/org/apache/cassandra/thrift/IndexClause.java
A 
interface/thrift/gen-java/org/apache/cassandra/thrift/SchemaDisagreementException.java
A interface/thrift/gen-java/org/apache/cassandra/thrift/IndexType.java
A 
interface/thrift/gen-java/org/apache/cassandra/thrift/TimedOutException.java
A interface/thrift/gen-java/org/apache/c

Jenkins build became unstable: Cassandra-0.7-Coverage #95

2011-12-07 Thread Apache Jenkins Server 
See

Jenkins build is still unstable: Cassandra-0.8-Coverage #155

2011-12-07 Thread Apache Jenkins Server 
See

Jenkins build is still unstable: Cassandra-Coverage #184

2011-12-07 Thread Apache Jenkins Server 
See

Re: Ticket CASSANDRA-3578 - Multithreaded CommitLog

2011-12-07 Thread Piotr Kołaczkowski 


Right, this would be the best option to have an ability to write into 
multiple log files, put on multiple disks. I'm not sure if it is part of 
that ticket, though. Maybe we should split it into two things: parallel 
serialization / CRC and parallel writes to multiple logfiles (as another 
ticket). Looks like a major commitlog refactoring, including touching 
the logfile segment management and logfile recovery code.


BTW: I'm not so sure if multiple, parallel writes to a memory mapped 
file would be actually slower or faster than sequential writes. I think 
the OS would optimise the writes so that physically they would be 
sequential, or even delay them until fsync (or low cached disk buffers), 
so no performance loss would occur, while moving data from temporary 
array to shared buffer memory would be actually faster (and possibility 
of avoiding temporary arrays by serializing directly into the shared 
buffer at all is also  promising). I think we should benchmark / profile 
this first (I can do it) and see how it is in reality, unless someone 
has already done that. If you are interested, I can find some time today 
evening to do it.



W dniu 2011-12-07 21:57, Jeremiah Jordan pisze:


Another option is to have multiple threads reading from the queue and 
writing to their own commit log files.  If you have multiple commit 
log directories with each having its own task writing to it, you can 
keep the "only sequential writes" optimization.  Multiple writers to 
one disk only makes sense if you are using a SSD for storage, other 
wise you don't only have have sequential writes, which would slow down 
the writing.


On 12/07/2011 10:56 AM, Piotr Kołaczkowski wrote:


Hello everyone,

As an interview task I've got to make CommitLog multithreaded. I'm 
new to Cassandra project and therefore, before I start modifying 
code, I have to make sure I understand what is going on there correctly.

Feel free to correct anything I got wrong or partially wrong.

1. The CommitLog singleton object is responsible for receiving 
RowMutation objects by its add method. The add method is thread-safe 
and is aimed to be called by many threads adding their RowMutations 
independently.


2. Each invocation of CommitLog#add  puts a new task onto the queue. 
This task is represented by LogRecordAdder callable object, which is 
responsible for actually calling the CommitLogSegment#write method 
for doing all the "hard work" of serializing the RowMutation object, 
calculating CRC and writing that to the memory mapped 
CommitLogSegment file buffer. The add method immediately returns a 
Future object, which can be waited for (if needed) - it will block 
until the row mutation is saved to the log file and (optionally) synced.


3. The queued tasks are processed one-by-one, sequentially by the 
appropriate ICommitLogExecutorService. This service also controls 
syncing the active memory mapped segments. There are two sync 
strategies available: periodic and batched. The periodic simply calls 
sync periodically by asynchronously putting appropriate sync task 
into the queue, inbetween the LogRecordAdder tasks. The 
LogRecordAdder tasks are "done" as soon as they are written to the 
log, so the caller *won't wait* for the sync. On the other hand, the 
batched strategy (BatchCommitLogExecutorService), performs the tasks 
in batches, each batch finished with an sync operation. The tasks are 
marked as done *after* the sync operation is finished. This deferred 
task marking  is achieved thanks to CheaterFutureTask class - 
allowing to run the task without immediately marking FutureTask as 
done. Nice. :)


4. The serialized size of the RowMutation object is calculated twice: 
once before submitting to the ExecutorService - to detect if it is 
not larger than the segment size, and then after being taken from the 
queue for execution - to check if it fits into the active 
CommitLogSegment, and if it doesn't, to activate a new 
CommitLogSegment. Looks to me like a point needing optimisation. I 
couldn't find any code for caching the serialized size to avoid doing 
it twice.


5. The serialization, CRC calculation and actual commit log writes 
are happening sequentially. The aim of this ticket is to make it 
parallel.


Questions:
1. What happens to the recovery, if the power goes off before the log 
has been synced, and it has been written partially (e.g. it is 
truncated in the middle of the RowMutation data)? Are incomplete 
RowMutation writes detected only by means of CRC (CommitLog around 
lines 237-240), or is there some other mechanism for it?


2. Is the CommitLog#add method allowed to do some heavier 
computations? What is the contract for it? Does it have to return 
immediately or can I move some code into it?


Solutions I consider (please comment):

1. Moving the serialized size calculation, serialization and CRC 
calculation totally before the executor service queue, so that these 
operations would be parallel, and performed once

Re: Ticket CASSANDRA-3578 - Multithreaded CommitLog

2011-12-07 Thread Jonathan Ellis 
2011/12/8 Piotr Kołaczkowski :
> Right, this would be the best option to have an ability to write into
> multiple log files, put on multiple disks. I'm not sure if it is part of
> that ticket, though.

It's not.  I don't think anyone needs more than 80MB/s or so of
commitlog bandwidth for a while.

> BTW: I'm not so sure if multiple, parallel writes to a memory mapped file
> would be actually slower or faster than sequential writes. I think the OS
> would optimise the writes so that physically they would be sequential, or
> even delay them until fsync (or low cached disk buffers), so no performance
> loss would occur

Right.  What we're trying to fix here is having a single thread doing
the copying + checksumming being a bottleneck.  The i/o pattern should
stay more or less the same.

-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com