Working on Wiki page for uploading to central
FYI, I have started working on a wiki page for uploading to Maven Central. http://wiki.apache.org/cassandra/HowToPublishToMavenCentral There is an open issue: namely where are the public GPG keys of Cassandra Developers kept so that 3rd parties can verify that the release they download from central is an official release. I am going to work on a tidy-up of my "works on my machine" patch to release to central, and I hope to have that in a form that can be committed to trunk and 0.7 in the next couple of days. Once that patch gets applied I will be able to finish the wiki page, at which point I suggest that we link to it from one of the developer pages. It might be no harm to re-name the page HowToRelease and add content about running a release build and the voting process... but I'm only suggesting like ;-) Anywho, this was just an update to let people know that I'm on the case! -Stephen
Re: Exposing Cassandra as a Web Service
It wouldn't be hard to do. Cassandra is structured in such a way that it is pretty easy to wrap a transport around a few classes that handle most of the client interaction. Last September I created a RESTful-like version of Cassandra in just a few hours. I haven't maintained the branch, but you can find the results of the work here: https://github.com/gdusbabek/cassandra Gary On Sun, Jan 16, 2011 at 10:50, indika kumara wrote: > Hi All, > > What would you think about the idea of exposing the Cassandra as a Web > Service so that any web service client can connect to the Cassandra server? > > This would require embedding a web service engine in the server side, > providing WSDL(s) for Cassandra’s services such as management, data access, > etc. > > If this task can be done, it is possible to connect to the Cassandra through > different languages, different transports such as TCP, HTTP, JMS, etc. > Moreover, the connection between clients and the Cassandra can be secure > (WS-Security), and reliable. > > If this task is worth, I may be able to contribute for implementing it. I am > familiar with the Axis2 web service engine [1], and an Apache committer. > > Your suggestions are welcome! > > Thanks, > > Indika > > [1] http://axis.apache.org/axis2/java/core/ >
Re: Exposing Cassandra as a Web Service
Hi Gary, Thank you very much for the information. BTW, It seems that your transport code is not in the Cassandra's trunk. Is it going add that code into the trunk? BTW, do you think what I suggested would be worth? If so, I would like to implement it. Thanks, Indika On Mon, Jan 17, 2011 at 7:05 PM, Gary Dusbabek wrote: > It wouldn't be hard to do. Cassandra is structured in such a way that > it is pretty easy to wrap a transport around a few classes that handle > most of the client interaction. Last September I created a > RESTful-like version of Cassandra in just a few hours. I haven't > maintained the branch, but you can find the results of the work here: > https://github.com/gdusbabek/cassandra > > Gary > > On Sun, Jan 16, 2011 at 10:50, indika kumara wrote: > > Hi All, > > > > What would you think about the idea of exposing the Cassandra as a Web > > Service so that any web service client can connect to the Cassandra > server? > > > > This would require embedding a web service engine in the server side, > > providing WSDL(s) for Cassandra’s services such as management, data > access, > > etc. > > > > If this task can be done, it is possible to connect to the Cassandra > through > > different languages, different transports such as TCP, HTTP, JMS, etc. > > Moreover, the connection between clients and the Cassandra can be secure > > (WS-Security), and reliable. > > > > If this task is worth, I may be able to contribute for implementing it. I > am > > familiar with the Axis2 web service engine [1], and an Apache committer. > > > > Your suggestions are welcome! > > > > Thanks, > > > > Indika > > > > [1] http://axis.apache.org/axis2/java/core/ > > >
Re: Exposing Cassandra as a Web Service
No, it's not in trunk. I created the branch as a proof-of-concept only. I can see the utility of a RESTful interface, but I doubt you'll be able to achieve performance comparable to thrift. Gary On Mon, Jan 17, 2011 at 07:23, indika kumara wrote: > Hi Gary, > > Thank you very much for the information. BTW, It seems that your transport > code is not in the Cassandra's trunk. Is it going add that code into the > trunk? BTW, do you think what I suggested would be worth? If so, I would > like to implement it. > > Thanks, > > Indika > > On Mon, Jan 17, 2011 at 7:05 PM, Gary Dusbabek wrote: >> >> It wouldn't be hard to do. Cassandra is structured in such a way that >> it is pretty easy to wrap a transport around a few classes that handle >> most of the client interaction. Last September I created a >> RESTful-like version of Cassandra in just a few hours. I haven't >> maintained the branch, but you can find the results of the work here: >> https://github.com/gdusbabek/cassandra >> >> Gary >> >> On Sun, Jan 16, 2011 at 10:50, indika kumara wrote: >> > Hi All, >> > >> > What would you think about the idea of exposing the Cassandra as a Web >> > Service so that any web service client can connect to the Cassandra >> > server? >> > >> > This would require embedding a web service engine in the server side, >> > providing WSDL(s) for Cassandra’s services such as management, data >> > access, >> > etc. >> > >> > If this task can be done, it is possible to connect to the Cassandra >> > through >> > different languages, different transports such as TCP, HTTP, JMS, etc. >> > Moreover, the connection between clients and the Cassandra can be secure >> > (WS-Security), and reliable. >> > >> > If this task is worth, I may be able to contribute for implementing it. >> > I am >> > familiar with the Axis2 web service engine [1], and an Apache committer. >> > >> > Your suggestions are welcome! >> > >> > Thanks, >> > >> > Indika >> > >> > [1] http://axis.apache.org/axis2/java/core/ >> > > >
Re: Exposing Cassandra as a Web Service
Thanks Gary A HTTP transport based on NIO [1] may offer a considerable performance. I may implement it locally and compare the performance. Indika [1] http://hc.apache.org/httpcomponents-core-ga/httpcore-nio/index.html On Mon, Jan 17, 2011 at 8:01 PM, Gary Dusbabek wrote: > No, it's not in trunk. I created the branch as a proof-of-concept > only. I can see the utility of a RESTful interface, but I doubt > you'll be able to achieve performance comparable to thrift. > > Gary > > On Mon, Jan 17, 2011 at 07:23, indika kumara > wrote: > > Hi Gary, > > > > Thank you very much for the information. BTW, It seems that your > transport > > code is not in the Cassandra's trunk. Is it going add that code into the > > trunk? BTW, do you think what I suggested would be worth? If so, I would > > like to implement it. > > > > Thanks, > > > > Indika > > > > On Mon, Jan 17, 2011 at 7:05 PM, Gary Dusbabek > wrote: > >> > >> It wouldn't be hard to do. Cassandra is structured in such a way that > >> it is pretty easy to wrap a transport around a few classes that handle > >> most of the client interaction. Last September I created a > >> RESTful-like version of Cassandra in just a few hours. I haven't > >> maintained the branch, but you can find the results of the work here: > >> https://github.com/gdusbabek/cassandra > >> > >> Gary > >> > >> On Sun, Jan 16, 2011 at 10:50, indika kumara wrote: > >> > Hi All, > >> > > >> > What would you think about the idea of exposing the Cassandra as a Web > >> > Service so that any web service client can connect to the Cassandra > >> > server? > >> > > >> > This would require embedding a web service engine in the server side, > >> > providing WSDL(s) for Cassandra’s services such as management, data > >> > access, > >> > etc. > >> > > >> > If this task can be done, it is possible to connect to the Cassandra > >> > through > >> > different languages, different transports such as TCP, HTTP, JMS, etc. > >> > Moreover, the connection between clients and the Cassandra can be > secure > >> > (WS-Security), and reliable. > >> > > >> > If this task is worth, I may be able to contribute for implementing > it. > >> > I am > >> > familiar with the Axis2 web service engine [1], and an Apache > committer. > >> > > >> > Your suggestions are welcome! > >> > > >> > Thanks, > >> > > >> > Indika > >> > > >> > [1] http://axis.apache.org/axis2/java/core/ > >> > > > > > >
Re: Cassandra-Maven-Plugin
https://issues.apache.org/jira/browse/CASSANDRA-1997 On 16 January 2011 19:59, Stephen Connolly wrote: > it will be an attachment to an as yet un raised jira. look out for it > tomorrow/tuesday > > - Stephen > > --- > Sent from my Android phone, so random spelling mistakes, random nonsense > words and other nonsense are a direct result of using swype to type on the > screen > > On 16 Jan 2011 17:52, "Hellmut Adolphs" wrote: >
Avro in the Cassandra core
Hi, I'd to discuss if/when we should be using Avro or any serialization tool in the Cassandra core. Some context: We have begun the process of removing Avro from the service layer CASSANDRA-926. We currently use Avro for schema migrations internally, and we have two open items that are using Avro for our internal file storage. CASSANDRA-1472 and CASSANDRA-674. My opinion is we need to control the lowest layers of the code and not rely on a third party library. By using a third party library like Avro, it becomes a black box that we need to deeply understand and work around. Also, since Avro is developed separately we have another core dependency that could disrupt releases (say a bug in Avro). The limitation of using a generic serialization tool is it uses the most general approach to things which may not be the best when you can optimize differently based on the specifics of your data. Examples: Block based compression, Auto-boxing of primitives, Code generation. Now, there may in fact be ways of doing everything we want in Avro. And I'm sure this mail will cause a lot of opinions to be voiced, but the thing I want everyone to keep in mind is we *ALL* would need to be willing to become experts in Avro to allow us to hack in and around it. If we don't we end up with a disjointed codebase. Thanks, -Jake
Re: Avro in the Cassandra core
On Mon, Jan 17, 2011 at 11:12 AM, Jake Luciani wrote: > My opinion is we need to control the lowest layers of the code and not rely > on a third party library. +1. I think that a large part of we want to do at the bits-on-disk level is a poor fit for the kind of serialized objects-with-fields that Avro is best at. -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of Riptano, the source for professional Cassandra support http://riptano.com
Re: Avro in the Cassandra core
On Mon, Jan 17, 2011 at 11:12, Jake Luciani wrote: > Hi, > > I'd to discuss if/when we should be using Avro or any serialization tool in > the Cassandra core. > > Some context: We have begun the process of removing Avro from the service > layer CASSANDRA-926. We currently use Avro for schema migrations internally, > and we have two open items that are using Avro for our internal file > storage. CASSANDRA-1472 and CASSANDRA-674. > > My opinion is we need to control the lowest layers of the code and not rely > on a third party library. By using a third party library like Avro, it > becomes a black box that we need to deeply understand and work around. +1. We need to control serialization in many cases so that we can provide interoperability in the face of radically changing the way we store bytes. It happens often enough that it is a valid concern. > > Now, there may in fact be ways of doing everything we want in Avro. And I'm > sure this mail will cause a lot of opinions to be voiced, but the thing I > want everyone to keep in mind is we *ALL* would need to be willing to become > experts in Avro to allow us to hack in and around it. If we don't we end up > with a disjointed codebase. I think serialization will have to be evaluated on a per-ticket basis. In some cases, it might make sense to hand it off to a library. As for standardizing on a particular lib for serialization--I prefer the promise of avro serialization to thrift (avro is a tad more flexible), but we already use thrift--maybe we should just standardize on it. My experience so far with using avro for migrations serialization indicates that we are either using avro inappropriately, or it just doesn't deliver on the promise of deserializing data with slightly different schemas. If you want to see first-hand what I'm talking about, copy system tables from an 0.7 cluster into a trunk config and watch the breakage. Gary.
Re: Avro in the Cassandra core
On Mon, 2011-01-17 at 12:12 -0500, Jake Luciani wrote: > Some context: We have begun the process of removing Avro from the > service layer CASSANDRA-926. We currently use Avro for schema > migrations internally, and we have two open items that are using Avro > for our internal file storage. CASSANDRA-1472 and CASSANDRA-674. FWIW, this should be done (removing the RPC interface). Anything missed is deserving of a bug report > My opinion is we need to control the lowest layers of the code and not > rely on a third party library. By using a third party library like > Avro, it becomes a black box that we need to deeply understand and > work around. Also, since Avro is developed separately we have another > core dependency that could disrupt releases (say a bug in Avro). +1 The Avro RPC interface was an experiment, and it was always the case that if it didn't supplant the Thrift interface as status quo, that it'd be removed. However, as I remember it, part of the justification for using it in migrations was that it was already there. In hindsight that was probably a mistake. Anyway, we have too many dependencies as it is, I'd rather move toward eliminating it entirely unless there is a very compelling reason not to (I don't think there is). -- Eric Evans eev...@rackspace.com
Re: Avro in the Cassandra core
On Mon, Jan 17, 2011 at 9:01 PM, Eric Evans wrote: > > My opinion is we need to control the lowest layers of the code and not > > rely on a third party library. By using a third party library like > > Avro, it becomes a black box that we need to deeply understand and > > work around. Also, since Avro is developed separately we have another > > core dependency that could disrupt releases (say a bug in Avro). > > +1 > +1 > Anyway, we have too many dependencies as it is, I'd rather move toward > eliminating it entirely unless there is a very compelling reason not to > (I don't think there is). +1. It seems that once we introduced the now removed avro interface, it started infecting everything else 'because it was there.' -Brandon
Re: Avro in the Cassandra core
On 18 January 2011 03:01, Eric Evans wrote: > On Mon, 2011-01-17 at 12:12 -0500, Jake Luciani wrote: >> Some context: We have begun the process of removing Avro from the >> service layer CASSANDRA-926. We currently use Avro for schema >> migrations internally, and we have two open items that are using Avro >> for our internal file storage. CASSANDRA-1472 and CASSANDRA-674. > > FWIW, this should be done (removing the RPC interface). Anything missed > is deserving of a bug report > >> My opinion is we need to control the lowest layers of the code and not >> rely on a third party library. By using a third party library like >> Avro, it becomes a black box that we need to deeply understand and >> work around. Also, since Avro is developed separately we have another >> core dependency that could disrupt releases (say a bug in Avro). > > +1 > > The Avro RPC interface was an experiment, and it was always the case > that if it didn't supplant the Thrift interface as status quo, that it'd > be removed. However, as I remember it, part of the justification for > using it in migrations was that it was already there. In hindsight that > was probably a mistake. > > Anyway, we have too many dependencies as it is, I'd rather move toward +1 I'd also be in favour of limiting the scope of dependencies so that they are not _everywhere_ -Stephen > eliminating it entirely unless there is a very compelling reason not to > (I don't think there is). > > -- > Eric Evans > eev...@rackspace.com > >
