Re: 2.0
I agree on 2.0. For the thrift part, we've said clearly that we wouldn't remove it any time soon so let's stick to that. Besides, I would agree it's too soon anyway. What we can do however in the relatively short term on that front, is to pull thrift in it's own jar (we've almost removed all internal dependencies on thrift, and the few remaining ones will be easy to kill) and make that jar optional if you don't want to use it. -- Sylvain On Sat, Dec 1, 2012 at 2:52 AM, Ray Slakinski wrote: > I agree, I don't think its a great idea to drop thrift until the back > end tools are 100% compatible and have some level of agreement from the > major users of > Cassandra. > > Paying off technical dept though I'm all for, and I think its key to the > long term success of the application. Right now Supercolumns to someone > new coming to the system might think "Hey, these things look great. Lets > use them" and in a few months time hate all things that are cassandra. > > Ray Slakinski > > On 12/01, Jonathan Ellis wrote: > > As attractive as it would be to clean house, I think we owe it to our > > users to keep Thrift around for the forseeable future rather than > > orphan all Thrift-using applications (which is virtually everyone) on > > 1.2. > > > > On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown > wrote: > > > Hi Jonathan, > > > > > > I'm in favor of paying off the technical debt, as well, and I wonder if > > > there is value in removing support for thrift with 2.0? We're > currently in > > > 'do as little as possible' mode with thrift, so should we aggressively > cast > > > it off and push the binary CQL protocol? Seems like a jump to '2.0', > along > > > with the other initiatives, would be a reasonable time/milestone to do > so. > > > > > > Thanks, > > > > > > -Jason > > > > > > > > > On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis > wrote: > > > > > >> The more I think about it, the more I think we should call 1.2-next, > > >> 2.0. I'd like to spend some time paying off our technical debt: > > >> > > >> - replace supercolumns with composites (CASSANDRA-3237) > > >> - rewrite counters (CASSANDRA-4775) > > >> - improve storage engine support for wide rows > > >> - better stage management to improve latency (disruptor? lightweight > > >> threads? custom executor + queue?) > > >> - improved repair (CASSANDRA-3362, 2699) > > >> > > >> Of course, we're planning some new features as well: > > >> - triggers (CASSANDRA-1311) > > >> - improved query fault tolerance (CASSANDRA-4705) > > >> - row size limits (CASSANDRA-3929) > > >> - cql3 integration for hadoop (CASSANDRA-4421) > > >> - improved caching (CASSANDRA-1956, 2864) > > >> > > >> -- > > >> Jonathan Ellis > > >> Project Chair, Apache Cassandra > > >> co-founder, http://www.datastax.com > > >> @spyced > > >> > > > > > > > > -- > > Jonathan Ellis > > Project Chair, Apache Cassandra > > co-founder, http://www.datastax.com > > @spyced >
Hector 0.8.0-2 update fails with : "All host pools marked down. Retry burden pushed out to client."
Hi, I am working on a POC for cassandra 1.1.6 with hector 0.8.0-2. It fails on a update with the following exception. Anyone has seen this error before? Please could you help.. Hector throws an HectorException: Exception message : "All host pools marked down. Retry burden pushed out to client." java.lang.NullPointerException at me.prettyprint.cassandra.model.MutatorImpl.execute(MutatorImpl.java:217) at me.prettyprint.cassandra.service.template.AbstractColumnFamilyTemplate.executeBatch(AbstractColumnFamilyTemplate.java:127) at me.prettyprint.cassandra.service.template.AbstractColumnFamilyTemplate.executeIfNotBatched(AbstractColumnFamilyTemplate.java:162) at me.prettyprint.cassandra.service.template.SuperCfTemplate.update(SuperCfTemplate.java:198) at com.visa.shared.cache.poc.dao.AbstractSuperColumnDataAccess.update(AbstractSuperColumnDataAccess.java:188) at com.visa.shared.cache.poc.domain.InterfaceStationProcessor.markLinkUp(InterfaceStationProcessor.java:47) at com.visa.shared.cache.poc.domain.InterfaceStationProcessor.processRequest(InterfaceStationProcessor.java:78) at com.visa.shared.cache.poc.domain.SwitchService.processRequest(SwitchService.java:78) at com.visa.shared.cache.poc.domain.SwitchService.access$000(SwitchService.java:13) at com.visa.shared.cache.poc.domain.SwitchService$1.run(SwitchService.java:59) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Exception in thread "switchServiceTaskExceutor-1242" java.lang.NullPointerException at me.prettyprint.cassandra.model.MutatorImpl.addInsertion(MutatorImpl.java:158) at me.prettyprint.cassandra.service.template.SuperCfUpdater.updateInternal(SuperCfUpdater.java:115) at me.prettyprint.cassandra.service.template.SuperCfTemplate.update(SuperCfTemplate.java:196) Cassandra logs in /var/log/cassandra/system.log: INFO [FlushWriter:8] 2012-12-01 17:01:13,200 Memtable.java (line 264) Writing Memtable-jpp_0001@1896150523(17573964/21967455 serialized/live bytes, 1055 ops) ERROR [Thrift:1030] 2012-12-01 17:01:14,324 CustomTThreadPoolServer.java (line 200) Thrift error occurred during processing of message. org.apache.thrift.TException: Negative length: -2147418111 at org.apache.thrift.protocol.TBinaryProtocol.checkReadLength(TBinaryProtocol.java:388) at org.apache.thrift.protocol.TBinaryProtocol.readBinary(TBinaryProtocol.java:363) at org.apache.cassandra.thrift.Cassandra$batch_mutate_args.read(Cassandra.java:19724) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:21) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34) at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Regards Jay VISA EUROPE is a technology business that provides the brand, systems, services and rules that make electronic payments between millions of consumers, retailers, businesses and governments happen. Visa Europe is a membership association of more than 3,700 members that includes banks and other payment service providers from 36 countries across Europe. We continually invest and innovate to create new and better ways to pay and be paid. For more information, please visit www.visaeurope.com. Please consider the environment before printing this email. This email (including attachments) is confidential and is solely intended for the addressee. Unless you are the addressee, you may not read, use or store this email in any way, or permit others to. If you have received it in error, please contact Visa Europe on +44 (0)20 7937 8111. Visa Europe Services Inc. is a company incorporated in Delaware USA, acting through its UK Establishment (UK Establishment number BR007632) whose registered office is at 1 Sheldon Square, London, W2 6TT.
Re: 2.0
I do not understand why everyone wants to force this issue on removing thrift. If cql, cql sparse tables and the new transport are better people will naturally begin to use them, but as it stands now I see the it this way: Thrift still has more clients for more languages, thrift has more higher level clients for more languages. Thrift has Hadoop support hive support and pig support in the wild. Thrift has third party tools like Orm tools, support for tools like flume. Most of cql3 features like collections do not work with compact tables, and compact tables are much more space efficient then their cql3 sparse counterparts, composite rows with UTf column names, blank rows, etc. Cql3 binary client is only available for in beta stage for a few languages. So the project can easily remove thrift today but until a majority of the tooling by the community adopts the transport and for the most part cqls sparse tables it is not going to mean anything. Many people already have code live in production working fine with the old toolset and will be unwilling to convert something just because Think about it like this a company like mine that already has something in production. Even if you could convince me us that cql native transport was better, which by the way no one has showed me a vast performance reason to this point, they still may not want to invest the resources to convert their app. Many companies endured the painful transition from Cassandra 0.6 to Cassandra 0.7 conversion and they are not eagerly going to entertain another change which is mostly cosmetic. Also I find issues like this extremely frustrating. https://issues.apache.org/jira/browse/CASSANDRA-4924 It seems like the project is drawing a hard line in the sand dividing people. Is it the case that cql3's sparse tables can't be accessed by thrift, or is it the case that no one wants to make this happen? Like is it technically impossible? It seems not to me in Cassandra Row key, column, and value are all still byte arrays right? So I do not see why thrift users need to be locked out of them. Just like composites we will figure out how to pack the bytes. I hope that we can stop talking about removing thrift until there is some consensus between active users that it is not in use anymore. This consensus is not as simple as n committers saying that something is technically not needed anymore. It has to look at the users, the number of clients, the number of languages, the number of high level tools available. In the mean time when issues like 4924 pop up it would be better if people tried to find solutions for maximum forward and backward compatibility instead of drawing a line and trying to shut thrift users out of things. Avro was much the same way . I had a spirited debate on irc and got basicallly insulted because i belived thrift was not dead. The glory of avro never came true because it really did not work for clients outside a few languages. Cql and the binary transport has to pass this same litmus test. Let it gain momentum and have rock solid clients for 5 languages and have higher level tools written on top of it then its easy to say thrift is not needed anymore. On Saturday, December 1, 2012, Sylvain Lebresne wrote: > I agree on 2.0. > > For the thrift part, we've said clearly that we wouldn't remove it any time > soon so let's stick to that. Besides, I would agree it's too soon anyway. > What we can do however in the relatively short term on that front, is to > pull thrift in it's own jar (we've almost removed all internal dependencies > on thrift, and the few remaining ones will be easy to kill) and make that > jar optional if you don't want to use it. > > -- > Sylvain > > > On Sat, Dec 1, 2012 at 2:52 AM, Ray Slakinski > > >wrote: > > > I agree, I don't think its a great idea to drop thrift until the back > > end tools are 100% compatible and have some level of agreement from the > > major users of > > Cassandra. > > > > Paying off technical dept though I'm all for, and I think its key to the > > long term success of the application. Right now Supercolumns to someone > > new coming to the system might think "Hey, these things look great. Lets > > use them" and in a few months time hate all things that are cassandra. > > > > Ray Slakinski > > > > On 12/01, Jonathan Ellis wrote: > > > As attractive as it would be to clean house, I think we owe it to our > > > users to keep Thrift around for the forseeable future rather than > > > orphan all Thrift-using applications (which is virtually everyone) on > > > 1.2. > > > > > > On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown > > > > > > > wrote: > > > > Hi Jonathan, > > > > > > > > I'm in favor of paying off the technical debt, as well, and I wonder > if > > > > there is value in removing support for thrift with 2.0? We're > > currently in > > > > 'do as little as possible' mode with thrift, so should we > aggressively > > cast > > > > it off and push the binary CQL protocol? Seems lik
Re: Hector 0.8.0-2 update fails with : "All host pools marked down. Retry burden pushed out to client."
after 1 year experience with Hector, i would recommend: stay away from hector if possible.
