Re: Problem in running code using thrift api and cassandra
Yes Pushkar.. even i thought so. But if it was a problem then it would not
have been executed while debugging also if i am not wrong.. and moreover
when i exclude cassandra and keep the same logic it works..
On Sun, Mar 10, 2013 at 7:22 PM, Pushkar Prasad <
pushkar.pra...@airtightnetworks.net> wrote:
> Sounds like you are hitting a 16 bit limit someplace.
>
> -Original Message-
> From: Annu Agrawal [mailto:annu.agrawa...@gmail.com]
> Sent: 10 March 2013 19:00
> To: dev@cassandra.apache.org
> Subject: Problem in running code using thrift api and cassandra
>
> Hi ,
>
> The following program has a loop which reads a value from cassandra column
> which is stored in the form of string. In every iteration of the loop it
> reads that value converts it in unsigned long and adds 4096 in it . Then
> again it inserts this value in cassandra.
>
> The loop stops incrementing after some iterations. I have tried the same
> program without inserting in cassandra and it works totally fine.
>
> But the following program works fine till 'size' variable reaches 61440.
> After this iteration i.e 61440+4096=65536 the value read from cassandra
> continues to be 61440 only untill the loop ends. So this value is not
> getting inserted in cassandra.
>
> This program works absolutely fine when debugged. So I am not able to find
> where is exactly the problem.
> I am deadly stuck up here.. Is there any problem with jvm heap space or
> should any changes be made in cassandra.yaml file. Please help..
>
>
> #include "../gen-cpp/Cassandra.h"
> #include
> #include
> #include
> #include
> #include
> #include
> #include
> #include
> #include "base64.h"
>
> using namespace std;
> using namespace apache::thrift;
> using namespace apache::thrift::protocol;
> using namespace apache::thrift::transport;
> using namespace org::apache::cassandra;
> using namespace boost;
>
> static string host("127.0.0.1");
> static int port= 9160;
>
> //generate timestamp
> int64_t getTS()
> {
> time_t ltime;
> ltime=time(NULL);
> return (int64_t)ltime;
> }
>
> int main()
> {
> shared_ptr socket(new TSocket(host, port));
> shared_ptr transport(new TFramedTransport(socket));
> shared_ptr protocol(new TBinaryProtocol(transport));
>
> string base64,metadata;
> unsigned long size=0, *sizep;
> int ch;
> sizep=&size;
>
> CassandraClient client(protocol);
> ColumnPath cpath;
> ColumnParent cp;
> ColumnOrSuperColumn csc;
> Column c1;
>
> cp.column_family.assign("mycolfamily");
> cp.super_column.assign("");
>
> cpath.column_family.assign("mycolfamily");
> cpath.__set_column("data");
>
> transport->open();
> client.set_keyspace("mykeyspace");
>
> base64= base64_encode(UCCP(sizep),sizeof(*sizep));//converts
> unsigned long to ASCII string
>
> c1.__set_name("data");
> c1.__set_value(base64);
> c1.__set_timestamp(getTS());
> client.insert("mykey", cp, c1,
> org::apache::cassandra::ConsistencyLevel::ONE);
>
> for(ch=0;ch<30;ch++)
> {
> //reading from cassandra
> client.get(csc, "mykey", cpath,
> org::apache::cassandra::ConsistencyLevel::ONE);
>
> metadata = base64_decode(csc.column.value); //converts string
> to unsigned long
> memcpy(sizep,metadata.data(),metadata.size());
> sizep = (unsigned long *)sizep;
> cout<<"\n Read value : "<< *sizep ;
>
> // incrementing size
> *sizep = *sizep + 4096;
>
> //writing size in cassandra
> base64= base64_encode(UCCP(sizep),sizeof(*sizep));//converts
> unsigned long to ASCII string
> c1.__set_name("data");
> c1.__set_value(base64);
> c1.__set_timestamp(getTS());
> client.insert("mykey", cp, c1,
> org::apache::cassandra::ConsistencyLevel::ONE);
>
> cout<<"\n Written value : "<< *sizep <<"\n\n";
> }
> transport->close();
> return 0;
> }
>
>
>
> --
> Thanks,
> Annu Agrawal
>
>
>
--
Thanks,
Annu Agrawal
Re: Cassandra 4021
I think you've nailed it. I've committed this fix to 1.1.11 and 1.2.3. On Fri, Mar 8, 2013 at 5:59 PM, Boris Yen wrote: > Use the following sstable on our file system as an example. > > rw-rr- 1 byan admin 333 Mar 5 15:26 indexBytes-hd-115-Index.db > rw-rr- 1 byan admin 66305 Mar 5 15:26 indexBytes-tmp-hd-115-Data.db > rw-rr- 1 byan admin 67 Mar 5 15:26 indexBytes-tmp-hd-115-Digest.sha1 > rw-rr- 1 byan admin 1936 Mar 5 15:26 indexBytes-tmp-hd-115-Filter.db > > The reason why we tweak the hashcode and equals methods is that the > Descriptor for indexBytes-hd-115-Index.db and indexBytes-tmp-hd-115-Data.db > are the same. If the Descriptor is used as key in a map and > indexBytes-hd-115-Index.db > get listed first, the C* will consider that the there are four components > for this Descriptor [Digest.sha1, Index.db, Filter.db, Data.db]. In this > case, it is not correct. When removing orphans for sstable, the Descriptor > will be used to create the filename to delete files, that is why C* cannot > find the file to delete. > > Regards, > Boris > > On Sat, Mar 9, 2013 at 12:54 AM, Jonathan Ellis wrote: > >> Thanks, Boris. Can you explain what the cause is, and why tweaking hashCode >> fixes it? >> On Mar 8, 2013 4:17 AM, "Boris Yen" wrote: >> >> > Hi, >> > >> > We have run into this a few times on 1.0.10. It seems this happens when >> > the file-rename after compaction is not completed. We have created a >> patch >> > for this. However, in order to make sure we do not break anything, I was >> > wondering if anyone could help us review the changes to see if this would >> > cause any issue or not. >> > >> > Thanks and Regards, >> > Boris >> > >> -- Jonathan Ellis Project Chair, Apache Cassandra co-founder, http://www.datastax.com @spyced
