Re: Unexpected value after insert
Given your use case I'd recommend not using a counter for the field. Are you logging the update command you are sending Cassandra? Are you sure you aren't overflowing the value either in Cassandra or in your batch code? On Monday, November 4, 2013, Luiz Carlos Jr wrote: > Hi ! > > I have a script written in python that should update a counter in > Cassandra. It acts like a batch that corrects the counter. > > What script does is: get the real value that should be at database, get the > current value that is persisted at database, calculates the difference > between values and finally send insert command to specific row key passing > the diff to update the counter. > > All works fine. I got the correct values and diffs but, after insert > command, the value of row key is not the expected. > > Example: > Real Value = 35000 > Current DB Value = 3 > Diff Value = 5000 > > I sent the insert command passing diff=5000 so, the expected new DB value > should be 35000. But sometimes the new DB value is a crazy negative number, > like -360.000. It doesn't make any sense, because there's no combination of > values and operators that could result on a value like that. > > And, if i run the script again, the final persisted value is correct. If I > try again later the final persisted value appears wrong again... > > Can anyone help me with this ? The same script sometimes works fine and > sometimes persists a crazy value. > > Thank you ! > -- Chris
Re: Unexpected value after insert
Hi Luiz, On 5 Nov 2013, at 12:37 am, Luiz Carlos Jr wrote: > What script does is: get the real value that should be at database, get the > current value that is persisted at database, calculates the difference > between values and finally send insert command to specific row key passing > the diff to update the counter. I would like to make sure I understand what your doing, basically you have added some logic that attempts to “fix” the counter, by reading it and writing back the correct value? What happens if you take out the code that attempts to “fix” the counter, and run the script again? i.e. Does the counter then eventually increment by the number you would expect it to increment. By eventually I mean a short period of time after the script has completed. Best regards, Jacob
Re: Unexpected value after insert
Hi Christopher, Actually it's not a counter, but just a field that stores integers. I'm logging the expected values but not the command itself because I'm using pycassa to send it. I'm absolutely sure that it's not an overflow case. I'm talking about numbers in order of 10~100k with no math operators that could raise an expection (only one subtraction operation). Any other ideas ? Thanks for the tips ! On Tue, Nov 5, 2013 at 12:29 PM, Christopher Smith wrote: > Given your use case I'd recommend not using a counter for the field. > > Are you logging the update command you are sending Cassandra? Are you sure > you aren't overflowing the value either in Cassandra or in your batch code? > > On Monday, November 4, 2013, Luiz Carlos Jr wrote: > > > Hi ! > > > > I have a script written in python that should update a counter in > > Cassandra. It acts like a batch that corrects the counter. > > > > What script does is: get the real value that should be at database, get > the > > current value that is persisted at database, calculates the difference > > between values and finally send insert command to specific row key > passing > > the diff to update the counter. > > > > All works fine. I got the correct values and diffs but, after insert > > command, the value of row key is not the expected. > > > > Example: > > Real Value = 35000 > > Current DB Value = 3 > > Diff Value = 5000 > > > > I sent the insert command passing diff=5000 so, the expected new DB value > > should be 35000. But sometimes the new DB value is a crazy negative > number, > > like -360.000. It doesn't make any sense, because there's no combination > of > > values and operators that could result on a value like that. > > > > And, if i run the script again, the final persisted value is correct. If > I > > try again later the final persisted value appears wrong again... > > > > Can anyone help me with this ? The same script sometimes works fine and > > sometimes persists a crazy value. > > > > Thank you ! > > > > > -- > Chris >
Re: Unexpected value after insert
If it isn't a counter type, then why calculate the diff value, just do the insert. On Tue, Nov 5, 2013 at 2:28 PM, Luiz Carlos Jr wrote: > Hi Christopher, > > Actually it's not a counter, but just a field that stores integers. I'm > logging the expected values but not the command itself because I'm using > pycassa to send it. I'm absolutely sure that it's not an overflow case. I'm > talking about numbers in order of 10~100k with no math operators that could > raise an expection (only one subtraction operation). > > Any other ideas ? Thanks for the tips ! > > > On Tue, Nov 5, 2013 at 12:29 PM, Christopher Smith >wrote: > > > Given your use case I'd recommend not using a counter for the field. > > > > Are you logging the update command you are sending Cassandra? Are you > sure > > you aren't overflowing the value either in Cassandra or in your batch > code? > > > > On Monday, November 4, 2013, Luiz Carlos Jr wrote: > > > > > Hi ! > > > > > > I have a script written in python that should update a counter in > > > Cassandra. It acts like a batch that corrects the counter. > > > > > > What script does is: get the real value that should be at database, get > > the > > > current value that is persisted at database, calculates the difference > > > between values and finally send insert command to specific row key > > passing > > > the diff to update the counter. > > > > > > All works fine. I got the correct values and diffs but, after insert > > > command, the value of row key is not the expected. > > > > > > Example: > > > Real Value = 35000 > > > Current DB Value = 3 > > > Diff Value = 5000 > > > > > > I sent the insert command passing diff=5000 so, the expected new DB > value > > > should be 35000. But sometimes the new DB value is a crazy negative > > number, > > > like -360.000. It doesn't make any sense, because there's no > combination > > of > > > values and operators that could result on a value like that. > > > > > > And, if i run the script again, the final persisted value is correct. > If > > I > > > try again later the final persisted value appears wrong again... > > > > > > Can anyone help me with this ? The same script sometimes works fine and > > > sometimes persists a crazy value. > > > > > > Thank you ! > > > > > > > > > -- > > Chris > > > -- Chris
Re: Unexpected value after insert
Hi Jacob ! How are you ? You understood perfectly the problem. I need to fix an integer field by reading it, calculating the difference and writing back the correct value (in other words, increment it with the diff, positive or negative to achieve the correct value). As I said it works sometimes and fails in other attempts. i.e. If i run now it fails... running again then works. I'm "fixing" few records (~30 records) and it fails only in one or two (but not always the same) and works perfectly to the others. And... if I run again all records got the correct value. And if I run again, some fails. In the future this script must work with millions of records but I confess I'm a little insecure right now because it's a very simple operation and it seems like a Cassandra's bug. Could it be ? If you need somre more information let me know. Thank you for the help ! Luiz On Tue, Nov 5, 2013 at 7:08 PM, Jacob Rhoden wrote: > Hi Luiz, > > On 5 Nov 2013, at 12:37 am, Luiz Carlos Jr wrote: > > What script does is: get the real value that should be at database, get > the > > current value that is persisted at database, calculates the difference > > between values and finally send insert command to specific row key > passing > > the diff to update the counter. > > > I would like to make sure I understand what your doing, basically you have > added some logic that attempts to “fix” the counter, by reading it and > writing back the correct value? > > What happens if you take out the code that attempts to “fix” the counter, > and run the script again? i.e. Does the counter then eventually increment > by the number you would expect it to increment. By eventually I mean a > short period of time after the script has completed. > > Best regards, > Jacob > >
Re: Unexpected value after insert
Hi Christopher, Sorry about the wrog information... I checked the column family and it's really defined as a counter. On Tue, Nov 5, 2013 at 8:39 PM, Christopher Smith wrote: > If it isn't a counter type, then why calculate the diff value, just do the > insert. > > > On Tue, Nov 5, 2013 at 2:28 PM, Luiz Carlos Jr > wrote: > > > Hi Christopher, > > > > Actually it's not a counter, but just a field that stores integers. I'm > > logging the expected values but not the command itself because I'm using > > pycassa to send it. I'm absolutely sure that it's not an overflow case. > I'm > > talking about numbers in order of 10~100k with no math operators that > could > > raise an expection (only one subtraction operation). > > > > Any other ideas ? Thanks for the tips ! > > > > > > On Tue, Nov 5, 2013 at 12:29 PM, Christopher Smith > >wrote: > > > > > Given your use case I'd recommend not using a counter for the field. > > > > > > Are you logging the update command you are sending Cassandra? Are you > > sure > > > you aren't overflowing the value either in Cassandra or in your batch > > code? > > > > > > On Monday, November 4, 2013, Luiz Carlos Jr wrote: > > > > > > > Hi ! > > > > > > > > I have a script written in python that should update a counter in > > > > Cassandra. It acts like a batch that corrects the counter. > > > > > > > > What script does is: get the real value that should be at database, > get > > > the > > > > current value that is persisted at database, calculates the > difference > > > > between values and finally send insert command to specific row key > > > passing > > > > the diff to update the counter. > > > > > > > > All works fine. I got the correct values and diffs but, after insert > > > > command, the value of row key is not the expected. > > > > > > > > Example: > > > > Real Value = 35000 > > > > Current DB Value = 3 > > > > Diff Value = 5000 > > > > > > > > I sent the insert command passing diff=5000 so, the expected new DB > > value > > > > should be 35000. But sometimes the new DB value is a crazy negative > > > number, > > > > like -360.000. It doesn't make any sense, because there's no > > combination > > > of > > > > values and operators that could result on a value like that. > > > > > > > > And, if i run the script again, the final persisted value is correct. > > If > > > I > > > > try again later the final persisted value appears wrong again... > > > > > > > > Can anyone help me with this ? The same script sometimes works fine > and > > > > sometimes persists a crazy value. > > > > > > > > Thank you ! > > > > > > > > > > > > > -- > > > Chris > > > > > > > > > -- > Chris >
Re: Unexpected value after insert
Okay, if it is a counter you are going to have a hell of a time getting it a value. Best advice I can give is to up consistency levels on operations on the fields. On Tuesday, November 5, 2013, Luiz Carlos Jr wrote: > Hi Christopher, > > Sorry about the wrog information... I checked the column family and it's > really defined as a counter. > > > On Tue, Nov 5, 2013 at 8:39 PM, Christopher Smith > > > wrote: > > > If it isn't a counter type, then why calculate the diff value, just do > the > > insert. > > > > > > On Tue, Nov 5, 2013 at 2:28 PM, Luiz Carlos Jr > > > > > > wrote: > > > > > Hi Christopher, > > > > > > Actually it's not a counter, but just a field that stores integers. I'm > > > logging the expected values but not the command itself because I'm > using > > > pycassa to send it. I'm absolutely sure that it's not an overflow case. > > I'm > > > talking about numbers in order of 10~100k with no math operators that > > could > > > raise an expection (only one subtraction operation). > > > > > > Any other ideas ? Thanks for the tips ! > > > > > > > > > On Tue, Nov 5, 2013 at 12:29 PM, Christopher Smith > > > > > > >wrote: > > > > > > > Given your use case I'd recommend not using a counter for the field. > > > > > > > > Are you logging the update command you are sending Cassandra? Are you > > > sure > > > > you aren't overflowing the value either in Cassandra or in your batch > > > code? > > > > > > > > On Monday, November 4, 2013, Luiz Carlos Jr wrote: > > > > > > > > > Hi ! > > > > > > > > > > I have a script written in python that should update a counter in > > > > > Cassandra. It acts like a batch that corrects the counter. > > > > > > > > > > What script does is: get the real value that should be at database, > > get > > > > the > > > > > current value that is persisted at database, calculates the > > difference > > > > > between values and finally send insert command to specific row key > > > > passing > > > > > the diff to update the counter. > > > > > > > > > > All works fine. I got the correct values and diffs but, after > insert > > > > > command, the value of row key is not the expected. > > > > > > > > > > Example: > > > > > Real Value = 35000 > > > > > Current DB Value = 3 > > > > > Diff Value = 5000 > > > > > > > > > > I sent the insert command passing diff=5000 so, the expected new DB > > > value > > > > > should be 35000. But sometimes the new DB value is a crazy negative > > > > number, > > > > > like -360.000. It doesn't make any sense, because there's no > > > combination > > > > of > > > > > values and operators that could result on a value like that. > > > > > > > > > > And, if i run the script again, the final persisted value is > correct. > > > If > > > > I > > > > > try again later the final persisted value appears wrong again... > > > > > > > > > > Can anyone help me with this ? The same script sometimes works fine > > and > > > > > sometimes persists a crazy value. > > > > > > > > > > Thank you ! > > > > > > > > > > > > > > > > > -- > > > > Chris > > > > > > > > > > > > > > > -- > > Chris > > > -- Chris
