Re: A blog about Cassandra in the IoT arena

2018-08-29 Thread Rahul Singh
Understood. Deep problems to consider. Partition size. I’ve been looking at how Yugabyte is using “tablets” of data which have data. It’s an interesting proposition. .. it all comes down to the token based addressing - which is optimized as a single dimension array and I think this is part of t

Re: A blog about Cassandra in the IoT arena

2018-08-24 Thread DuyHai Doan
No what I meant by infinite partition is not auto sub-partitioning, even at server-side. Ideally Cassandra should be able to support infinite partition size and make compaction, repair and streaming of such partitions manageable: - compaction: find a way to iterate super efficiently through the wh

Re: A blog about Cassandra in the IoT arena

2018-08-23 Thread Rahul Singh
Agreed. One of the ideas I had on partition size is to automatically synthetically shard based on some basic patterns seen in the data. It could be implemented as a tool that would create a new table with an additional part of the key that is an automatic created shard, or it would use an exist

Re: A blog about Cassandra in the IoT arena

2018-08-23 Thread DuyHai Doan
As I used to tell some people, the day we make : 1. partition size unlimited, or at least huge partition easily manageable (compaction, repair, streaming, partition index file) 2. tombstone a non-issue that day, Cassandra will dominate any other IoT technology out there Until then ... On Thu, A

Re: A blog about Cassandra in the IoT arena

2018-08-23 Thread Rahul Singh
Good analysis of how the different key structures affect use cases and performance. I think you could extend this article with potential evaluation of FiloDB which specifically tries to solve the OLAP issue with arbitrary queries. Another option is leveraging Elassandra (index in Elasticsearch c

A blog about Cassandra in the IoT arena

2018-08-23 Thread Affan Syed
Hi, we wrote a blog about some of the results that engineers from AN10 shared earlier. I am sharing it here for greater comments and discussions. http://www.an10.io/technology/cassandra-and-iot-queries-are-they-a-good-match/ Thank you. - Affan