Re: Apache Cassandra meetup @ Instagram HQ
Great ! Thanks Dinesh Le mer. 20 févr. 2019 à 19:57, dinesh.jo...@yahoo.com.INVALID a écrit : > Just heard back. Unfortunately they cannot live stream but they'll try to > record it and post it online. > Dinesh > > On Tuesday, February 19, 2019, 3:55:15 PM PST, Sundaramoorthy, > Natarajan wrote: > > Dinesh Thank you. > > > > -Original Message- > From: andre wilkinson [mailto:antonio...@me.com.INVALID] > Sent: Tuesday, February 19, 2019 5:14 PM > To: dev@cassandra.apache.org > Subject: Re: Apache Cassandra meetup @ Instagram HQ > > Thank you for reaching out > > Sent from my iPhone > > > On Feb 19, 2019, at 8:45 AM, Dinesh Joshi > wrote: > > > > I’ve emailed the organizers. They didn’t intend to live stream it but > will get back to us if they can. > > > > Cheers, > > > > Dinesh > > > >> On Feb 19, 2019, at 1:11 AM, andre wilkinson > wrote: > >> > >> Yes is there anyway to attend remotely? > >> > >> Sent from my iPhone > >> > >>> On Feb 18, 2019, at 6:40 PM, Shaurya Gupta > wrote: > >>> > >>> Hi, > >>> > >>> This looks very interesting to me. Can I attend this remotely? > >>> > >>> Thanks > >>> Shaurya > >>> > >>> > >>> On Tue, Feb 19, 2019 at 5:37 AM dinesh.jo...@yahoo.com.INVALID > >>> wrote: > >>> > >>>> Hi all, > >>>> > >>>> Apologies for the cross-post. In case you're in the SF Bay Area, > Instagram > >>>> is hosting a meetup. Interesting talks on Cassandra Traffic > management, > >>>> Cassandra on Kubernetes. See details in the attached link - > >>>> > >>>> > >>>> > https://www.eventbrite.com/e/cassandra-traffic-management-at-instagram-cassandra-and-k8s-with-instaclustr-tickets-54986803008 > >>>> > >>>> Thanks, > >>>> > >>>> Dinesh > >>>> > >>> > >>> > >>> -- > >>> Shaurya Gupta > >> > >> > >> - > >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > >> For additional commands, e-mail: dev-h...@cassandra.apache.org > >> > > > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > > This e-mail, including attachments, may include confidential and/or > proprietary information, and may be used only by the person or entity > to which it is addressed. If the reader of this e-mail is not the intended > recipient or his or her authorized agent, the reader is hereby notified > that any dissemination, distribution or copying of this e-mail is > prohibited. If you have received this e-mail in error, please notify the > sender by replying to this message and delete this e-mail immediately. > B�CB��[��X��ܚX�KK[XZ[�]�][��X��ܚX�P�\��[��K�\X�K�ܙ�B��܈Y][ۘ[��[X[��K[XZ[�]�Z[�\��[��K�\X�K�ܙ�B�B > -- Cordialement; Ahmed ELJAMI
Re: Need information related to Cassandra 4.x Release date
Hi, Not yet fixed, probably Q4 2019. Please find more informations on this thread: https://lists.apache.org/thread.html/246a5d79240b7701455360d650de7acb11c66e53d007babe206fe0a7@%3Cdev.cassandra.apache.org%3E
Re: Contributing cassandra-diff
Great addition! Thanks Marcus. +1 for cassandra-compare as said by Jeremy. We can also think about other features like: - Comparing just the count between 2 tables. In some cases, It will be enough to say that our copy is OK. - Making a difference on a set of partition ==> This will avoid comparing the full of data in case of large volumes and when a set of data will be enough to be sure of our copy. Thanks Le jeu. 22 août 2019 à 09:49, Jeremy Hanna a écrit : > It’s great to contribute such a tool. The change between 2.x and 3.0 > brought a translation layer from thrift to cql that is hard to validate on > real clusters without something like this. Thank you. > > As for naming, perhaps cassandra-compare might be clearer as diff is an > overloaded word but that’s a bikeshed sort of argument. > > > On Aug 22, 2019, at 12:32 AM, Vinay Chella > wrote: > > > > This is a great addition to our Cassandra validation framework/tools. I > can > > see many teams in the community get benefited from tooling like this. > > > > I like the idea of the generic repo (repos/asf/cassandra-contrib.git > > or *whatever > > the name is*) for tools like this, for the following 2 main reasons. > > > > 1. Easily accessible/ reachable/ searchable > > 2. Welcomes community in Cassandra ecosystem to contribute more easily > > > > > > > > Thanks, > > Vinay Chella > > > > > >> On Wed, Aug 21, 2019 at 11:39 PM Marcus Eriksson > wrote: > >> > >> Hi, we are about to open source our tooling for comparing two cassandra > >> clusters and want to get some feedback where to push it. I think the > >> options are: (name bike-shedding welcome) > >> > >> 1. create repos/asf/cassandra-diff.git > >> 2. create a generic repos/asf/cassandra-contrib.git where we can add > more > >> contributed tools in the future > >> > >> Temporary location: https://github.com/krummas/cassandra-diff > >> > >> Cassandra-diff is a spark job that compares the data in two clusters - > it > >> pages through all partitions and reads all rows for those partitions in > >> both clusters to make sure they are identical. Based on the > configuration > >> variable “reverse_read_probability” the rows are either read forward or > in > >> reverse order. > >> > >> Our main use case for cassandra-diff has been to set up two identical > >> clusters, transfer a snapshot from the cluster we want to test to these > >> clusters and upgrade one side. When that is done we run this tool to > make > >> sure that 2.1 and 3.0 gives the same results. A few examples of the > bugs we > >> have found using this tool: > >> > >> * CASSANDRA-14823: Legacy sstables with range tombstones spanning > multiple > >> index blocks create invalid bound sequences on 3.0+ > >> * CASSANDRA-14803: Rows that cross index block boundaries can cause > >> incomplete reverse reads in some cases > >> * CASSANDRA-15178: Skipping illegal legacy cells can break reverse > >> iteration of indexed partitions > >> > >> /Marcus > >> > >> - > >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > >> For additional commands, e-mail: dev-h...@cassandra.apache.org > >> > >> > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > > -- Cordialement; Ahmed ELJAMI
cassandra--dtest: module 'pytest' has no attribute 'config'
Hi folks,
I'm trying to run dtest on local and getting the following error (I
configure it to run the only refresh_test.py test):
$ pytest --cassandra-dir=/home/aeljami/workspace/git/cstar/cassandra
== test session
> starts ==
> platform linux -- Python 3.6.9, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
> rootdir: /home/aeljami/workspace/git/cassandra-dtest, configfile:
> pytest.ini
> plugins: flaky-3.7.0
> collected 1 item
>
> refresh_test.py E
> [100%]
>
> ERRORS
> =
> __ ERROR at setup of
> TestRefresh.test_refresh_deadlock_startup
> __
>
> request = test_refresh_deadlock_startup>>
>
> @pytest.fixture(scope="function", autouse=True)
> def fixture_logging_setup(request):
> # set the root logger level to whatever the user asked for
> # all new loggers created will use the root logger as a template
> # essentially making this the "default" active log level
> log_level = logging.INFO
> try:
> # first see if logging level overridden by user as command
> line argument
> > log_level_from_option = pytest.config.getoption("--log-level")
> E AttributeError: module 'pytest' has no attribute 'config'
>
> conftest.py:135: AttributeError
>
short test summary
> info
> ERROR refresh_test.py::TestRefresh::test_refresh_deadlock_startup -
> AttributeError: module 'pytest' has no attribute 'config'
>
$ cat pytest.ini
[pytest]
> cassandra_dir= /home/aeljami/workspace/git/cstar/cassandra
> python_files = refresh_test.py
> junit_suite_name = Cassandra dtests
> log_print = True
> log_level = INFO
> log_format = %(asctime)s,%(msecs)d %(name)s %(levelname)s %(message)s
> timeout = 900
>
Any idea please ?
Thanks
Re: cassandra--dtest: module 'pytest' has no attribute 'config'
Hi Adam,
Thanks for your reply.
It's indeed a pytest version issue. Resolved by using virtualenv as
mentioned in the doc :)
cheers
Le lun. 7 déc. 2020 à 21:44, Adam Holmberg a
écrit :
> Hello. It looks like your environment might be using a newer version of
> pytest than is supported by these tests.
> https://docs.pytest.org/en/latest/deprecations.html#pytest-config-global
>
> Note that the requirements file specifies a fixed, older version of pytest:
>
> https://github.com/apache/cassandra-dtest/blob/192b70607a0c4a96f524bd5dc0c19c3a28f938f0/requirements.txt#L9
>
> On Fri, Dec 4, 2020 at 5:32 AM Ahmed Eljami
> wrote:
>
> > Hi folks,
> >
> > I'm trying to run dtest on local and getting the following error (I
> > configure it to run the only refresh_test.py test):
> >
> > $ pytest --cassandra-dir=/home/aeljami/workspace/git/cstar/cassandra
> >
> > == test session
> > > starts ==
> > > platform linux -- Python 3.6.9, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
> > > rootdir: /home/aeljami/workspace/git/cassandra-dtest, configfile:
> > > pytest.ini
> > > plugins: flaky-3.7.0
> > > collected 1 item
> > >
> > > refresh_test.py E
> > > [100%]
> > >
> > > ERRORS
> > > =
> > > __ ERROR at setup of
> > > TestRefresh.test_refresh_deadlock_startup
> > > __
> > >
> > > request = > > test_refresh_deadlock_startup>>
> > >
> > > @pytest.fixture(scope="function", autouse=True)
> > > def fixture_logging_setup(request):
> > > # set the root logger level to whatever the user asked for
> > > # all new loggers created will use the root logger as a
> template
> > > # essentially making this the "default" active log level
> > > log_level = logging.INFO
> > > try:
> > > # first see if logging level overridden by user as command
> > > line argument
> > > > log_level_from_option =
> > pytest.config.getoption("--log-level")
> > > E AttributeError: module 'pytest' has no attribute 'config'
> > >
> > > conftest.py:135: AttributeError
> > >
> >
> > short test
> summary
> > > info
> > > ERROR refresh_test.py::TestRefresh::test_refresh_deadlock_startup -
> > > AttributeError: module 'pytest' has no attribute 'config'
> > >
> >
> > $ cat pytest.ini
> >
> > [pytest]
> > > cassandra_dir= /home/aeljami/workspace/git/cstar/cassandra
> > > python_files = refresh_test.py
> > > junit_suite_name = Cassandra dtests
> > > log_print = True
> > > log_level = INFO
> > > log_format = %(asctime)s,%(msecs)d %(name)s %(levelname)s %(message)s
> > > timeout = 900
> > >
> >
> > Any idea please ?
> >
> > Thanks
> >
>
>
> --
> Adam Holmberg
> e. adam.holmb...@datastax.com
> w. www.datastax.com
>
--
Cordialement;
Ahmed ELJAMI
Modeling nested collection with C* 2.0
Hi,
I need your help for modeling a nested collection with cassanrda2.0 (UDT no,
no fozen)
My users table contains emails by type, each type of email contains multiple
emails.
Example:
Type: pro. emails: {a...@mail.com, b...@mail.com ...}
Type: private. emails: {c...@mail.com, d...@mail.com}
.
The user table also contains addresses, address type with fields.
Example:
Type: Pro. address {Street= aaa, number = 123, apartment = bbb}
Type: Private. address {Street = bbb, number = 123, apartment = kkk }
I am looking for a solution to store all these columns in one table.
Thank you.
Re: Cassandra doesn't flush any commit log files into cdc_raw directory
Hi Bingqin, When cdc_raw directory is full, Cassandra rejects new writes on this node with the following message in the log: - Rejecting Mutation containing CDC-enabled table. https://github.com/apache/cassandra/blob/cassandra-3.11.9/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerCDC.java#L125 So, I'm not sorry if WriteTimeoutException is related to cdc feature in your case. For your issue with no commitLog flushed/copied into cdc_raw directory, can you try with an explicit nodetool flush and see if commitLog will be transferred ? For tables with a "very low" write rates, memtable can take a lot of time before be flushed on disk. Cheers, Ahmed Le jeu. 29 avr. 2021 à 00:23, Bingqin Zhou a écrit : > Hi all, > > We're working on a Kafka connector to capture data changes in Cassandra by > processing commit log files in the cdc_raw directory. After we enabled CDC > on a few tables, we didn't observe any commit log files getting flushed > into cdc_raw directory as expected, but got WriteTimeoutException in > Cassandra DB. > > Here's how we reproduce the issue: > > 1. Our Cassandra Settings: > > - Cassandra Version: 3.11.9 > - Related configs in Cassandra.yaml: >- cdc_enabled: true >- cdc_total_space_in_mb: 4096 >- commitlog_segment_size_in_mb: 32mb >- commitlog_total_space_in_mb: 8192 >- commitlog_sync: periodic >- commitlog_sync_period_in_ms: 1 > > 2. Enable CDC on a few tables by CQL: > ALTER TABLE foo WITH cdc=true; > > 3. After a few days, we get *WriteTimeoutException* in Cassandra DB. > However at the same time, cdc_raw directory is still empty with no commit > log flushed/copied into it at all. > > I want to understand why there's no commit log file flushed into > cdc_raw directory at all even when the threshold cdc_total_space_in_mb has > been reached and write suspension has been triggered in Cassandra DB. This > sounds like a bug and currently makes the CDC feature useless. > > Thanks so much, > Bingqin Zhou > -- Cordialement; Ahmed ELJAMI
