Re: Questions and problems about the state of Python 3 support on 4.0

[Eduard Tudenhoefner]

Hi all,

I took a look at CASSANDRA-15659
 / CASSANDRA-15573
 and addressed both.

For CASSANDRA-15659 
the requirement of *cqlsh* only working with Python 2.7 & 3.6 was relaxed
by allowing Python 3.6+.
Python 3.8 testing was added to the CircleCI config so that we can actually
see what else breaks with newer Python versions.

A new Docker images with Ubuntu 19.10 was required for testing (
https://github.com/apache/cassandra-builds/pull/17). This docker image sets
up Python 2.7/3.6/3.7/3.8 with their respective virtual environments, which
are then being used by the CircleCI yaml.

The image *spod/cassandra-testing-ubuntu1810-java11-w-dependencies:20190306*
couldn't be updated unfortunately because it can't be built anymore, due to
Ubuntu 18.10 being EOL.

For CASSANDRA-15573 
a Python 3.8 compatible SaferScanner implementation was introduced.

Both tickets are ready to be reviewed, and I would appreciate any feedback.

– Eduard

On Tue, Mar 24, 2020 at 8:16 PM Stefan Miklosovic <
stefan.mikloso...@instaclustr.com> wrote:

> Ok, I created two JIRAs. The first one is targeting 3.6/3.7 issue, the
> second one is about the bug I hit. I made two issues so they can be
> worked on separately.
>
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_CASSANDRA-2D15659&d=DwIBaQ&c=adz96Xi0w1RHqtPMowiL2g&r=Mvswwxt6BEQ7eAT5OkvJjbzf_MmqFVTWWvPSfg8SU4k&m=8G4dKwG6Dcb0xNDtgLJSiM8eBZ3NvZEacaowcwlR0ls&s=uaomCLkBsDorb3EHBGVtW5FlB6lJCKGYPfA7b0Tep1s&e=
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_CASSANDRA-2D15660&d=DwIBaQ&c=adz96Xi0w1RHqtPMowiL2g&r=Mvswwxt6BEQ7eAT5OkvJjbzf_MmqFVTWWvPSfg8SU4k&m=8G4dKwG6Dcb0xNDtgLJSiM8eBZ3NvZEacaowcwlR0ls&s=XsWkaLxudaHLlha1D9k-BjXseLmwU7FsC2cXQ0L0OHc&e=
>
> Cheers
>
> On Tue, 24 Mar 2020 at 19:45, Jon Haddad  wrote:
> >
> > I don't think supporting only 3.6 not 3.7 was a deliberate move, it's
> > likely just an oversight.  Yes, we should address that.  Mind filing a
> > JIRA?
> >
> > On Tue, Mar 24, 2020 at 11:33 AM Stefan Miklosovic <
> > stefan.mikloso...@instaclustr.com> wrote:
> >
> > > Hi,
> > >
> > > I built deb package for Debian from current trunk as of today and I
> > > wanted to test cqlsh and I got this error:
> > >
> > > me@machine:/$ python --version
> > > Python 2.7.16
> > > me@machine:/$ cqlsh -u cassandra -p cassandra --execute="select * from
> > > system_auth.roles" machine
> > > Usage: cqlsh.py [options] [host [port]]
> > >
> > > cqlsh.py: error: 'boot' is not a valid port number.
> > >
> > > This is Debian Buster.
> > >
> > > Please keep in mind that _without_ "--execute" / "-e", it just
> > > connects fine. So I think this is a bug. I am pretty sure this was
> > > working before.
> > >
> > > I wanted to try this against Python 3 and as I was digging deeper I
> > > saw that there was this set of changes merged recently (1) with this
> > > function specifically (2) Hence, I can see that there is support for
> > > Python 3.6 and 2.7.
> > >
> > > My question is why are we so strict when it comes to the version of
> Python
> > > 3?
> > >
> > > Secondly, if this project is building debs for Debian (I suppose so),
> > > what version of Debian it is actually built for then as there is not
> > > 3.6 version of Python out of the box for any release.
> > >
> > > For example, quickly looking here (3), there is Python 3.7 for Buster
> > > and 2.7 for Stretch, Jessie and even Wheezy. But no 3.6! You can check
> > > official Python version here (4)
> > >
> > > If I understand this correctly, this means that if one installs a deb
> > > and he wants to run it with Python 3.6 (the only Python version this
> > > cqlsh supports), he can not install it from repositories but he has to
> > > chase that Python version himself and build it from source and so on
> > > and so on ...
> > >
> > > Could we either make this compatible with 3.7 and switch to that
> > > version or delete so specific requirement (3.6) from the code? I
> > > humbly think that if we distribute debs we should also say what
> > > release we are targetting and it should be possible to install it all
> > > by standard means from the repos.
> > >
> > > Regards
> > >
> > > (1)
> > >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_cassandra_commit_bf9a1d487b9ba469e8d740cf7d1cd419535a7e79&d=DwIBaQ&c=adz96Xi0w1RHqtPMowiL2g&r=Mvswwxt6BEQ7eAT5OkvJjbzf_MmqFVTWWvPSfg8SU4k&m=8G4dKwG6Dcb0xNDtgLJSiM8eBZ3NvZEacaowcwlR0ls&s=85RWaTe_R3oB7dEEBu2rBFFE_rA3zKoHiNptrJRAtoE&e=
> > > (2)
> > >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_cassandra_blob_bf9a1d487b9ba469e8d740cf7d1cd419535a7e79_bin_cqlsh-23L57-2DL65&d=DwIBaQ&c=adz96Xi0w1RHqtPMowiL2g&r=Mvswwxt6BEQ7eAT5OkvJjbzf_MmqFVTWWvPSfg8SU4k&m=8G4dKwG6Dcb0xNDtg

Re: Cassandra CI Status – 2020-04-06

[Joshua McKenzie]

I spoke with Mick offline about this a bit but wanted to relay it here for
posterity:

** Why are cdc and compression unit tests run separately?
>
In the case of cdc, I erred on the side of caution in impl assuming that
most people would not be using it so any degradation in performance for the
non-cdc case would be net negative for the majority of users. As such, the
CDC mutation allocation and segment management codepaths are separate from
the regular. Were we to only run tests with cdc enabled, we'd never
exercise the code-paths for what would presumably be the majority use-case
of the CommitLog. Which I say would be bad. :)

This amounts to checking a bool in Mutation a couple times and
synchronizing on an object during new segment creation and deletion, so it
may be worth considering unifying the commit log allocator paths on the CDC
path and remove the need for separate testing in 5.0. There are quite a few
different users running CDC out in the world today without significant
issue, and some robust system level benchmarking and micro-benchmarking of
CommitLog behavior at boundary conditions should establish further
confidence in the acceptable nature of a change like that.

On Mon, Apr 6, 2020 at 3:20 AM Mick Semb Wever  wrote:

> It's been three weeks since the last update¹. What follows is a quick
> state of affairs and a few questions for anyone to jump in and help
> with.
>
> The ASF CI is running at
> https://ci-cassandra.apache.org/view/branches/ and will soon have 36
> builds agents in total.  Thanks to Instaclustr, Amazon, iland, and
> DataStax. The previous `builds.apache.org` is now retired for all
> Cassandra builds.
>
>
> ** Broken Builds
>
> Tests results continue to improve. Last week at one point we saw trunk
> down to 24 test failures for the whole pipeline.  That's including
> three runs of unit tests (normal, cdc, compression) and three runs of
> dtests (normal, novnode, offheap).   Because of that it was easy to
> see the breakage CASSANDRA-15684. Breakages happen, but to be able to
> spot them so quickly is a win!
>
>
> ** Emails on Broken Builds
>
> Currently you will only get a broken build email if you break the
> artifacts stage of the pipeline.
>
> Pipeline stages that are now consistently passing and are candidates
> for adding broken build email notifications to are:
>   - Cassandra-2.2-jvm-dtest
>   - Cassandra-2.2-long-test
>   - Cassandra-3.0-jvm-dtest
>   - Cassandra-3.0-long-test
>   - Cassandra-3.0-test
>   - Cassandra-3.11-jvm-dtest
>   - Cassandra-3.11-long-test
>   - Cassandra-3.11-test
>   - Cassandra-3.11-stress-test
>   - Cassandra-3.11-test-compression
>   - Cassandra-trunk-fqltool-test
>   - Cassandra-trunk-stress-test
>   - Cassandra-trunk-test
>   - Cassandra-trunk-test-compression
>   - Cassandra-trunk-test-burn
>
> Given no objection, I will add broken email notification to these,
> starting with the smaller stages that have been longest stable.
> Please make sure that you are not filtering out emails from
> jenk...@builds.apache.org so you get to know you broke it before
> someone has to tell you.
>
>
> ** Our ci-cassandra Jenkins versus CircleCI
>
> Thanks to David Capwell, who spent some time comparing ci-cassandra to
> CircleCI.
>
> His findings were…
>   - There's a reasonable match across the unit, stress, fql, and
> jvm-dtest tests.
>   - CircleCI does not run the cdc unit, burn, long unit, cqlsh tests,
> or offheap dtests.
>   - ci-cassandra does not have JDK 11 test runs, or the upgrade
> (jvm-dtest and dtest) tests.
>   - The cqlsh tests: which only ci-cassandra runs; are broken in trunk
> with the recent python3 upgrade.
>
>
> ** Why are cdc and compression unit tests run separately?
>
> Why do we duplicate unit test runs for cdc and compression?
> Is there any ML or ticket to provide history on this?
> Can we not instead just enable these settings in the normal unit run?
>
>
> regards,
> Mick
>
>
> ¹)
> https://lists.apache.org/thread.html/re8122e4fdd8629e7fbca2abf27d72054b3bc0e3690ece8b8e66f618b%40%3Cdev.cassandra.apache.org%3E
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

20200407 4.0 Status Update

[Jordan West]

Hi Everyone,

The board can be found here:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355

[Tickets That Need Attention]
A reminder that Josh has added a new 'Needs Attention' filter to show any
tasks that are stalled, need an assignee or need a reviewer. Makes it easy
if you would like to find something to work on that helps push us closer to
4.0.0's release.

Needs Attention:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&quickFilter=1719

[Alpha Status]
We continue to have no tickets that need assignees in alpha. Of the
remaining tickets in alpha, 8 have a reviewer / reviews are in progress, 3
are in need of reviewers, 3 are in progress, and 3 are not started or
require more information (or are being worked on but haven't had ticket
metadata updated). The tickets in need of a reviewer or are not yet started
are a great place to help out if you are looking for something to
contribute.

Needs Reviewer:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&quickFilter=1659

[Stalled Tickets]
63 tickets are stalled (have not been updated in >14d). 6 of these are
tagged for alpha. Of the remaining, 14 are in beta and 43 are in rc.

Stalled:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&quickFilter=1694

[Open vs. Closed Last 7 Days & Cumulative Flow]
We opened 14 issues in the last 7 days and closed 26 (8 of which were new).
Most of the new tickets were test failures that were fixed in alpha. While
the number of issues in the release continues to grow (many of which are
flaky test failures), we continue to make more progress completing issues
(as shown by this week's net of 12 closed tickets.

Notable New Tickets:
https://issues.apache.org/jira/browse/CASSANDRA-15690 is a critical read
path bug that can lead to a transient incorrect response. This affects 4.0
as well as previous versions of Cassandra including 3.11 and 3.0.x.

Opened:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&quickFilter=1670
Closed:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&quickFilter=1671

Cumulative Flow:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&view=reporting&chart=cumulativeFlowDiagram&swimlane=939&swimlane=936&swimlane=931&column=1505&column=1506&column=1514&column=1509&column=1512&column=1507&days=30

Thanks for all your efforts! Also, if there is anything else you would like
to see in these weekly updates please let us know.

Jordan

Re: 20200407 4.0 Status Update

[Manish G]

Hi,

Can there be a filter like 'good first issue' which new people can use to
find issues to start with?

Manish

On Tue, Apr 7, 2020 at 10:44 PM Jordan West  wrote:

> Hi Everyone,
>
> The board can be found here:
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355
>
> [Tickets That Need Attention]
> A reminder that Josh has added a new 'Needs Attention' filter to show any
> tasks that are stalled, need an assignee or need a reviewer. Makes it easy
> if you would like to find something to work on that helps push us closer to
> 4.0.0's release.
>
> Needs Attention:
>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&quickFilter=1719
>
> [Alpha Status]
> We continue to have no tickets that need assignees in alpha. Of the
> remaining tickets in alpha, 8 have a reviewer / reviews are in progress, 3
> are in need of reviewers, 3 are in progress, and 3 are not started or
> require more information (or are being worked on but haven't had ticket
> metadata updated). The tickets in need of a reviewer or are not yet started
> are a great place to help out if you are looking for something to
> contribute.
>
> Needs Reviewer:
>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&quickFilter=1659
>
> [Stalled Tickets]
> 63 tickets are stalled (have not been updated in >14d). 6 of these are
> tagged for alpha. Of the remaining, 14 are in beta and 43 are in rc.
>
> Stalled:
>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&quickFilter=1694
>
> [Open vs. Closed Last 7 Days & Cumulative Flow]
> We opened 14 issues in the last 7 days and closed 26 (8 of which were new).
> Most of the new tickets were test failures that were fixed in alpha. While
> the number of issues in the release continues to grow (many of which are
> flaky test failures), we continue to make more progress completing issues
> (as shown by this week's net of 12 closed tickets.
>
> Notable New Tickets:
> https://issues.apache.org/jira/browse/CASSANDRA-15690 is a critical read
> path bug that can lead to a transient incorrect response. This affects 4.0
> as well as previous versions of Cassandra including 3.11 and 3.0.x.
>
> Opened:
>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&quickFilter=1670
> Closed:
>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&quickFilter=1671
>
> Cumulative Flow:
>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&view=reporting&chart=cumulativeFlowDiagram&swimlane=939&swimlane=936&swimlane=931&column=1505&column=1506&column=1514&column=1509&column=1512&column=1507&days=30
>
> Thanks for all your efforts! Also, if there is anything else you would like
> to see in these weekly updates please let us know.
>
> Jordan
>

Re: 20200407 4.0 Status Update

[e . dimitrova]

Hi Manish,
You can check those tickets which have Complexity: Low Hanging Fruit

Ekaterina

Sent from my iPhone

> On 7 Apr 2020, at 20:04, Manish G  wrote:
> 
> Hi,
> 
> Can there be a filter like 'good first issue' which new people can use to
> find issues to start with?
> 
> Manish
> 
>> On Tue, Apr 7, 2020 at 10:44 PM Jordan West  wrote:
>> 
>> Hi Everyone,
>> 
>> The board can be found here:
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355
>> 
>> [Tickets That Need Attention]
>> A reminder that Josh has added a new 'Needs Attention' filter to show any
>> tasks that are stalled, need an assignee or need a reviewer. Makes it easy
>> if you would like to find something to work on that helps push us closer to
>> 4.0.0's release.
>> 
>> Needs Attention:
>> 
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&quickFilter=1719
>> 
>> [Alpha Status]
>> We continue to have no tickets that need assignees in alpha. Of the
>> remaining tickets in alpha, 8 have a reviewer / reviews are in progress, 3
>> are in need of reviewers, 3 are in progress, and 3 are not started or
>> require more information (or are being worked on but haven't had ticket
>> metadata updated). The tickets in need of a reviewer or are not yet started
>> are a great place to help out if you are looking for something to
>> contribute.
>> 
>> Needs Reviewer:
>> 
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&quickFilter=1659
>> 
>> [Stalled Tickets]
>> 63 tickets are stalled (have not been updated in >14d). 6 of these are
>> tagged for alpha. Of the remaining, 14 are in beta and 43 are in rc.
>> 
>> Stalled:
>> 
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&quickFilter=1694
>> 
>> [Open vs. Closed Last 7 Days & Cumulative Flow]
>> We opened 14 issues in the last 7 days and closed 26 (8 of which were new).
>> Most of the new tickets were test failures that were fixed in alpha. While
>> the number of issues in the release continues to grow (many of which are
>> flaky test failures), we continue to make more progress completing issues
>> (as shown by this week's net of 12 closed tickets.
>> 
>> Notable New Tickets:
>> https://issues.apache.org/jira/browse/CASSANDRA-15690 is a critical read
>> path bug that can lead to a transient incorrect response. This affects 4.0
>> as well as previous versions of Cassandra including 3.11 and 3.0.x.
>> 
>> Opened:
>> 
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&quickFilter=1670
>> Closed:
>> 
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&quickFilter=1671
>> 
>> Cumulative Flow:
>> 
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&view=reporting&chart=cumulativeFlowDiagram&swimlane=939&swimlane=936&swimlane=931&column=1505&column=1506&column=1514&column=1509&column=1512&column=1507&days=30
>> 
>> Thanks for all your efforts! Also, if there is anything else you would like
>> to see in these weekly updates please let us know.
>> 
>> Jordan
>> 

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org