Planning to port cqlsh to Python 3 (CASSANDRA-10190)
I propose porting cqlsh and cqlshlib to Python 3. End-of-life for Python 2.7 <https://legacy.python.org/dev/peps/pep-0373/> is currently planned for 1 January 2020. We should prepare to port the tool to a version of Python that will be officially supported. I'm seeking input on three questions: - Should we port it to straight Python 3, or Python 2/3 cross compatible? - How much more testing is needed? - Can we wait until after 4.0 for this? I have an implementation <https://github.com/ptbannister/cassandra/tree/cqlshlib3> to go with my proposal. In parallel with getting the dtest cqlsh_tests working again, I ported cqlsh.py and cqlshlib to Python 3. It passes with almost all of the dtests and the unittests, so it's in pretty good shape, although it's not 100% done (more on that below). *Python 3 or 2/3 cross compatible?* There are plenty of examples of Python libraries that are compatible with both Python 2 and Python 3 (notably the Cassandra Python driver), so I think this is achievable. The question is, do we want to pay the price of cross compatibility? If we write cqlsh to be 2/3 cross compatible, we'll carry a long term technical debt to maintain that feature. The value of continuing to support Python 2 will diminish over time. However, a cross compatible implementation may ease the transition for some users, especially if there are users who have made significant custom modifications to the Python 2.7 implementation of cqlsh, so I think we must at least consider the question. *What additional testing is needed before we could release it?* I used coverage.py to check on the code coverage of our existing dtest cqlsh_tests and cqlshlib unittests. There are several blind spots in our current testing that should be addressed before we release a port of cqlsh. Details of this are available on JIRA ticket CASSANDRA-10190 <https://issues.apache.org/jira/browse/CASSANDRA-10190> in the attachment coverage_notes.txt <https://issues.apache.org/jira/secure/attachment/12926015/coverage_notes.txt>. Beyond that, I've made no efforts to test on platforms other than Ubuntu and CentOS, so Windows testing is needed if we're making efforts to support Windows. It would also be preferable for some real users to try out the port before it replaces the Python 2.7 cqlsh in a release. Besides this, there are a couple of test failures I'm still trying to figure out, notably tests involving user defined map types (a task made more interesting by Python's general lack of support for immutable map types). *Can we wait until after 4.0 for this?* I don't think it's reasonable to try to release this with 4.0 given the current consensus around a feature freeze in the next few months. My feeling is that our testers and committers are already very busy with the currently planned changes for 4.0. I recommend planning toward a release to occur after 4.0. If we run up against Python 2.7 EOL before we can cut the next release, we could consider releasing a ported cqlsh independently, for installation through distutils or pip. Patrick Bannister
Re: Planning to port cqlsh to Python 3 (CASSANDRA-10190)
Kurt's proposal to freeze the 2.7 implementation and offer a Python 3 implementation alongside it appeals to my desire to free ourselves of design choices made for the sake of Python 2/3 cross compatibility. It also has the advantage of keeping Python 2.7 users on a time tested product. On the downside, offering two separate implementations would come at the cost of maintaining both, which would siphon community resources away from other areas more beneficial to the project. On that basis, I advocate for a cross compatible implementation over releasing two implementations. Patrick Bannister On Fri, Jun 1, 2018 at 6:49 PM, kurt greaves wrote: > Seems pretty straightforward to me. Create a python 3 version as soon as > possible and make it available, keep the python 2.7 version as default > until the next major release after 4.0 (assuming around/after python 2.7 > EOL), then switch default and leave continued support for 2.7 cqlsh up to > the community and whoever needs it. > > On Sat., 2 Jun. 2018, 04:45 J. D. Jordan, > wrote: > > > All for using six and supporting both. Sorry, I read your initial email > as > > wanting to drop support for 2 at the end of the year. > > > > > On Jun 1, 2018, at 1:01 PM, Jonathan Haddad wrote: > > > > > > And that's why I said supporting both with six is the right path > > > forward, later dropping support for 2. I'm not advocating we drop 2 > > > support now, and I'm not asking for any sort of commitment. I didn't > > > think adding support for 3 would be so controversial. > > > On Fri, Jun 1, 2018 at 9:40 AM Jeremiah D Jordan > > > wrote: > > >> > > >> The community of people doing python development and the community of > > people running Cassandra servers are not the same. I am not fine riding > > the coat tails of libraries used in python development. As others have > > stated we need to be following the lead of the OS vendors that people > will > > be deploying Cassandra on top of. And those will not be dropping Python > 2 > > at the end of the year. > > >> > > >> -Jeremiah > > >> > > >>> On Jun 1, 2018, at 12:37 PM, Jonathan Haddad > > wrote: > > >>> > > >>> Both can work. I did a lot of the work on the port of the Python > > >>> driver's object mapper (formerly cqlengine) to Python 3. It's > > >>> reasonably straightforward if you use the six library. > > >>> > > >>> Both pandas and numpy are dropping support for Python 2 at the end of > > >>> this year. I'm fine with riding on their coattails. > > >>>> On Fri, Jun 1, 2018 at 9:21 AM Russell Bateman < > r...@windofkeltia.com> > > wrote: > > >>>> > > >>>> Support for, but not the very script, right? Because, as gently > > pointed > > >>>> out by several realists here, Python 2 is far from dead and arguably > > >>>> still the majority usage. That's only just now beginning to change. > I > > >>>> think it will be more than 2 years before people begin asking what > > >>>> Python 2 was. > > >>>> > > >>>> > > >>>>> On 06/01/2018 10:10 AM, Jonathan Haddad wrote: > > >>>>> Supporting both as a next step is logical, removing support for 2 > in > > the > > >>>>> next year or two seems reasonable enough. Gotta rip the band aid > off > > at > > >>>>> some point. > > >>>>> > > >>>>>> On Fri, Jun 1, 2018 at 2:34 AM Michael Burman < > mibur...@redhat.com> > > wrote: > > >>>>>> > > >>>>>> Hi, > > >>>>>> > > >>>>>> Deprecating in this context does not mean removing it or it being > > >>>>>> replaced by 3 (RHEL 7.x will remain with Python 2.x as default). > It > > >>>>>> refers to future versions (>7), but there are none at this point. > It > > >>>>>> appears Ubuntu has deviated from Debian in this sense, but Debian > > has > > >>>>>> not changed yet (likely Debian 10 will, but that's not out yet and > > has > > >>>>>> no announced release date). > > >>>>>> > > >>>>>> Thus, 2.x still remains the most used version for servers. And > > servers > > >>>>>> deployed at this point of ti
Re: cqlsh tests and Python 3
I did a lot of work to make cqlsh compatible with Python 3 (and also Python 2.7) under CASSANDRA-10190. CASSANDRA-10190 has been blocked by CASSANDRA-14298, which got about two thirds of the cqlsh dtests to work. If somebody could commit to reviewing CASSANDRA-14298, I'd be willing to pick it back up and get it working with the current head of develop. That way we would be in a better place to address CASSANDRA-10190 and the Python 3 issue. Patrick Bannister On Tue, Feb 12, 2019 at 2:46 PM dinesh.jo...@yahoo.com.INVALID wrote: > I saw that thread and the tickets. They haven't had any activity recently. > Given that it is already Feb 2019 and Python 2.7 is getting close to EOL'd, > I think it's worth moving forward with deprecating Python 2.7 support and > adding 3.0 support prior to 4.0 release. I am not sure what the timeline is > looking like for 4.0 right now but it would be silly if we release 4.0 so > close to Python 2.7 EOL. > Dinesh > > On Tuesday, February 12, 2019, 7:00:45 AM PST, Stefan Podkowinski < > s...@apache.org> wrote: > > Previous discussion can be found here: > > > https://lists.apache.org/thread.html/cbc50f5ac085ac759b52eb7e87277a3b82e2773c6d507c4b525d@%3Cdev.cassandra.apache.org%3E > > > On 11.02.19 19:58, Ariel Weisberg wrote: > > Hi, > > > > Do you mean Python 2/3 compatibility? > > > > This has been discussed earlier and I think that being compatible with > both is an easier sell. > > > > Ariel > > > >> On Feb 11, 2019, at 1:24 PM, dinesh.jo...@yahoo.com.INVALID < > dinesh.jo...@yahoo.com.invalid> wrote: > >> > >> Hey all, > >> We've gotten the cqlsh tests running in the Cassandra repo (these are > distinct from the cqlsh tests in dtests repo). They're in Python 2.7 and > using the nosetests. We'd like to make them consistent with the rest of the > tests which means moving them to Python 3 & Pytest framework. However this > would involve migrating cqlsh to Python 3. Does anybody have any concerns > if we move cqlsh to Python 3? Please note that Python 2 is EOL'd and will > be unsupported in about 10 months. > >> So here are the options - > >> 1. Leave cqlsh in Python 2.7 & nosetests. Just make sure they're > running as part of the build process.2. Move cqlsh to Python 3 & pytests.3. > Leave cqlsh in Python 2.7 but move to Pytests. This option doesn't really > add much value though. > >> Thanks, > >> Dinesh > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
Choosing a supported Python 3 major version for cqlsh
Hello, I'm resuming work on Python 3 support for cqlsh (CASSANDRA-10190). As discussed before, the plan for this work is to get it working on Python 3 while keeping it compatible with Python 2.7. I'd like to settle on a Python 3 release to be our officially supported tested version. It would be burdensome for the project to test and maintain cqlshlib on more than one Python 3 release. I recommend we pick the longest supported stable release available. That would be Python 3.7, which is planned to get its last release in 2023, four years from now. - Python 3.5 was planned to get its last major release yesterday - Python 3.6 is planned to get its last major release in December 2021, about three years from now Any feedback on picking a tested Python version for cqlshlib? I'm inclined to focus on Python 3.7 as we push toward finishing the ticket. Patrick Bannister
