Re: [Python-Dev] Some PRs to merge?
On 10/20, Victor Stinner wrote: Le ven. 19 oct. 2018 à 19:01, Stephane Wirtel a écrit : total: 49 PRs is:open is:pr review:approved status:success label:"awaiting merge" -label:"DO-NOT-MERGE" label:""LA signed"" I merged many PRs and closed a few (2 if I recall correctly). Your query now counts 24 PRs. Really nice for your job, I think the contributors will appreciate a lot. Victor I would like to know if you are interested by this kind of reports. I can interact with the GraphQL api of GitHub and just provide the report via email to python-dev@python.org Have a nice day and thank you for your merges. Stéphane -- Stéphane Wirtel - https://wirtel.be - @matrixise ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Some PRs to merge?
This is terrific work. We all know that the best way to encourage contributors is to use their usable contributions. Thank you very much, Stephane and Victor (again)! Steve Holden On Sat, Oct 20, 2018 at 10:32 AM, Stephane Wirtel wrote: > On 10/20, Victor Stinner wrote: > >> Le ven. 19 oct. 2018 à 19:01, Stephane Wirtel a >> écrit : >> >>> total: 49 PRs >>> is:open is:pr review:approved status:success label:"awaiting merge" >>> -label:"DO-NOT-MERGE" label:""LA signed"" >>> >> >> I merged many PRs and closed a few (2 if I recall correctly). Your >> query now counts 24 PRs. >> > Really nice for your job, I think the contributors will appreciate a > lot. > >> >> Victor >> > > I would like to know if you are interested by this kind of reports. I > can interact with the GraphQL api of GitHub and just provide the report > via email to python-dev@python.org > > Have a nice day and thank you for your merges. > > Stéphane > > -- > Stéphane Wirtel - https://wirtel.be - @matrixise > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve% > 40holdenweb.com > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] The future of the wchar_t cache
Currently the PyUnicode object contains two caches: for UTF-8 representation and for wchar_t representation. They are needed not for optimization but for supporting C API which returns borrowed references for such representations. The UTF-8 cache always was in unicode objects (but in Python 2 it was not a UTF-8 cache, but a 8-bit representation cache). Initially it was needed for compatibility with 8-bit str, for implementing the "s" and "z" format units in PyArg_Parse(). Now it is used also for PyUnicode_AsUTF8() and PyUnicode_AsUTF8AndSize(). The wchar_t cache was added with PEP 393 in 3.3 as a replacement for the former Py_UNICODE representation. Now Py_UNICODE is defined as an alias of wchar_t, and the C API which returned a pointer to Py_UNICODE content returns now a pointer to the cached wchar_t representation. It is the "u" and "Z" format units in PyArg_Parse(), PyUnicode_AsUnicode(), PyUnicode_AsUnicodeAndSize(), PyUnicode_GET_SIZE(), PyUnicode_GET_DATA_SIZE(), PyUnicode_AS_UNICODE(), PyUnicode_AS_DATA(). All this increase the size of the unicode object. It includes the constant overhead of additional pointer and size fields, and the overhead of the cached representation proportional to the string length. The following table contains number of bytes per character for different kinds, with and without filling specified caches. raw +utf8 +wchar_t +utf8+wchar_t Windows Linux Windows Linux ASCII 1 1 3 53 5 UCS112-3 3 5 4-5 6-7 UCS223-5 2 6 3-5 7-9 UCS445-8 6-8 4 7-125-8 There is also a new C API added in 3.3 for getting wchar_t representation without using the cache: PyUnicode_AsWideChar() and PyUnicode_AsWideCharString(). Currently it uses the cache, this has benefits and disadvantages. Old Py_UNICODE based API is deprecated, and will be removed eventually. I want to ask about the future of the wchar_t cache. Is the benefit of caching the wchar_t representation larger the disadvantage of spending more memory? The wchar_t representation is so natural for Windows API as the UTF8 representation for POSIX API. But in all other cases it is just waste of memory. Are there reasons of keeping the wchar_t cache after removing the deprecated API? I have rewrote PyUnicode_AsWideChar() and PyUnicode_AsWideCharString(). They were implemented via the old Py_UNICODE based API, and now they don't use deprecated functions. They still use the wchar_t cache if it was created by previous use of the deprecated API, but don't create it themselves. Is this the correct decision? https://bugs.python.org/issue30863 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Some PRs to merge?
20.10.18 04:08, Victor Stinner пише: Le ven. 19 oct. 2018 à 19:01, Stephane Wirtel a écrit : total: 49 PRs is:open is:pr review:approved status:success label:"awaiting merge" -label:"DO-NOT-MERGE" label:""LA signed"" I merged many PRs and closed a few (2 if I recall correctly). Your query now counts 24 PRs. Thank you Victor! I prefer to merge my PRs and PRs assigned to me myself, but I am not sure that I would merge all PRs that can be merged in the nearest future. ;) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The future of the wchar_t cache
+1 to remove wchar_t cache. I hope we can remove it at Python 3.9 or 3.10. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The future of the wchar_t cache
Serhiy Storchaka schrieb am 20.10.2018 um 13:06: > Currently the PyUnicode object contains two caches: for UTF-8 > representation and for wchar_t representation. They are needed not for > optimization but for supporting C API which returns borrowed references for > such representations. > > The UTF-8 cache always was in unicode objects (but in Python 2 it was not a > UTF-8 cache, but a 8-bit representation cache). Initially it was needed for > compatibility with 8-bit str, for implementing the "s" and "z" format units > in PyArg_Parse(). Now it is used also for PyUnicode_AsUTF8() and > PyUnicode_AsUTF8AndSize(). > > The wchar_t cache was added with PEP 393 in 3.3 as a replacement for the > former Py_UNICODE representation. Now Py_UNICODE is defined as an alias of > wchar_t, and the C API which returned a pointer to Py_UNICODE content > returns now a pointer to the cached wchar_t representation. It is the "u" > and "Z" format units in PyArg_Parse(), PyUnicode_AsUnicode(), > PyUnicode_AsUnicodeAndSize(), PyUnicode_GET_SIZE(), > PyUnicode_GET_DATA_SIZE(), PyUnicode_AS_UNICODE(), PyUnicode_AS_DATA(). > > All this increase the size of the unicode object. It includes the constant > overhead of additional pointer and size fields, and the overhead of the > cached representation proportional to the string length. The following > table contains number of bytes per character for different kinds, with and > without filling specified caches. > > raw +utf8 +wchar_t +utf8+wchar_t > Windows Linux Windows Linux > ASCII 1 1 3 5 3 5 > UCS1 1 2-3 3 5 4-5 6-7 > UCS2 2 3-5 2 6 3-5 7-9 > UCS4 4 5-8 6-8 4 7-12 5-8 > > There is also a new C API added in 3.3 for getting wchar_t representation > without using the cache: PyUnicode_AsWideChar() and > PyUnicode_AsWideCharString(). Currently it uses the cache, this has > benefits and disadvantages. > > Old Py_UNICODE based API is deprecated, and will be removed eventually. > I want to ask about the future of the wchar_t cache. Is the benefit of > caching the wchar_t representation larger the disadvantage of spending more > memory? The wchar_t representation is so natural for Windows API as the > UTF8 representation for POSIX API. But in all other cases it is just waste > of memory. Are there reasons of keeping the wchar_t cache after removing > the deprecated API? I'd be happy to get rid of it. But regarding the use under Windows, I wonder if there's interest in keeping it as a special Windows-only feature, e.g. to speed up the data exchange with the Win32 APIs. I guess it would have to provide a visible (performance?) advantage to justify such special casing over the code removal. Stefan ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The future of the wchar_t cache
On 20Oct2018 0901, Stefan Behnel wrote: I'd be happy to get rid of it. But regarding the use under Windows, I wonder if there's interest in keeping it as a special Windows-only feature, e.g. to speed up the data exchange with the Win32 APIs. I guess it would have to provide a visible (performance?) advantage to justify such special casing over the code removal. I think these cases would be just as well served by maintaining the original UCS-2 representation even if the maximum character fits into UCS-1, and only do the conversion when Python copies the string into a new location. I don't have numbers, but my instinct says the most impacted operations would be retrieving collections of strings from the OS (avoiding a scan/conversion for each one), comparisons against these collections (in-memory handling for hash/comparison of mismatched KIND), and passing some of these strings back to the OS (conversion back into UCS-2). This is basically a glob/fnmatch/stat sequence, and is the main real scenario I can think of where Python's overhead might become noticeable. Another option that might be useful is some way to allow the UCS-1/4 <-> UCS-2 conversion to occur outside the GIL. Most of the time when we need to convert we're about to release the GIL (or have just recovered it), so even without the cache we could probably recover some of the performance impact in parallelism. (That said, these are often tied up in conditions and generated code, so it may not be as easy to do this as retaining the original format.) Some sort of tracing to see how often the cache is reused after being generated would be interesting, as well as how often the cache is being generated for a string that was originally in UCS-2 but we changed it to UCS-1. Cheers, Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] [RELEASE] Python 3.7.1 and 3.6.7 are now available
On behalf of the Python development community and the Python 3.7 release team, we are pleased to announce the availability of Python 3.7.1. Python 3.7.1 is the first maintenance release of the newest feature release of the Python language. You can find Python 3.7.1 here: https://www.python.org/downloads/release/python-371/ See the What’s New In Python 3.7 document for more information about the many new features and optimizations included in the 3.7 series. Detailed information about the changes made in 3.7.1 can be found in its change log: https://docs.python.org/3.7/whatsnew/3.7.html https://docs.python.org/3.7/whatsnew/changelog.html#python-3-7-1-final We are also happy to announce the availability of Python 3.6.7, the next maintenance release of Python 3.6: https://www.python.org/downloads/release/python-367/ Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation. -- Ned Deily n...@python.org -- [] ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.7.1 and 3.6.7 are now available
On 10/20/2018 1:37 PM, Ned Deily wrote: We are also happy to announce the availability of Python 3.6.7, the next maintenance release of Python 3.6: https://www.python.org/downloads/release/python-367/ Am I correct in thinking that there will be just one more maintenance release before going on security-only status? -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.7.1 and 3.6.7 are now available
On Oct 20, 2018, at 15:54, Terry Reedy wrote: > > On 10/20/2018 1:37 PM, Ned Deily wrote: > >> We are also happy to announce the availability of Python 3.6.7, the next >> maintenance release of Python 3.6: >> https://www.python.org/downloads/release/python-367/ > > Am I correct in thinking that there will be just one more maintenance release > before going on security-only status? Yes, that's the published plan: https://www.python.org/dev/peps/pep-0494/ 3.6.8 schedule (2018-12) Final maintenance mode release, final binary releases. 3.6.9 and beyond schedule Security fixes only, as needed, until 2021-12 -- Ned Deily n...@python.org -- [] ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Servicing pypi.python.org
El jue., 18 de oct. de 2018 a la(s) 00:07, Donald Stufft (don...@stufft.io) escribió: > tl;dr: can we have a (semi)permanent redirect from pypi.python.org to > pypi.org? > > > > This already exists: Indeed, it works! I was in a place with a very crappy internet, but I never suspected the issue I was getting was part of that. Anyway, everything is fine! Thanks for the help and sorry for the noise :) Regards, -- .Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org.ar/ Twitter: @facundobatista ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
