Re: [Python-Dev] PEP 561: Distributing and Packaging Type Information
On Thu, Oct 26, 2017 at 3:42 PM, Ethan Smith wrote:
> However, the stubs may be put in a sub-folder
> of the Python sources, with the same name the ``*.py`` files are in. For
> example, the ``flyingcircus`` package would have its stubs in the folder
> ``flyingcircus/flyingcircus/``. This path is chosen so that if stubs are
> not found in ``flyingcircus/`` the type checker may treat the subdirectory as
> a normal package.
I admit that I find this aesthetically unpleasant. Wouldn't something
like __typestubs__/ be a more Pythonic name? (And also avoid potential
name clashes, e.g. my async_generator package has a top-level export
called async_generator; normally you do 'from async_generator import
async_generator'. I think that might cause problems if I created an
async_generator/async_generator/ directory, especially post-PEP 420.)
I also don't understand the given rationale -- it sounds like you want
to be able say well, if ${SOME_DIR_ON_PYTHONPATH}/flyingcircus/
doesn't contain stubs, then just stick the
${SOME_DIR_ON_PYTHONPATH}/flyingcircus/ directory *itself* onto
PYTHONPATH, and then try again. But that's clearly the wrong thing,
because then you'll also be adding a bunch of other random junk into
that directory into the top-level namespace. For example, suddenly the
flyingcircus.summarise_proust module has become a top-level
summarise_proust package. I must be misunderstanding something?
> Type Checker Module Resolution Order
>
>
> The following is the order that type checkers supporting this PEP should
> resolve modules containing type information:
>
> 1. User code - the files the type checker is running on.
>
> 2. Stubs or Python source manually put in the beginning of the path. Type
>checkers should provide this to allow the user complete control of which
>stubs to use, and patch broken stubs/inline types from packages.
>
> 3. Third party stub packages - these packages can supersede the installed
>untyped packages. They can be found at ``pkg-stubs`` for package ``pkg``,
>however it is encouraged to check the package's metadata using packaging
>query APIs such as ``pkg_resources`` to assure that the package is meant
>for type checking, and is compatible with the installed version.
Am I right that this means you need to be able to map from import
names to distribution names? I.e., if you see 'import foo', you need
to figure out which *.dist-info directory contains metadata for the
'foo' package? How do you plan to do this?
The problem is that technically, import names and distribution names
are totally unrelated namespaces -- for example, the '_pytest' package
comes from the 'pytest' distribution, the 'pylab' package comes from
'matplotlib', and 'pip install scikit-learn' gives you a package
imported as 'sklearn'. Namespace packages are also challenging,
because a single top-level package might actually be spread across
multiple distributions.
-n
--
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 561: Distributing and Packaging Type Information
On Thu, 26 Oct 2017 15:42:19 -0700 Ethan Smith wrote: > Stub Only Packages > '' > > For package maintainers wishing to ship stub files containing all of their > type information, it is prefered that the ``*.pyi`` stubs are alongside the > corresponding ``*.py`` files. However, the stubs may be put in a sub-folder > of the Python sources, with the same name the ``*.py`` files are in. For > example, the ``flyingcircus`` package would have its stubs in the folder > ``flyingcircus/flyingcircus/``. This path is chosen so that if stubs are > not found in ``flyingcircus/`` the type checker may treat the subdirectory as > a normal package. The normal resolution order of checking ``*.pyi`` before > ``*.py`` will be maintained. I am not sure I understand the rationale for this. What would be the problem with looking for the stubs in a directory named, e.g; "flyingcircus/__typing__"? Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 561: Distributing and Packaging Type Information
On Fri, 27 Oct 2017 11:31:04 +0200 Antoine Pitrou wrote: > On Thu, 26 Oct 2017 15:42:19 -0700 > Ethan Smith wrote: > > Stub Only Packages > > '' > > > > For package maintainers wishing to ship stub files containing all of their > > type information, it is prefered that the ``*.pyi`` stubs are alongside the > > corresponding ``*.py`` files. However, the stubs may be put in a sub-folder > > of the Python sources, with the same name the ``*.py`` files are in. For > > example, the ``flyingcircus`` package would have its stubs in the folder > > ``flyingcircus/flyingcircus/``. This path is chosen so that if stubs are > > not found in ``flyingcircus/`` the type checker may treat the subdirectory > > as > > a normal package. The normal resolution order of checking ``*.pyi`` before > > ``*.py`` will be maintained. > > I am not sure I understand the rationale for this. What would be the > problem with looking for the stubs in a directory named, e.g; > "flyingcircus/__typing__"? I just saw Nathaniel asked the same question above. Sorry for the noise! Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] \G (match last position) regex operator non-existant in python?
All, perl has a regex assertion (\G) that allows multiple-match regular expressions to be able to use the position of the last match. Perl's documentation puts it this way: \G Match only at pos() (e.g. at the end-of-match position of prior m//g) Anyways, this is exceedingly powerful for matching regularly structured free-form records, and I was really surprised when I found out that python did not have it. For example, if findall supported this, it would be possible to write things like this (a quick and dirty ifconfig parser): pat = re.compile(r'\G(\S+)(.*?\n)(?=\S+|\Z)', re.S) val = """ eth2 Link encap:Ethernet HWaddr xx inet addr: xx.xx.xx.xx Bcast:xx.xx.xx.xx Mask:xx.xx.xx.xx ... loLink encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 """ matches = re.findall(pat, val) So - why doesn't python have this? is it something that simply was overlooked, or is there another method of doing the same thing with arbitrarily complex freeform records? thanks much.. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] If aligned_alloc() is missing on your platform, please let us know.
Hello, we want to add aligned versions of allocation functions to 3.7: https://bugs.python.org/issue18835 C11 has aligned_alloc(). Linux, BSD, OSX, MSVC, Android all have either posix_memalign() or _aligned_malloc(). Cygwin apparently has posix_memalign(). MinGW has: https://github.com/Alexpux/mingw-w64/blob/master/mingw-w64-crt/misc/mingw-aligned-malloc.c Victor wrote a patch and would like to avoid adding a (probably unnecessary) emulation function. I agree with that. So if any platform does not have some form of aligned_alloc(), please speak up. Stefan Krah ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Migrate python-dev to Mailman 3?
Barry Warsaw wrote: > In fact, we come with plugins for mail-archive.com and MHonarc. MHonarc output is nice, practically the same as pipermail: https://lists.debian.org/debian-x/2010/12/threads.html If it is possible to enable (and maintain!) a MHonarc archive with just blue links along with the new interface, everyone should be happy. Stefan Krah ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP Post-History (was Re: PEP 561: Distributing and Packaging Type Information)
On Oct 27, 2017, at 00:12, Guido van Rossum wrote: > > Heh, you're right that was the reasoning. But I think python-list is much > less valuable than python-ideas for PEP authors. So let's change it. Sounds good. I just want to make sure we keep python-dev in the loop. This is a process change though, so I’ll work with the PR#441 author to get the update into the PEPs, and then make the announcement on the relevant mailing lists. Cheers, -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] \G (match last position) regex operator non-existant in python?
The "why" question is not very interesting -- it probably wasn't in PCRE and nobody was familiar with it when we moved off PCRE (maybe it wasn't even in Perl at the time -- it was ~15 years ago). I didn't understand your description of \G so I googled it and found a helpful StackOverflow article: https://stackoverflow.com/questions/21971701/when-is-g-useful-application-in-a-regex. >From this I understand that when using e.g. findall() it forces successive matches to be adjacent. In general this seems to be a unique property of \G: it preserves *state* from one match to the next. This will make it somewhat difficult to implement -- e.g. that state should probably be thread-local in case multiple threads use the same compiled regex. It's also unclear when that state should be reset. (Only when you compile the regex? Each time you pass it a different source string?) So I'm not sure it's reasonable to add. But I also don't see a reason why it shouldn't be added -- presuming we can decide on good answer for the questions above about the "scope" of the anchor. I think it's okay to start a discussion on bugs.python.org about the precise specification of \G for Python. OTOH I expect that most core devs won't find this a very interesting problem (Python relies on regexes for parsing a lot less than Perl does). Good luck! On Thu, Oct 26, 2017 at 11:03 PM, Ed Peschko wrote: > All, > > perl has a regex assertion (\G) that allows multiple-match regular > expressions to be able to use the position of the last match. Perl's > documentation puts it this way: > > \G Match only at pos() (e.g. at the end-of-match position of prior > m//g) > > Anyways, this is exceedingly powerful for matching regularly > structured free-form records, and I was really surprised when I found > out that python did not have it. For example, if findall supported > this, it would be possible to write things like this (a quick and > dirty ifconfig parser): > > pat = re.compile(r'\G(\S+)(.*?\n)(?=\S+|\Z)', re.S) > > val = """ > eth2 Link encap:Ethernet HWaddr xx > inet addr: xx.xx.xx.xx Bcast:xx.xx.xx.xx Mask:xx.xx.xx.xx > ... > loLink encap:Local Loopback >inet addr:127.0.0.1 Mask:255.0.0.0 > """ > matches = re.findall(pat, val) > > So - why doesn't python have this? is it something that simply was > overlooked, or is there another method of doing the same thing with > arbitrarily complex freeform records? > > thanks much.. > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP Post-History (was Re: PEP 561: Distributing and Packaging Type Information)
Great! On Fri, Oct 27, 2017 at 8:27 AM, Barry Warsaw wrote: > On Oct 27, 2017, at 00:12, Guido van Rossum wrote: > > > > Heh, you're right that was the reasoning. But I think python-list is > much less valuable than python-ideas for PEP authors. So let's change it. > > Sounds good. I just want to make sure we keep python-dev in the loop. > > This is a process change though, so I’ll work with the PR#441 author to > get the update into the PEPs, and then make the announcement on the > relevant mailing lists. > > Cheers, > -Barry > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] \G (match last position) regex operator non-existant in python?
Note that Matthew Barnett's `regex` module already supports \G, and a great many other features that weren't around 15 years ago ;-) either: https://pypi.python.org/pypi/regex/ I haven't followed this in detail. I'm just surprised once per year that it hasn't been folded into the core ;-) [nothing new below] On Fri, Oct 27, 2017 at 10:35 AM, Guido van Rossum wrote: > The "why" question is not very interesting -- it probably wasn't in PCRE and > nobody was familiar with it when we moved off PCRE (maybe it wasn't even in > Perl at the time -- it was ~15 years ago). > > I didn't understand your description of \G so I googled it and found a > helpful StackOverflow article: > https://stackoverflow.com/questions/21971701/when-is-g-useful-application-in-a-regex. > From this I understand that when using e.g. findall() it forces successive > matches to be adjacent. > > In general this seems to be a unique property of \G: it preserves *state* > from one match to the next. This will make it somewhat difficult to > implement -- e.g. that state should probably be thread-local in case > multiple threads use the same compiled regex. It's also unclear when that > state should be reset. (Only when you compile the regex? Each time you pass > it a different source string?) > > So I'm not sure it's reasonable to add. But I also don't see a reason why it > shouldn't be added -- presuming we can decide on good answer for the > questions above about the "scope" of the anchor. > > I think it's okay to start a discussion on bugs.python.org about the precise > specification of \G for Python. OTOH I expect that most core devs won't find > this a very interesting problem (Python relies on regexes for parsing a lot > less than Perl does). > > Good luck! > > On Thu, Oct 26, 2017 at 11:03 PM, Ed Peschko wrote: >> >> All, >> >> perl has a regex assertion (\G) that allows multiple-match regular >> expressions to be able to use the position of the last match. Perl's >> documentation puts it this way: >> >> \G Match only at pos() (e.g. at the end-of-match position of prior >> m//g) >> >> Anyways, this is exceedingly powerful for matching regularly >> structured free-form records, and I was really surprised when I found >> out that python did not have it. For example, if findall supported >> this, it would be possible to write things like this (a quick and >> dirty ifconfig parser): >> >> pat = re.compile(r'\G(\S+)(.*?\n)(?=\S+|\Z)', re.S) >> >> val = """ >> eth2 Link encap:Ethernet HWaddr xx >> inet addr: xx.xx.xx.xx Bcast:xx.xx.xx.xx Mask:xx.xx.xx.xx >> ... >> loLink encap:Local Loopback >>inet addr:127.0.0.1 Mask:255.0.0.0 >> """ >> matches = re.findall(pat, val) >> >> So - why doesn't python have this? is it something that simply was >> overlooked, or is there another method of doing the same thing with >> arbitrarily complex freeform records? >> >> thanks much.. >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/guido%40python.org > > > > > -- > --Guido van Rossum (python.org/~guido) > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/tim.peters%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] \G (match last position) regex operator non-existant in python?
Oh. Yes, that is being discussed about once a year two. It seems Matthew isn't very interested in helping out with the port, and there are some concerns about backwards compatibility with the `re` module. I think it needs a champion! On Fri, Oct 27, 2017 at 8:50 AM, Tim Peters wrote: > Note that Matthew Barnett's `regex` module already supports \G, and a > great many other features that weren't around 15 years ago ;-) either: > > https://pypi.python.org/pypi/regex/ > > I haven't followed this in detail. I'm just surprised once per year > that it hasn't been folded into the core ;-) > > [nothing new below] > > On Fri, Oct 27, 2017 at 10:35 AM, Guido van Rossum > wrote: > > The "why" question is not very interesting -- it probably wasn't in PCRE > and > > nobody was familiar with it when we moved off PCRE (maybe it wasn't even > in > > Perl at the time -- it was ~15 years ago). > > > > I didn't understand your description of \G so I googled it and found a > > helpful StackOverflow article: > > https://stackoverflow.com/questions/21971701/when-is-g- > useful-application-in-a-regex. > > From this I understand that when using e.g. findall() it forces > successive > > matches to be adjacent. > > > > In general this seems to be a unique property of \G: it preserves *state* > > from one match to the next. This will make it somewhat difficult to > > implement -- e.g. that state should probably be thread-local in case > > multiple threads use the same compiled regex. It's also unclear when that > > state should be reset. (Only when you compile the regex? Each time you > pass > > it a different source string?) > > > > So I'm not sure it's reasonable to add. But I also don't see a reason > why it > > shouldn't be added -- presuming we can decide on good answer for the > > questions above about the "scope" of the anchor. > > > > I think it's okay to start a discussion on bugs.python.org about the > precise > > specification of \G for Python. OTOH I expect that most core devs won't > find > > this a very interesting problem (Python relies on regexes for parsing a > lot > > less than Perl does). > > > > Good luck! > > > > On Thu, Oct 26, 2017 at 11:03 PM, Ed Peschko wrote: > >> > >> All, > >> > >> perl has a regex assertion (\G) that allows multiple-match regular > >> expressions to be able to use the position of the last match. Perl's > >> documentation puts it this way: > >> > >> \G Match only at pos() (e.g. at the end-of-match position of prior > >> m//g) > >> > >> Anyways, this is exceedingly powerful for matching regularly > >> structured free-form records, and I was really surprised when I found > >> out that python did not have it. For example, if findall supported > >> this, it would be possible to write things like this (a quick and > >> dirty ifconfig parser): > >> > >> pat = re.compile(r'\G(\S+)(.*?\n)(?=\S+|\Z)', re.S) > >> > >> val = """ > >> eth2 Link encap:Ethernet HWaddr xx > >> inet addr: xx.xx.xx.xx Bcast:xx.xx.xx.xx Mask:xx.xx.xx.xx > >> ... > >> loLink encap:Local Loopback > >>inet addr:127.0.0.1 Mask:255.0.0.0 > >> """ > >> matches = re.findall(pat, val) > >> > >> So - why doesn't python have this? is it something that simply was > >> overlooked, or is there another method of doing the same thing with > >> arbitrarily complex freeform records? > >> > >> thanks much.. > >> ___ > >> Python-Dev mailing list > >> Python-Dev@python.org > >> https://mail.python.org/mailman/listinfo/python-dev > >> Unsubscribe: > >> https://mail.python.org/mailman/options/python-dev/guido%40python.org > > > > > > > > > > -- > > --Guido van Rossum (python.org/~guido) > > > > ___ > > Python-Dev mailing list > > Python-Dev@python.org > > https://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > > https://mail.python.org/mailman/options/python-dev/ > tim.peters%40gmail.com > > > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2017-10-20 - 2017-10-27) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open6260 ( +0) closed 37377 (+59) total 43637 (+59) Open issues with patches: 2411 Issues opened (51) == #18835: Add aligned memory variants to the suite of PyMem functions/ma https://bugs.python.org/issue18835 reopened by skrah #28224: Compilation warnings on Windows: export 'PyInit_xx' specified https://bugs.python.org/issue28224 reopened by skrah #30768: PyThread_acquire_lock_timed() should recompute the timeout whe https://bugs.python.org/issue30768 reopened by haypo #31811: async and await missing from keyword list in lexical analysis https://bugs.python.org/issue31811 reopened by yselivanov #31828: Support Py_tss_NEEDS_INIT outside of static initialisation https://bugs.python.org/issue31828 opened by scoder #31829: Portability issues with pickle https://bugs.python.org/issue31829 opened by serhiy.storchaka #31830: asyncio.create_subprocess_exec doesn't capture all stdout outp https://bugs.python.org/issue31830 opened by cannedrag #31831: EmailMessage.add_attachment(filename="long or spécial") crash https://bugs.python.org/issue31831 opened by calimeroteknik #31834: BLAKE2: the (pure) SSE2 impl forced on x86_64 is slower than r https://bugs.python.org/issue31834 opened by mgorny #31836: test_code_module fails after test_idle https://bugs.python.org/issue31836 opened by serhiy.storchaka #31837: ParseError in test_all_project_files() https://bugs.python.org/issue31837 opened by serhiy.storchaka #31839: datetime: add method to parse isoformat() output https://bugs.python.org/issue31839 opened by orent #31841: Several methods of collections.UserString do not return instan https://bugs.python.org/issue31841 opened by vaultah #31842: pathlib: "Incorrect function" during resolve() https://bugs.python.org/issue31842 opened by earonesty2 #31843: sqlite3.connect() should accept PathLike objects https://bugs.python.org/issue31843 opened by Allen Li #31844: HTMLParser: undocumented not implemented method https://bugs.python.org/issue31844 opened by srittau #31846: Error in 3.6.3 epub docs https://bugs.python.org/issue31846 opened by n8henrie #31848: "aifc" module does not always initialize "Aifc_read._ssnd_chun https://bugs.python.org/issue31848 opened by Zero #31849: Python/pyhash.c warning: comparison of integers of different s https://bugs.python.org/issue31849 opened by xdegaye #31850: test_nntplib failed with "nntplib.NNTPDataError: line too long https://bugs.python.org/issue31850 opened by haypo #31851: test_subprocess hangs randomly on x86 Windows7 3.x https://bugs.python.org/issue31851 opened by haypo #31852: Crashes with lines of the form "async \" https://bugs.python.org/issue31852 opened by Alexandre Hamelin #31853: Use super().method instead of socket.method in SSLSocket https://bugs.python.org/issue31853 opened by earonesty #31854: Add mmap.ACCESS_DEFAULT to namespace https://bugs.python.org/issue31854 opened by Justus Schwabedal #31855: mock_open is not compatible with read(n) (and pickle.load) https://bugs.python.org/issue31855 opened by ron.rothman #31858: IDLE: cleanup use of sys.ps1 and never set it. https://bugs.python.org/issue31858 opened by terry.reedy #31859: sharedctypes.RawArray initialization https://bugs.python.org/issue31859 opened by meetaig #31860: IDLE: Make font sample editable https://bugs.python.org/issue31860 opened by serhiy.storchaka #31861: aiter() and anext() built-in functions https://bugs.python.org/issue31861 opened by davide.rizzo #31862: Port the standard library to PEP 489 multiphase initialization https://bugs.python.org/issue31862 opened by Dormouse759 #31863: Inconsistent returncode/exitcode for terminated child processe https://bugs.python.org/issue31863 opened by Akos Kiss #31865: html.unescape does not work as per documentation https://bugs.python.org/issue31865 opened by cardin #31867: Duplicated keys in MIME type_map with different values https://bugs.python.org/issue31867 opened by Mark.Shannon #31868: Null pointer dereference in ndb.ndbm get when used with a defa https://bugs.python.org/issue31868 opened by tmiasko #31869: commentary on ssl.PROTOCOL_TLS https://bugs.python.org/issue31869 opened by J Sloot #31870: add timeout parameter for get_server_certificate in ssl.py https://bugs.python.org/issue31870 opened by Nixawk #31871: Support for file descriptor params in os.path https://bugs.python.org/issue31871 opened by Mateusz Kurek #31872: SSL BIO is broken for internationalized domains https://bugs.python.org/issue31872 opened by asvetlov #31873: Inconsistent capitalization of proper noun - Unicode. https://bugs.python.org/issue31873 opened by toonarmycaptain #31874: [feature] runpy.run_module should mimic script launch behavior
Re: [Python-Dev] \G (match last position) regex operator non-existant in python?
> From this I understand that when using e.g. findall() it forces successive > matches to be adjacent. yes, I admit that this is a clearer description of what \G does. My only defense is that I wrote my description when it was late. :) I can only stress how useful it is, especially for debugging regexes. Basically if you are cutting up any string into discrete chunks, you want to make sure that you aren't missing any chunks in the middle when you do the cut. without \G, you can miss large sections of string, and it is easy to overlook. with \G, you are guaranteed to see exactly where your regex falls down. In addition, there are specific regexes that you can only write with \G (eg. c parsers) Anyways, I'll look at regex. On Fri, Oct 27, 2017 at 8:35 AM, Guido van Rossum wrote: > The "why" question is not very interesting -- it probably wasn't in PCRE and > nobody was familiar with it when we moved off PCRE (maybe it wasn't even in > Perl at the time -- it was ~15 years ago). > > I didn't understand your description of \G so I googled it and found a > helpful StackOverflow article: > https://stackoverflow.com/questions/21971701/when-is-g-useful-application-in-a-regex. > From this I understand that when using e.g. findall() it forces successive > matches to be adjacent. > > In general this seems to be a unique property of \G: it preserves *state* > from one match to the next. This will make it somewhat difficult to > implement -- e.g. that state should probably be thread-local in case > multiple threads use the same compiled regex. It's also unclear when that > state should be reset. (Only when you compile the regex? Each time you pass > it a different source string?) > > So I'm not sure it's reasonable to add. But I also don't see a reason why it > shouldn't be added -- presuming we can decide on good answer for the > questions above about the "scope" of the anchor. > > I think it's okay to start a discussion on bugs.python.org about the precise > specification of \G for Python. OTOH I expect that most core devs won't find > this a very interesting problem (Python relies on regexes for parsing a lot > less than Perl does). > > Good luck! > > On Thu, Oct 26, 2017 at 11:03 PM, Ed Peschko wrote: >> >> All, >> >> perl has a regex assertion (\G) that allows multiple-match regular >> expressions to be able to use the position of the last match. Perl's >> documentation puts it this way: >> >> \G Match only at pos() (e.g. at the end-of-match position of prior >> m//g) >> >> Anyways, this is exceedingly powerful for matching regularly >> structured free-form records, and I was really surprised when I found >> out that python did not have it. For example, if findall supported >> this, it would be possible to write things like this (a quick and >> dirty ifconfig parser): >> >> pat = re.compile(r'\G(\S+)(.*?\n)(?=\S+|\Z)', re.S) >> >> val = """ >> eth2 Link encap:Ethernet HWaddr xx >> inet addr: xx.xx.xx.xx Bcast:xx.xx.xx.xx Mask:xx.xx.xx.xx >> ... >> loLink encap:Local Loopback >>inet addr:127.0.0.1 Mask:255.0.0.0 >> """ >> matches = re.findall(pat, val) >> >> So - why doesn't python have this? is it something that simply was >> overlooked, or is there another method of doing the same thing with >> arbitrarily complex freeform records? >> >> thanks much.. >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/guido%40python.org > > > > > -- > --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP Post-History
We’ve made a small change to the PEP process which may affect readers of python-list and python-ideas, so I’d like to inform you of it. This change was made to PEP 1 and PEP 12. PEPs must have a Post-History header which records the dates at which the PEP is posted to mailing lists, in order to keep the general Python community in the loop as a PEP moves to final status. Until now, PEPs in development were supposed to be posted at least to python-dev and optionally to python-list[1]. This guideline predated the creation of the python-ideas mailing list. We’ve now changed this guideline so that Post-History will record the dates at which the PEP is posted to python-dev and optionally python-ideas. python-list is dropped from this requirement. python-dev is always the primary mailing list of record for Python development, and PEPs under development should be posted to python-dev as appropriate. python-ideas is the list for discussion of more speculative changes to Python, and it’s often where more complex PEPs, and even proto-PEPs are first raised and their concepts are hashed out. As such, it makes more sense to change the guideline to include python-ideas and/or python-dev. In the effort to keep the forums of record to a manageable number, python-list is dropped. If you have been watching for new PEPs to be posted to python-list, you are invited to follow either python-dev or python-ideas. Cheers, -Barry (on behalf of the Python development community) https://mail.python.org/mailman/listinfo/python-dev https://mail.python.org/mailman/listinfo/python-ideas Both python-dev and python-ideas are available via Gmane. [1] PEPs may have a Discussions-To header which changes the list of forums where the PEP is discussed. In that case, Post-History records the dates that the PEP is posted to those forums. See PEP 1 for details. signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] \G (match last position) regex operator non-existant in python?
* Guido van Rossum , 2017-10-27, 08:35: The "why" question is not very interesting -- it probably wasn't in PCRE and nobody was familiar with it when we moved off PCRE (maybe it wasn't even in Perl at the time -- it was ~15 years ago). Perl supports \G since v5.0, released in 1994. PCRE supports it since v4.0, released in 2003. -- Jakub Wilk ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] If aligned_alloc() is missing on your platform, please let us know.
On 27 October 2017 at 18:17, Stefan Krah wrote: > Victor wrote a patch and would like to avoid adding a (probably > unnecessary) > emulation function. I agree with that. > > So if any platform does not have some form of aligned_alloc(), please > speak up. > I think Victor's suggested strategy in https://bugs.python.org/issue18835#msg305122 of deferring emulation support to its own follow-up issue is a good one, since there are two distinct cases to consider: 1. CPython's own support for platforms where we don't have a native aligned memory allocation API to call is covered by PEP 11, so if all current buildbots still work, then it will be up to the folks interested in a platform that *doesn't* offer aligned allocations to provide both a suitable emulation and a buildbot to test it on. 2.While all of the CPython-provided memory allocators will implement the new slots, the folks implementing their own custom allocators will need a defined upgrade path in the "Porting" section of the What's New guide. For that, an explicit error on 3.7+ that says "Configured custom allocator doesn't implement aligned memory allocations" is likely going to be easier to debug than subtle issues with the way an implicit emulation layer interacts with the other memory allocator slots. To appropriately address 2, we need more info not about which platforms natively support aligned memory allocations, but rather from folks that actually are implementing their own custom allocators. It may be that making the low level cross-platform raw alligned allocators available as a public API (independently of the switchable allocator machinery) will be a more appropriate way of handling that case than providing an emulation layer in terms of the old slots. That is, the suggested approach for custom allocator developers would be: 1. Test against 3.7, get an error complaining the new slots aren't defined 2. Set the new slots to the CPython provided cross-platform abstraction (if appropriate), or else wrap that abstraction layer as needed, or else implement your own aligned allocation routines If the only reason for the custom allocator was to allow tracemalloc to track additional memory that wouldn't otherwise be handled through CPython's memory allocators, then consider switching to using PyTraceMalloc_Track()/PyTraceMalloc_Untrack() instead. Either way, that transition strategy discussion shouldn't block making the feature itself available - it will be a lot easier to discuss transition enablement if potentially affected folks can download 3.7.0a3 and tell us what actually breaks, rather than asking them to speculate about what they think *might* break. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
