Re: [Python-Dev] Remaining decisions on PEP 471 -- os.scandir()
I was going to stay out of this one... On 14Jul2014 10:25, Victor Stinner wrote: 2014-07-14 4:17 GMT+02:00 Nick Coghlan : Or the ever popular symlink to "." (or a directory higher in the tree). "." and ".." are explicitly ignored by os.listdir() an os.scandir(). I think os.walk() is a good source of inspiration here: call the flag "followlink" and default it to False. I also think followslinks should be spelt like os.walk, and also default to False. IMO the specific function os.walk() is not a good example. It includes symlinks to directories in the dirs list and then it does not follow symlink, I agree that is a bad mix. it is a recursive function and has a followlinks optional parameter (default: False). Which I think is desirable. Moreover, in 92% of cases, functions using os.listdir() and os.path.isdir() *follow* symlinks: https://mail.python.org/pipermail/python-dev/2014-July/135435.html Sigh. This is a historic artifact, a convenience, and a side effect of bring symlinks into UNIX in the first place. The objective was that symlinks should largely be transparent to users for naive operation. So the UNIX calls open/cd/listdir all follow symlinks so that things work transparently and a million C programs do not break. However, so do chmod/chgrp/chown, for the same reasons and with generally less desirable effects. Conversely, the find command, for example, does not follow symlinks and this is generally a good thing. "ls" is the same. Like os.walk, they are for inspecting stuff, and shouldn't indirect unless asked. I think following symlinks, especially for something like os.walk and os.scandir, should default to False. I DO NOT want to quietly wander to remote parts of the file space because someone has stuck a symlink somewhere unfortunate, lurking like a little bomb (or perhaps trapdoor, waiting to suck me down into an unexpected dark place). It is also slower to follow symlinks by default. I am also against flag parameters that default to True, on the whole; they are a failure of ergonomic design. Leaving off a flag should usually be like setting it to False. A missing flag is an "off" flag. For these reasons (and others I have not yet thought through:-) I am voting for a: followlinks=False optional parameter. If you want to follow links, it is hardly difficult. Cheers, Cameron Simpson Our job is to make the questions so painful that the only way to make the pain go away is by thinking.- Fred Friendly ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bytes path support
On 20Aug2014 16:04, Chris Barker - NOAA Federal wrote: but disallowing them in higher level > explicitly cross platform abstractions like pathlib. I think the trick here is that posix-using folks claim that filenames are just bytes, and indeed they can be passed around with a char*, so they seem to be. but you can't possible do anything other than pass them around if you REALLY think they are just bytes. So really, people treat them as "bytes-in-some-arbitrary-encoding-where-at-least the-slash-character-(and maybe a couple others)-is-ascii-compatible" As someone who fought long and hard in the surrogate-escape listdir() wars, and was won over once the scheme was thoroughly explained to me, I take issue with these assertions: they are bogus or misleading. Firstly, POSIX filenames _are_ just byte strings. The only forbidden character is the NUL byte, which terminates a C string, and the only special character is the slash, which separates pathanme components. Second, a bare low level program cannot do _much_ more than pass them around. It certainly can do things like compute their basename, or other path related operations. The "bytes in some arbitrary encoding where at least the slash character (and maybe a couple others) is ascii compatible" notion is completely bogus. There's only one special byte, the slash (code 47). There's no OS-level need that it or anything else be ASCII compatible. I think characterisations such as the one quoted are activately misleading. The way you get UTF-8 (or some other encoding, fortunately getting less and less common) is by convention: you decide in your environment to work in some encoding (say utf-8) via the locale variables, and all your user-facing text gets used in UTF-8 encoding form when turned into bytes for the filename calls because your text<->bytes methods say to do so. I think we'd all agree it is nice to have a system where filenames are all Unicode, but since POSIX/UNIX predates it by decades it is a bit late to ignore the reality for such systems. I certainly think the Window-side Babel of code pages and multiple code systems is far far worse. (Disclaimer: not a Windows programmer, just based on hearing them complain.) I'm +1000 on systems where the filesystem enforces Unicode (eg Plan 9 or Mac OSX, which forces a specific UTF-8 encoding in the bytes POSIX APIs - the underlying filesystems reject invalid byte sequences). [...] Antoine Pitrou wrote: To elaborate specifically about pathlib, it doesn't handle bytes paths but allows you to generate them if desired: https://docs.python.org/3/library/pathlib.html#operators but that uses os.fsencode: Encode filename to the filesystem encoding As I understand it, the whole problem with some posix systems is that there is NO filesystem encoding -- i.e. you can't know for sure what encoding a filename is in. So you need to be able to pass the bytes through as they are. Yes and no. I made that argument too. There's no _external_ "filesystem encoding" in the sense of something recorded in the filesystem that anyone can inspect. But there is the expressed locale settings, available at runtime to any program that cares to pay attention. It is a workable situation. Oh, and I reject Nick's characterisation of POSIX as "broken". It's perfectly internally consistent. It just doesn't match what he wants. (Indeed, what I want, and I'm a long time UNIX fanboy.) Cheers, Cameron Simpson God is real, unless declared integer. - Johan Montald, jo...@ingres.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bytes path support
On 21Aug2014 09:20, Antoine Pitrou wrote: Le 21/08/2014 00:52, Cameron Simpson a écrit : The "bytes in some arbitrary encoding where at least the slash character (and maybe a couple others) is ascii compatible" notion is completely bogus. There's only one special byte, the slash (code 47). There's no OS-level need that it or anything else be ASCII compatible. Of course there is. Try to split an UTF-16-encoded file path on the byte 47 and you'll get a lot of garbage. So, yes, POSIX implicitly mandates an ASCII-compatible encoding for file paths. [Rolls eyes.] Looking at the UTF-16 encoding, it looks like it also embeds NUL bytes for various codes below 32768. How are they handled? As remarked, codes 0 (NUL) and 47 (ASCII slash code) _are_ special to UNIX filename bytes strings. If you imagine you can embed bare UTF-16 freely even excluding code 47, I think one of us is missing something. That's not "ASCII compatible". That's "not all byte codes can be freely used without thought", and any multibyte coding will have to consider such things when embedding itself in another coding scheme. Cheers, Cameron Simpson Microsoft: Committed to putting the "backward" into "backward compatibility." ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sad status of Python 3.x buildbots
On 03Sep2014 00:13, Victor Stinner wrote: AMD64 Snow Leop 3.x: many tests are not reliable (stable) on this platform. For example, test_logging.test_race() sometimes fail with PermissionError(1, "Operation not permitted: '/tmp/test_logging-3-bjulw8iz.log'"). Another example, test_asyncio.test_stdin_broken_pipe() sometimes fail with an assertion error because BrokenPipeError was not raised. Do we still support this old version of Mac OS X? Released in 2009, it is 5 years old. Is upgrading to Maverick (10.9) free and possible on old Mac computers? I don't have access to this old OS. As a negative data point, my GF runs Snow Leopard by choice and will not upgrade that machine; we've both got Mavericks laptops and there are major regressions in the UI and OS behaviour (Apple UI and apps, not Python). I would imagine she's not alone in resisting change. Cheers, Cameron Simpson Life IS pain, highness... anyone who tries to tell you different is trying to sell you something. - Wesley, The_Princess_Bride ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sad status of Python 3.x buildbots
On 03Sep2014 11:47, Stephen J. Turnbull wrote: Nick Coghlan writes: > Sorry, I haven't been a very good maintainer for that buildbot (the main > reason it never graduated to the "stable" list). If you send me your public > SSH key, I can add it (I think - if not, I can ask Luke to do it). > Alternatively, CentOS 6 may exhibit the same problem. I wonder how many of these buildbots could be maintained by the kind of folks who show up on core-mentorship asking "how can I help?" Just a thought -- I wouldn't be surprised if the reaction is universal horror and the answer is "Are you crazy? Zero! Z-E-R-O!!" And of course most want to write code, not sysadm. I do both. Happy to help in a small way if wanted. Cheers, Cameron Simpson Maintainer's Motto: If we can't fix it, it ain't broke. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX
On 26Sep2014 00:17, Antoine Pitrou wrote: On Thu, 25 Sep 2014 13:00:16 -0700 Bob Hanson wrote: Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX: <http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-6271> [...] Fortunately, Python's subprocess has its `shell` argument default to False. However, `os.system` invokes the shell implicitly and is therefore a possible attack vector. Only if /bin/sh is bash :-) Not always the case, fortunately. Cheers, Cameron Simpson Death is life's way of telling you you've been fired. - R. Geis ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX
On 26Sep2014 09:40, Steven D'Aprano wrote:
On Fri, Sep 26, 2014 at 12:17:46AM +0200, Antoine Pitrou wrote:
On Thu, 25 Sep 2014 13:00:16 -0700
Bob Hanson wrote:
> Critical bash vulnerability CVE-2014-6271 may affect Python on
> *n*x and OSX:
[...]
See also:
http://adminlogs.info/2014/09/25/again-bash-cve-2014-7169/
Fortunately, Python's subprocess has its `shell` argument default to
False. However, `os.system` invokes the shell implicitly and is
therefore a possible attack vector.
Perhaps I'm missing something, but aren't there easier ways to attack
os.system than the bash env vulnerability? If I'm accepting and running
arbitrary strings from an untrusted user, there's no need for them to go
to the trouble of feeding me:
"env x='() { :;}; echo gotcha' bash -c 'echo do something useful'"
when they can just feed me:
"echo gotcha"
In other words, os.system is *already* an attack vector, unless you only
use it with trusted strings. I don't think the bash env vulnerability
adds to the attack surface.
Have I missed something?
Yes. Although it is possible to craft safe things for os.system, the issue is
delivery of the attacks: plenty of totally standard things expose the shell to
outside-supplied strings. And on most of those things, "the shell" is bash.
The issue with the bash-imports-functions-badly issue is that shell functions
are exported through the environment. Other things use the environment to pass
info.
The loud examples on the net are CGI scripts (query parameters passed through
the environment, along with other things) and hostile DHCP servers (DHCP
settings passed by the _client_ dhcpd to action scripts, as root).
Both of these let someone outside your system deliver bash-exploit strings to
bash scripts.
Your cable/adsl modem? Probably an embedded Linux box, possibly using bash, and
certainly a dhcp client of the ISP. Better still, for many people that same
comprimisable modem is the DHCP _server_ for their home LAN...
Cheers,
Cameron Simpson
Rebel without a clue, Born to be mild.
- kevin.lo...@usask.ca, DoD #0975
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX
On 25Sep2014 21:30, Tres Seaver wrote: On 09/25/2014 08:59 PM, Cameron Simpson wrote: Your cable/adsl modem? Probably an embedded Linux box, possibly using bash, and certainly a dhcp client of the ISP. Better still, for many people that same comprimisable modem is the DHCP _server_ for their home LAN... Generally those devices are *not* using bash as '/bin/sh': it is too heavyweigh. Most will use 'busybox' or some other Swiss-army command for stuff which is separate commands on a "normal" linux system. Fair point. Cheers, Cameron Simpson There is no reason anyone would want a computer in their home. --Ken Olson, president, chairman and founder of Digital Equipment Corp., 1977 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX
On 26Sep2014 13:16, Antoine Pitrou wrote: On Fri, 26 Sep 2014 01:10:53 -0700 Hasan Diwan wrote: On 26 September 2014 00:28, Matěj Cepl wrote: > Where does your faith that other /bin/sh implementations (dash, > busybox, etc.) are less buggy comes from? The fact that they are simpler, in terms of lines of code. It's no guarantee, but the less a given piece of code does, the less bugs it will have. -- H And that they have less "features" (which is certainly correlated to their simplicity). IIUC, the misimplemented feature leading to this vulnerability is a bash-ism. IIRC you could export functions in ksh. Or maybe only aliases. But that implies most POSIX shells may support it. I've never seen the point myself; it is not a feature I've ever needed. Cheers, Cameron Simpson Follow! But! Follow only if ye be men of valor, for the entrance to this cave is guarded by a creature so foul, so cruel that no man yet has fought with it and lived! Bones of four fifty men lie strewn about its lair. So, brave knights, if you do doubt your courage or your strength, come no further, for death awaits you all with nasty big pointy teeth. - Tim The Enchanter ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Improvements for Pathlib
On 08Nov2014 17:46, Ionel Cristian Mărieș wrote: #2. A context manager for changing working directory (that switches to the old cwd on exit). Eg: with path.cd(): assert os.getcwd() == path As elsewhere remarked, the current working directory is process global state. Changing it can easily have unwanted (and invisible until breakage becomes glaring) side effects. -1 on this element from me I'm afraid. Cheers, Cameron Simpson I couldn't think of anything else to do with it, so I put it on the web. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] WebM MIME type in mimetypes module
On 02Dec2014 21:16, Terry Reedy wrote: On 12/2/2014 7:07 PM, Chris Rebert wrote: To summarize the issue, it proposes adding an entry for WebM ( http://www.webmproject.org/docs/container/#naming ) to the mimetypes standard library module's file-extension to MIME-type database. (Specifically: .webm => video/webm ) [...] If it has remained a defacto standard for the two years since your made that list, that would be a point in favor of recognizing it. Have .webm files become more common in actual use? Subjectively I've seen a few more about that I think I used to. And there are definitely some .webm files on some websites I support. Can't say if they're more common in terms of hard data though. But if most browsers expect them, arguably we should recognise their existence. Usual disclaimer: I am not a python-dev. Cheers, Cameron Simpson The nice thing about standards is that you have so many to choose from; furthermore, if you do not like any of them, you can just wait for next year's model. - Andrew S. Tanenbaum ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Hi, I am new to this board and have a question
On 04Feb2015 23:18, Jianhua Zhou wrote: Hi Everyone, I am a core software engineer at Rocket Software Inc. I am working on database system called UniData and Universe. Now we plan to introduce Python as the new programming language to our customer. When I try to build the python 3.4.1 on Red Hat Linux platform. I found some problem and need help. After I run configure and make the python, at the very end, it said some optional modules we missing, Python build finished successfully! The necessary bits to build these optional modules were not found: _bz2 _lzma _ssl _tkinter zlib So, I went to python development guide site and found out I have to download some other source to build the optional modules. Since I am on RH Linux and yum is installed on my Linux box, so I run following command # yum install yum-utils [...] Package yum-utils-1.1.30-14.el6.noarch already installed and latest version Looks like yum-utils is already installed. Then # yum-builddep python3 Loaded plugins: product-id, refresh-packagekit No such package(s): python3 Redhat don't supply python 3 on RedHat 5 or 6. I don't have access to a RHEL7 box, but if they did and you're using RHEL7 you could just "yum install" it and not bother with building anything. If you're building from source you do not need to involve RPM or yum at all. Unless you intend to ship python3 RPMs to your customers. Do you? Otherwise you can just: - install the relevant library and -devel modules from RedHat, at a guess from a RHEL6 host: _bz2: bzip2-devel bzip2-libs _tkinter: tkinter tk-devel _lzma: xz-devel xz-libs xz-lzma-compat zlib: zlib zlib-devel _ssl: openssl openssl-devel - rebuild from source as before and see what it says (configure, make, etc) Cheers, Cameron Simpson ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 471 (scandir): Add a new DirEntry.inode() method?
On 14Feb2015 11:35, Stephen J. Turnbull wrote: Victor Stinner writes: > *** Now the real question: is it useful to know the inode number > (st_ino) without the device number (st_dev)? *** > > On POSIX, you can still get the st_dev from DirEntry.stat(), but it > always require a system call. So you loose the whole purpose of > DirEntry (no extra syscall). True, but I suppose in many cases the user will know that all file system objects handled are on the same device, or will be willing to risk an occasional anomoly. In POSIX, all filsystem objects named by a directory are on the same device unless one is a mount point. (And in that case, d_ino from stat won't match d_ino from scandir; I expect.) IMO: Document the limitation (if no extra syscall) or inefficiency (with the syscall), and let the user choose. +1 on .inode(): d_ino has been available in the directory data on POSIX since at least V7 UNIX (1970s), almost certainly earlier. Agree the limitation should be mentioned. The remaining issue is whether to provide a convenience function for the device number, with appropriately loud warnings about how inefficient it is, or to deter the user with the need to call .stat() and extract the device number. -1 on that. People will use it! Given the doco above, it should be obvious under what circumstances one might choose to call stat, and making that stat overt means it is less likely to be called unwisely. Since scandir is all about efficiency, providing a very costly convenience function seems to go against the grain. Regarding usefulness: Victor, you've got the typical use case in another post (i.e. useful as in "advantageous"), and your own tests show that st_dev of the dir matches st_dev of a dir's entries in all normal/regular filesystems (i.e. useful as in "meaningful/consistent"). Special filesystems like /dev may be weird, but people relying on this should be aware of the constraint anyway. Since a directory at the low level is essentially a mapping of names to inodes within the directory's filesystem, this is to be expected. Cheers, Cameron Simpson Uh, this is only temporary...unless it works. - Red Green ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Use ptyhon -s as default shbang for system python executables/daemons
On 19Mar2015 19:57, Sturla Molden wrote: Orion Poplawski wrote: It would be good to get some feedback from the broader python community before implementing anything, so I'm asking for feedback here. On my systems I have /use/bin/python for the system and ~/anaconda/python for me. Apple and Ubuntu can do whatever they want with their Python's, I am not touching them. The main thing is to keep the system Python and the user Python separate. That is, both the executable and the folder where packages are installed. Whatever I install for myself can fuck up ~/anaconda, but is not allowed to mess with the system files or system folders. Me too. I keep my default virtualenvs in ~/var/venv, and have personal "py2" and "py3" scripts that invoke via the venvs. Having a similar separation in the system would be a good thing, for the same reasons. Also, it would let the OS supplier keep a _much_ smaller package/addon list for the "core" admin python instance, making QA etc easier and more reliable. Popular OSes let the local sysadmin (== "the user") pull in all sorts of stuff to the "system" Python, even from the supplier's repositories. Having a walled off "core" admin python as well seems very prudent. Cheers, Cameron Simpson conclude that this language as a tool is an open invitation for clever tricks; and while exactly this may be the explanation for some of its appeal, /viz./ to those who like to show how clever they are. I am sorry, but I must regard this as one of the most damning things that can be said about a programming language. - Edsger Dijkstra, _The Humble Programmer_ (CACM, Ocotber, 1972) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Use ptyhon -s as default shbang for system python executables/daemons
On 21Mar2015 14:29, Donald Stufft wrote: On Mar 21, 2015, at 7:52 AM, Nick Coghlan wrote: On 19 March 2015 at 07:51, Donald Stufft wrote: I’ve long wished that the OS had it’s own virtual environment. A lot of problems seems to come from trying to cram the things the OS wants with the things that the user wants into the same namespace. I'm more wanting to go in the other direction and suggest to folks that if they're not *scripting the OS*, then the system Python isn't what they want, and they should be using at least a virtual environment, preferably an entirely separate software collection that they can upgrade on their own timeline, independently of what they system does. It’s likely easier to get the OS to move it’s own things to a virtual environment than it is to convince every single person who uses an OS to never install globally. I agree. And just as a data point, this cropped up on the Fedora list yesterday: I broke Yum (by messing with Python libs) http://www.spinics.net/linux/fedora/fedora-users/msg458069.html TL;DR: OP used pip on his system python. Yum broke. Probably hampered his attempts to repair, too. Cheers, Cameron Simpson The mind reigns, but does not govern. - Paul Valery ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Use ptyhon -s as default shbang for system python executables/daemons
On 23Mar2015 21:58, Gregory P. Smith wrote: While people sometimes suggest virtualenv as a solution for this. It isn't really the same thing. It isn't a hermetic clone of the original interpreter. It copies the main binary but symlinks back to much of the stdlib. Oh. I had thought a non-standalone venv arranged sys.path to fall back to the source interpreter. Clearly I have not paid attention. Cheers, Cameron Simpson Yes, sometimes Perl looks like line-noise to the uninitiated, but to the seasoned Perl programmer, it looks like checksummed line-noise with a mission in life.- The Llama Book ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tracker reviews look like spam
On 12May2015 22:15, David Wilson wrote:
SPF only covers the envelope sender, so it should be possible to set
that to something that validates with SPF, keep the RFC822 From: header
as it is, and maybe(?) include a separate Sender: header matching the
envelope address.
David
Indeed. That sounds sane to me too. Google's complaint is SPF specific, so
hopefully that is the criterion for the spam rating.
It looks like bugs.python.org does not have an SPF record at all - neither SPF
not TXT. (You really want both, same format, to support older DNS clients).
I'm not sure you need a Sender: (though it wouldn't hurt), given that the From:
is already a "system" like address (""Terry Reedy " as a mailing list would do.
Cheers,
Cameron Simpson
On Tue, May 12, 2015 at 06:08:30PM -0400, Terry Reedy wrote:
Gmail dumps patch review email in my junk box. The problem seems to be the
spoofed From: header.
Received: from psf.upfronthosting.co.za ([2a01:4f8:131:2480::3])
by mx.google.com with ESMTP id
m1si26039166wjy.52.2015.05.12.00.20.38
for ;
Tue, 12 May 2015 00:20:38 -0700 (PDT)
Received-SPF: softfail (google.com: domain of transitioning
storch...@gmail.com does not designate 2a01:4f8:131:2480::3 as permitted
sender) client-ip=2a01:4f8:131:2480::3;
Tracker reviews are the only false positives in my junk list. Otherwise, I
might stop reviewing. Verizon does not even deliver mail that fails its junk
test, so I would not be surprised if there are people who simply do not get
emailed reviews.
Tracker posts are sent from Person Name
Perhaps reviews could come 'from' Person Name
Even direct tracker posts just get a neutral score.
Received-SPF: neutral (google.com: 2a01:4f8:131:2480::3 is neither permitted
nor denied by best guess record for domain of
roundup-ad...@psf.upfronthosting.co.za) client-ip=2a01:4f8:131:2480::3;
SPF is Sender Policy Framework
https://en.wikipedia.org/wiki/Sender_Policy_Framework
Checkins mail, for instance, gets an SPF 'pass' because python.org
designates mail.python.org as a permitted sender.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/cs%40zip.com.au
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The current dict is not an "OrderedDict"
On 08Nov2017 10:28, Antoine Pitrou wrote: On Wed, 8 Nov 2017 13:07:12 +1000 Nick Coghlan wrote: On 8 November 2017 at 07:19, Evpok Padding wrote: > On 7 November 2017 at 21:47, Chris Barker wrote: >> if dict order is preserved in cPython , people WILL count on it! > > I won't, and if people do and their code break, they'll have only themselves > to blame. > Also, what proof do you have of that besides anecdotal evidence ? ~27 calendar years of anecdotal evidence across a multitude of CPython API behaviours (as well as API usage in other projects). Other implementation developers don't say "CPython's runtime behaviour is the real Python specification" for the fun of it - they say it because "my code works on CPython, but it does the wrong thing on your interpreter, so I'm going to stick with CPython" is a real barrier to end user adoption, no matter what the language specification says. Yet, PyPy has no reference counting, and it doesn't seem to be a cause of concern. Broken code is fixed along the way, when people notice. I'd expect that this may be because that would merely to cause temporary memory leakage or differently timed running of __del__ actions. Neither of which normally affects semantics critical to the end result of most programs. However, code which relies on an ordering effect which works in the usual case but (often subtly) breaks in some unusual case can be hard to debug, because (a) recognising the salient error situation may be hard to do and (b) reasoning about the failure is difficult when the language semantics are not what you thought they were. I think the two situations are not as parallel as you think. Cheers, Cameron Simpson (formerly c...@zip.com.au) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] assignment expressions: an alternative proposal
On 24Apr2018 08:51, Ethan Furman wrote: When I compare to variables from outer scopes they *usually* are on the *right* side of '=='. You mean something like if 2 == x: ? I never write code like that, and I haven't seen it, either. Just to this, I also never write code like that but I've certainly seen it advocated. I think the rationale was that it places the comparison value foremost in one's mind, versus the name being tested. I'm not persuaded, but it is another subjective situation. Cheers, Cameron Simpson ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 498 f-string: please remove the special case for spaces
On 11Aug2015 01:00, Victor Stinner wrote: By the way, I don't think that fu'...' syntax should be allowed. IMHO u'...' was only reintroduced to Python 3.3 to ease transition from Python 2 to Python 3 of the existing u'...' Syntax. Since f'...' is a new syntax, backward compatibility doesn't matter here. There's another reason to resist the fu'...' prefix: political correctness. To illustrate, there's a consumer rights TV snow here with a segment called "F.U. Tube", where members of the public describe ripoffs and other product failures in video form. While a phonetic play on the name "YouTube", the abbreviation also colloquially means just what you think it might. I can just imagine reciting one of these new strings out loud... Cheers, Cameron Simpson People shouldn't be allowed to build overpasses ... - Dianne "I know what's best for you" Feinstein after the '94 LA quake. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 498 f-string: please remove the special case for spaces
On 11Aug2015 18:07, Greg Ewing wrote: Cameron Simpson wrote: To illustrate, there's a consumer rights TV snow here with a segment called "F.U. Tube", where members of the public describe ripoffs and other product failures in video form. While a phonetic play on the name "YouTube", the abbreviation also colloquially means just what you think it might. I can just imagine reciting one of these new strings out loud... We could require it to be spelled "uf" unless "from __future__ import billy_connolly_as_FLUFL" is in effect. That seems like a reasoned and measured response. Cheers, Cameron Simpson For those who understand, NO explanation is needed, for those who don't understand, NO explanation will be given! - Davey D ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bitwise operations for bytes and bytearray
On 07Jan2016 16:12, Python-Dev wrote: On Jan 7, 2016, at 15:57, Martin Panter wrote: On 7 January 2016 at 22:26, Blake Griffith wrote: I'm interested in adding the functionality to do something like: b'a' ^ b'b' b'\x03' Instead of the good ol' TypeError. I think both bytes and bytearray should support all the bitwise operations. There is a bug open about adding this kind of functionality: <https://bugs.python.org/issue19251>. And it's in the needs patch stage, which makes it perfect for the OP: in addition to learning how to hack on builtin types, he can also learn the other parts of the dev process. (Even if the bug is eventually rejected, as seems likely given that it sat around for three years with no compelling use case and then Guido added a "very skeptical" comment.) The use case which springs immediately to my mind is cryptography. To encrypt a stream symmetrically you can go: cleartext-bytes ^ cryptographicly-random-bytes-from-cipher so with this one could write: def crypted(byteses, crypto_source): ''' Accept an iterable source of bytes objects and a preprimed source of crypto bytes, yield encrypted versions of the bytes objects. ''' for bs in byteses: cbs = crypto_source.next_bytes(len(bs)) yield bs ^ cbs Cheers, Cameron Simpson ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unicode Exception messages in py2.7
On 14Nov2013 15:57, Chris Barker - NOAA Federal wrote: > (amazing to me how many people are still using <=2.7, actually, even > for new projects .. thank you Red Hat "Enterprise" Linux ;-) ) Well, one of the things RHEL gets you is platform stability (they backport fixes; primarily security in the older RHEL streams). So of course the Python dates to the time of the release. I install a current Python 2.7 into /usr/local on many RHEL boxes and target that for custom code. -- Cameron Simpson There is this special biologist word we use for 'stable'. It is 'dead'. - Jack Cohen ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unicode Exception messages in py2.7
On 15Nov2013 14:08, Steven D'Aprano wrote: > On Thu, Nov 14, 2013 at 04:02:17PM -0800, Chris Barker wrote: > > right -- any bugfix changes behaviour > > It isn't clear that this is a bug at all. > > Non-ascii Unicode strings are just a special case of the more general > problem of what to do if printing the exception raises. If > str(exception.message) raises, suppressing the message seems like a > perfectly reasonable approach to me. Not to me. Silent failure is really nasty. In fact, doesn't the Zen speak explicitly against it? I'm debugging a program right now with silent failures; my own code, with functions submitted to a queue for asynchronous execution, and the queue preserves the function result (or exception) for collection later; if that collection doesn't happen you get... silent failure! I think that if an exception escapes to the outside for reporting, if the reporting raises an exception (especially an "expectable" one like unicode coding/decoding errors), the reporting should have at least a layer of "ouch, report failed, try something uglier but more conservative". At least you'd know there had been a failure. Cheers, -- Cameron Simpson Windows is really user friendly - it doesn't crash on its own, it first opens a dialog box, saying it will crash and you have to click OK :-) - Zoltan Kocsi ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Running the unit test as root/administrator
On 03Dec2013 08:25, Guido van Rossum wrote: > I would be rather worried about some accidental Trojen running that way. Or even just a badly framed clean-up-temp-files step. -- Cameron Simpson manual, n.: A unit of documentation. There are always three or more on a given item. One is on the shelf; someone has the others. The information you need is in the others. - Ray Simard ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 460: Add bytes % args and bytes.format(args) to Python 3.5
On 11Jan2014 00:43, Juraj Sukop wrote:
> On Fri, Jan 10, 2014 at 11:12 PM, Victor Stinner
> wrote:
> > What not building "10 0 obj ... stream" and "endstream endobj" in
> > Unicode and then encode to ASCII? Example:
> >
> > data = b''.join((
> > ("%d %d obj ... stream" % (10, 0)).encode('ascii'),
> > binary_image_data,
> > ("endstream endobj").encode('ascii'),
> > ))
>
> The key is "encode to ASCII" which means that the result is bytes. Then,
> there is this "11 0 obj" which should also be bytes. But it has no
> "binary_image_data" - only lots of numbers waiting to be somehow converted
> to bytes. I already mentioned the problems with ".encode('ascii')" but it
> does not stop here. Numbers may appear not only inside "streams" but almost
> anywhere: in the header there is PDF version, an image has to have "width"
> and "height", at the end of PDF there is a structure containing offsets to
> all of the objects in file. Basically, to ".encode('ascii')" every possible
> number is not exactly simple or pretty.
Hi Juraj,
Might I suggest a helper function (outside the PEP scope) instead
of arguing for support for %f et al?
Thus:
def bytify(things, encoding='ascii'):
for thing:
if isinstance(thing, bytes):
yield thing
else:
yield str(thing).encode('ascii')
Then one's embedding in PDF might become, more readably:
data = b' '.join( bytify( [ 10, 0, obj, binary_image_data, ... ] ) )
Of course, bytify might be augmented with whatever encoding facilities
might suit your needs.
Cheers,
--
Cameron Simpson
We tend to overestimate the short-term impact of technological change and
underestimate its long-term impact. - Amara's Law
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 460: Add bytes % args and bytes.format(args) to Python 3.5
On 11Jan2014 13:15, Juraj Sukop wrote:
> On Sat, Jan 11, 2014 at 5:14 AM, Cameron Simpson wrote:
> > data = b' '.join( bytify( [ 10, 0, obj, binary_image_data, ... ] ) )
>
> Thanks for the suggestion! The problem with "bytify" is that some items
> might require different formatting than other items. For example, in
> "Cross-Reference Table" there are three different formats: non-padded
> integer ("1"), 10- and 15digit integer, ("03", "65535").
Well, this is partly my point: you probably want to exert more
control that is reasonable for the PEP to offer, and you're better
off with a helper function of your own. In particular, aside from
passing in a default char=>bytes encoding, you can provide your own
format hooks.
In particular, str already provides a completish % suite and you
have no issue with encodings in that phase because it is all Unicode.
So the points where you're treating PDF as text are probably best
tackled as text and then encoded with a helper like bytify when you
have to glom bytes and "textish" stuff together.
Crude example, hacked up from yours:
data = b''.join( bytify(
("%d %d obj ... stream" % (10, 0)),
binary_image_data,
"endstream endobj",
)))
where bytify swallows your encoding decisions.
Since encoding anything-not-bytes into a bytes sequence inherently
involves an encoding decision, I think I'm +1 on the PEP's aim of
never mixing bytes with non-bytes, keeping all the encoding decisions
in the caller's hands.
I quite understand not wanting to belabour the code with
".encode('ascii')" but that should be said somewhere, so best to
do so yourself in as compact and ergonomic fashion as possible.
Cheers,
--
Cameron Simpson
Serious error.
All shortcuts have disappeared.
Screen. Mind. Both are blank.
- Haiku Error Messages
http://www.salonmagazine.com/21st/chal/1998/02/10chal2.html
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Trying to focus the whole bytes/str formatting discussion
On 12Jan2014 17:46, Brett Cannon wrote: > THE EIBTI group are willing to support PEP 460 but beyond that don't want > to have in Python itself anything for bytes.format() which takes in a > string and spits out bytes. It's bytes in->bytes out and not bytes & str > in->bytes out as the PBP group is after. The EIBTI group are arguing that > letting str into bytes.format() and then automatically be converted to > strict ASCII leads to conflating the text/bytes divide as well as being too > magical, e.g. what if you actually wanted UTF-16 for you number string > instead of ASCII; the EIBTI group **wants** to force people to make a > decision. They are also less concerned with making users update Python 2 > code to handle this as it already needs to be updated for other Python 3 > things anyway. [...] I'm in the EIBTI on the whole, but I would also be happy for the bytes.format() function to accept strings (and floats or whatever the str.format supports) _provided_ it required an explicit encoding= parameter to enable it. i.e. make it easy to use, _but_ require an overt specification of the str->bytes encoding. You don't even need a special mode, but have it raise a ValueError if the (default) encoding is None when an encoding became needed. Just my 2c on Brett's EIBTI vs PBP divide. I'll try to stay off this thread now and bikeshed only in the others... -- Cameron Simpson You can blip it twice to clear the bore, But blip it thrice, and you've sinned once more. - Tom Warner ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 460 reboot
On 14Jan2014 11:43, Jim Jewett wrote: > Greg Ewing replied: > >> ... ASCII compatible binary data is a > >> *subset* of arbitrary binary data. > > I wrote: [...] > >(2) It *may* be worth creating a virtual > > split in the documentation. [...] > > Ethan likes the idea, but points out that the term > "Virtual" is confusing here. [...] > (A) What word should I use instead of "Virtual"? > Imaginary? Pretend? I'd title it in terms of a common use case, not a "virtual class". You even phrase the opening sentence as a use case already. > (B) Would it be good/bad/at least make the docs > easier to create an actual class (or alias)? > (C) Same question for a pair of classes provided > only in the documentation, like example code. I don't think so. People might use it:-( [...] > > A Bytes object could represent anything, [...] Tiny nit: shouldn't that be "bytes", not "Bytes"? > > appropriate as the underlying storage for a sound sample > > or image file. > > > > Virtual subclass ASCIIStructuredBytes > > ==== Possible alternate title: Common use case: bytes containing text sequences, especially ASCII Cheers, -- Cameron Simpson I think... Therefore I ride. I ride... Therefore I am. - Mark Pope ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 460 reboot
On 14Jan2014 20:23, Antoine Pitrou wrote: > On Tue, 14 Jan 2014 10:52:05 -0800 > Guido van Rossum wrote: > > Quite a few people have spoken out in favor of loud > > failures rather than silent "wrong" output. But I think that in the > > specific context of formatting output, there is a long and IMO good > > tradition of producing (slightly) wrong output in favor of more > > strict behavior. Consider for example what to do when a number > > doesn't fit in the given width. Would you rather raise an exception, > > truncate the > > value, or mess up the formatting? All languages newer than Fortran > > that I've used have chosen the latter, and I still agree it's a good > > idea. > > Well that's useful when printing out human-readable stuff on stdout, > much less when you're emitting binary data that's supposed to conform > to a well-defined protocol. I expect bytes formatting to be used for > the latter, not the former. I'm 12 hours behind in this thread still, but I'm with Antoine here. With protocols, there's a long and IMO good tradition in the RFC world of being generous in what you accept and conservative in what you send, and writing bytes data constitutes "send" to my mind. While having numbers overflow their widths is (only) often ok for human reports, even that is a PITA for machine parsing later. By way of a text example, my personal bugbear is the UNIX "ps" command in its many flavours. It has fixed width columns with fields that frequently overflow these days, and the overflowing numbers abut each other. Post processing this rubbish is a disaster (I don't want to write "ps", but I have written things that want to read its output). Of course the fix is easy in some ways, use format strings saying "%-5d %-5d %-5d" instead of "%-6d%-6d%-6d". But the authors of ps didn't. And quietly overflowing these fields is exactly what breaks my post processing programs. Morally, this is the same as mojibake. Therefore I am firmly in the "fail loudly" camp: if the format string doesn't behave as you naively expected it to, find out early while you can easily fix it. Cheers, -- Cameron Simpson Motorcycles are like peanuts... who can stop at just one? - Zebee Johnstone aus.motorcycles Poser Permit #1 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] News from asyncio
On 27Jan2014 09:15, Devin Jeanpierre wrote: > On Mon, Jan 27, 2014 at 5:21 AM, Victor Stinner > wrote: > > - asyncio.IncompleReadError.expected is the total expected size, not > > the remaining size > > Why not be consistent with the meaning of > http.client.IncompleteRead.expected? The current meaning can be > recovered via len(e.partial) + e.expected. Which you could also expose as a property name ".requested", since this value must have been thought useful or Victor wouldn't have defined .expected to be it:-) Seems reasonable, for all that the above computation is trivial. Just a thought, -- Cameron Simpson Artificial intelligence won't make a micrometer out of a monkeywrench. - Rick Gordon ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
On 22Feb2014 17:56, Ethan Furman wrote: > Please let me know if anything else needs tweaking. > [...] > This area of programming is characterized by a mixture of binary data and > ASCII compatible segments of text (aka ASCII-encoded text). > [...] > %-interpolation > > All the numeric formatting codes (such as ``%x``, ``%o``, ``%e``, ``%f``, > ``%g``, etc.) will be supported, and will work as they do for str, including > the padding, justification and other related modifiers. I would like a single sentence here clarifying that the formatting of numeric values uses an ASCII encoding. It might be inferred from the earlier context, but I do not think it can be deduced and therefore I think it should be said outright. All the other formatting codes are quite explicit about how their arguments transform into bytes, but the numeric codes just quietly assume ASCII. The PEP should be blatant. Otherwise I think the PEP is clear and reasonable. Cheers, -- Cameron Simpson ASCII n s. [from the greek] Those people who, at certain times of the year, have no shadow at noon; such are the inhabitatants of the torrid zone. - 1837 copy of Johnson's Dictionary ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
On 23Feb2014 16:31, Nick Coghlan wrote:
> On 23 February 2014 13:47, Cameron Simpson wrote:
> > On 22Feb2014 17:56, Ethan Furman wrote:
> >> Please let me know if anything else needs tweaking.
> >> [...]
> >> This area of programming is characterized by a mixture of binary data and
> >> ASCII compatible segments of text (aka ASCII-encoded text).
> >> [...]
> >> %-interpolation
> >>
> >> All the numeric formatting codes (such as ``%x``, ``%o``, ``%e``, ``%f``,
> >> ``%g``, etc.) will be supported, and will work as they do for str,
> >> including
> >> the padding, justification and other related modifiers.
> >
> > I would like a single sentence here clarifying that the formatting
> > of numeric values uses an ASCII encoding.
> >
> > It might be inferred from the earlier context, but I do not think
> > it can be deduced and therefore I think it should be said outright.
> > All the other formatting codes are quite explicit about how their
> > arguments transform into bytes, but the numeric codes just quietly
> > assume ASCII. The PEP should be blatant.
>
> Specifically, I believe the PEP should state that, for the numeric codes:
>
> b"%x" % val
>
> is equivalent to:
>
> b"%s" % (("%x" % val).encode("ascii"))
>
> The rationale for including them is the unreadability of the latter form :)
Hmm. Isn't:
("%x" % val).encode("ascii")
sufficient here?
I still think that the term ASCII should appear in the prose, rather
than forcing the reader to decode the above. Example, shoehorning
off Ethan's response:
The substituted bytes will be an ASCII encoding of the corresponding str
formatting codes. Specificaly, for any numeric formatting code "%x":
b"%x" % val
is equivalent to:
("%x" % val).encode("ascii")
That ticks my wishes and includes Nick's explicit algorithmic
expression of the process.
Cheers,
--
Cameron Simpson
Me, I'm looking for obituaries. Lately a gratifyingly large number of my
most odious near-contemporaries are achieving their long-deserved quietus.
Not enough, and not always the right ones, but their time will come.
Peeve: I may not live to see them dead.
- Lee Rudolph, rudo...@cis.umassd.edu
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
On 23Feb2014 12:30, Victor Stinner wrote: > > All the numeric formatting codes (such as ``%x``, ``%o``, ``%e``, ``%f``, > > ``%g``, etc.) will be supported, and will work as they do for str, including > > the padding, justification and other related modifiers. > > IMO you should give the exhaustive list here and we should only > support one formatter for integers: %d. Python 2 supports "%d", "%u" > and "%i" with "%u" marked as obsolete. Python 3.5 should not > reintroduce obsolete formatters. If you want to use the same code base > for Python 2.6, 2.7 and 3.5: modify your code to only use %d. Same > rule apply for 2to3 tool: modify your source code to be compatible > with Python 3. > Please also mention all flags: #, +, -, '0', ' '. Is this really necessary? Can't one just refer the the str %-formatting section of the doco? By section and title to make it easy to find. I think this should just refer the reader to the str %-formatting doco for the numeric codes and their meanings, along with the flags. Otherwise the PEP will get unreadable, to no value that I can see. If we include Nick's equivalent code example, there is no ambiguity or vagueness. I'm against restricting to just %d for int too; if the current Python supports others (eg %o, %x) for str, so should this PEP for bytes. > > ``%c`` will insert a single byte, either from an ``int`` in range(256), or > > from > > a ``bytes`` argument of length 1, not from a ``str``. > > I'm not sure that supporting bytes argument of 1 byte is useful, but > it should not be hard to implement and may be convinient. I'm +0.5 for a bytes argument of length 1; while bytes are arrays of small ints, just as str has no distinct "char" type a bytes has no distinct byte type. With a string we commonly use s str of length 1 to denote a single character in isolation; the same programming idioms will get you a bytes of length 1 in situations when you mean a byte. > (You forgot "/U" representation (it's an antislah, but I don't > see the key on my Mac keyboard?).) My Mac has one above the "return" key. Um, non-English locale? Curious. Cheers, -- Cameron Simpson 16 October. I also asked Anthea how many mature oaks she thought it would have taken to build a top-of-the-line ship in Nelson's day. She guessed ten. The astonishing answer (from Brewer's) is about 3,500 - 900 acres of oak forest. She said, "I wonder what we're doing now that's as wasteful as that". I said it's still called Defence. - Brian Eno, _A Year With Swollen Appendices_ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
On 23Feb2014 22:56, Victor Stinner wrote: > > An aid to debugging -- need to see what's what at that moment? Toss it > > into %a. It is not intended for production code, but is included to > > hopefully circumvent the inappropriate use of __bytes__ methods on classes. > > How do you plan to use this output? Write it into a socket or a file? > When I debug, I use print & logging which both expect text string. So I > think that b'%a' is useless. The case from the email thread, which I support at +0.5 or maybe only +0.1, is printing to a binary log. The classic example that comes to mind is syslog packets. I agree %a invites data mangling. One would hope it doesn't see use in wire protocols, only in debugging scenarios. Regrettably, syslog is such a binary logging protocol, purportedly for "text". Cheers, -- Cameron Simpson We had the experience, but missed the meaning. - T.S. Eliot ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 4: don't remove anything, don't break backward compatibility
On 10Mar2014 14:55, Victor Stinner wrote: > Last 5 years, I spend significant time to port a lot of Python 2 code > on Python 3. [... troubles ...] > So can we please try to stop scheduling another major Python version > breaking almost all modules and all applications just to be pendantic? > No, we should not remove any old feature in Python 4. Python 4 should > be just a minor release following the previous 3.x release. Maybe that will be the case, when it rolls around in the ordinary sequence of things. Assuming nothing groundbreaks shows up. > I don't want another six, nine or whatever module to fill the gap > between Python 3 and Python 4. But this I do not understand. If 4.0 is in your vision to be a regular release, why should you care that it may be years off? > For example, I propose to release the next major Python version (3.5) > with the version 4.0 but without removing anything. (It's just an > example, it can wait another release.) Just in case this is not a joke or hyperbole: I am -1 on this. Just stick with the expected numbering scheme. If there are no incompatible but desired changes queued up by the time 4.0, perhaps that will be a normal release also. If not, perhaps it will be a good time to bite the bullet. > I mean that any incompatible > change must following the classic transition plan: tag the feature as > deprecated and wait at least one major version before dropping it, or > just keep it forever. We can expect that just changing the major > version (3 => 4) will already break enough applications (where > "python3" or "version == 3" is hardcoded")... I tend to spell this >= 3, etc. Maybe I'm being simplistic. > Instead of asking application developers to try to follow each major > Python release, we should try to keep the maintaince pain in Python > core. > > What do you think? I agree there shouldn't be a major porting pain release just for the sake of a number change; anything like that should need justification. But conversely, I'm dead against bringing forward version 4.0 just to break the expectation of breakage. Cheers, -- Cameron Simpson The nice thing about standards is that you have so many to choose from; furthermore, if you do not like any of them, you can just wait for next year's model. - Andrew S. Tanenbaum ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods
On 20Apr2014 14:32, Mark Lawrence wrote: On 20/04/2014 06:31, Ethan Furman wrote: Thank you for taking the time to write this up, Nick. However, I am -1 on it. One of the allures of Python 3 is the increase in simplicity and elegance. Restoring cruft does not help with that. Python 2 idioms that get restored to Python 3 must have real value: unicode literals, wire-protocol interpolations -- I don't feel that this comes any where close to meeting that bar. ~Ethan~ +1 for this summary which echoes my sentiments entirely. Me too. I'm against iteritems and friends coming back. I've been burned in the past with the burden of writing a mapping class with the many methods such a thing must support; both items() and iteritems() and so forth. For the example I have in mind I eventually abandoned the objective of being a full mapping and am going back to just a few methods to support easy element access such as __getitem__ and __contains__. I have a small python module of my own to aid my python 2+3 efforts, and am of the opinion that it is better to add iteritems elper functions to a popular module like six than to left the noise back into the python 3 mapping interface. Cheers, Cameron Simpson ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods
On 20Apr2014 20:12, Devin Jeanpierre wrote: On Sun, Apr 20, 2014 at 8:01 PM, Cameron Simpson wrote: Me too. I'm against iteritems and friends coming back. I've been burned in the past with the burden of writing a mapping class with the many methods such a thing must support; both items() and iteritems() and so forth. [...] As far as I know, all you have to implement yourself, to support the dict-like interface, are: - __getitem__ - __setitem__ - __delitem__ - __iter__ - __len__ MutableMapping can implement the rest. This wouldn't change with the re-addition of the iter* methods. You really have to use MutableMapping now that keys/items/values are complicated objects: it's much harder to implement dict-like objects from scratch in 3.x than 2.x. Fair point. Thank you, Cameron Simpson Draw little boxes with arrows. It helps. - Michael J. Eager ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Where is our official policy of what platforms we do support?
On 14May2014 14:45, Brett Cannon wrote: On Wed May 14 2014 at 10:43:18 AM, Antoine Pitrou wrote: On Wed, 14 May 2014 14:20:26 + Brett Cannon wrote: > Over the past week or so there have been 2 patches to add support for > various UNIX OSs. Now I thought we had stopped trying to add new esoteric > OSs (e.g. I had never heard of MirOS until the patch for it came in), but I > can't find a PEP that spells out what it takes to get a platform supported ( > http://legacy.python.org/dev/peps/pep-0011/ is about removing platforms, > not keeping them or adding them unless you are re-adding one which > apparently just takes a volunteer). OTOH you can fix a platform bug without officially supporting it. If someone files an OpenBSD-specific patch, it may make sense to commit it even without officially supporting OpenBSD. In practice it all depends on how intrusive / reasonable the patch is, and whether it is working around a platform-specific bug rather than a standards-compliant limitation. (we could call those "stochastically supported platforms" :-)) Very true, but these patches are all for e.g. configure to recognize a specific platform by listing them in some constant. Changing code to be more general I have no issue with since that's just good practice. Recognition of a special platform isn't "full support", just addition of recognition making possible partial support for special cases. Unless that makes for some horrendous recognition decision tree somewhere I would have thought that's a pretty low bar to accept, and worth doing. Leaving aside any bug actually fixed, it makes it much easier for someone else to fix a platform specific bug by making a test constant available for turning on whatever special mode/code is wanted. More context on the example patch that triggered this query? Just 2c, Cameron Simpson ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Accessing mailing list archives
On 30Jul2018 13:40, Bob Purvy wrote: I've been trying to figure out how to access the archives programmatically. I'm sure this is easy once you know, but googling various things hasn't worked. What I want to do is graph the number of messages about PEP 572 by time. (or has someone already done that?) I installed GNU Mailman, and downloaded the gzip'ed archives for a number of months and unzipped them, and I suspect that there's some way to get them all into a single database, but it hasn't jumped out at me. If I count the "Message-ID" lines, the "Subject:" lines, and the "\nFrom " lines in one of those text files, I get slightly different numbers for each. Alternatively, they're maybe *already* in a database, and I just need API access to do the querying? Can someone help me out? Like Victor, I download mailing list archives. Between pulling them in and also subscribing, ideally I get a complete history in my "python" mail folder. Likewise for other lists. The mailman archives are UNIX mbox files, compressed, with a bit of header munging (to make address harvesting harder). You can concatenate them and uncompress and reverse the munging like this: cat *.gz | gunzip | fix-mail-dates --mbox | un-at- where fix-mail-dates is here: https://bitbucket.org/cameron_simpson/css/src/tip/bin/fix-mail-dates and un-at- is here: https://bitbucket.org/cameron_simpson/css/src/tip/bin/un-at- and the output is a nice UNIX mbox file. You can load that into most mail readers or parse it with Python's email modules (in the stdlib). It should be easy enough to scan such a thing and count header contents etc. Ignore the "From " line content, prefer the "From:" header. (Separate messages on "From " of course, just don't grab email addresses from it.) Cheers, Cameron Simpson ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove tempfile.mktemp()
On 20Mar2019 12:53, Jeroen Demeyer wrote: On 2019-03-20 12:45, Victor Stinner wrote: You can watch the /tmp directory using inotify and "discover" immediately the "secret" filename, it doesn't depend on the amount of entropy used to generate the filename. That's not the problem. The security issue here is guessing the filename *before* it's created and putting a different file or symlink in place. So I actually do think that mktemp() could be made secure by using a longer name generated by a secure random generator. I know it is days later, but to add a little nuance: the security issue is guessing the filename before it is _used_. Consider: path = tempfile.mktemp() with open(path, "w"): write some secret stuff ... call_other_function(path) If an attacker gets in _after_ the open (which creates the file) by using something like inotify to _observe_ the pathname instead of guessing and supplants the file then, call_other_function is then subverted. Also, the common examples are attackers who are not the user making the tempfile, in which case the _default_ mktemp is sort of secure with the above because it gets made in /tmp which on a modern POSIX system prevents _other_ uses from removing/renaming a file. (And Eryk I think described the Windows situation which is similarly protected). However, mktemp somewhere else is not so protected. And the attacker might be malware running as the orignal user (yes the game may already be overin that case for other reasons). However, I wanted to make the point that the security issue isn't around creation but use - trusting the mktemp pathname to be the same state as it was earlier. Cheers, Cameron Simpson ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] BDFL-Delegate appointments for several PEPs
On 24Mar2019 17:02, Terry Reedy wrote: On 3/24/2019 8:21 AM, Nick Coghlan wrote: * PEP 499: Binding "-m" executed modules under their module name as well as `__main__` My brief response: +1 unless there is a good reason not. There turn out to be some subtle side effects. The test suite turned up one (easily fixed) in pdb, but there are definitely some more things to investigate. Nick has pointed out pickle and the "python -i" option. I'm digging into these. (Naturally, I have _never_ before used the pdb or pickle modules, or the -i option :-) There have been multiple double module problems reported on python-list and likely stackoverflow. And would there be any impact on circular imports? Well, by binding the -m module to both __main__ and its name as denoted on the command line one circular import is directly short circuited. Aside from the -m module itself, I don't think there should be any other direct effect on circular imports. Did you have a specific scenario in mind? Cheers, Cameron Simpson ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] BDFL-Delegate appointments for several PEPs
On 24Mar2019 23:22, Terry Reedy wrote: On 3/24/2019 10:01 PM, Terry Reedy wrote: On 3/24/2019 7:00 PM, Cameron Simpson wrote: Did you have a specific scenario in mind? I was thinking about IDLE and its tangled web of circular inports, but I am now convinced that this change will not affect it. Indeed, idlelib/pyshell.py already implements idea of the proposal, ending with if __name__ == "__main__": sys.modules['pyshell'] = sys.modules['__main__'] main() After more investigation, I realized that to stop having duplicate modulue: 1. The alias should be 'idlelib.pyshell', not 'pyshell', at least when imports are all absolute. The PEP499 patch effectively uses __main__.__spec__.name for the name of the alias. Does that simplify your issue? The current PR is here if you want to look at it: https://github.com/python/cpython/pull/12490 2. It should be done at the top of the file, before the import of modules that import pyshell. Hmm, if PEP499 comes in you shouldn't need to do this at all. If PEP499 gets delayed or rejected I guess you're supporting this without it. Yes, you'll want to do it before any other imports happen (well, as you say, before any which import pyshell). What about (untested): if __name__ == '__main__': if __spec__.name not in sys.modules: sys.modules[__spec__.name] = sys.modules['__main__'] as a forward compatible setup? If I run python f:/dev/3x/lib/idlelib/pyshell.py, the PEP patch would have to notice that pyshell is a module within idlelib and alias '__main__' to 'idlelib.pyshell', not 'pyshell'. Would the same be true if within-package import were all relative? I think so because we're using .__spec__.name, which I though was post import name resolution. Testing in my PEP499 branch: Test 1: [~/src/cpython-cs@github(git:PEP499-cs)]fleet*> ./python.exe -i Lib/idlelib/pyshell.py Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'run' >>> sys.modules['__main__'] )> >>> sys.modules['pyshell'] )> >>> sys.modules['idlelib.pyshell'] So pyshell and idlelib.pyshell are distinct here. __main__ and pyshell are the same module, courtesy of your sys.modules assignment at the bottom of pyshell.py. Test 3 below will be with that commented out. Test 2: [~/src/cpython-cs@github(git:PEP499-cs)]fleet*> PYTHONPATH=$PWD/Lib ./python.exe -i -m idlelib.pyshell Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'run' >>> sys.modules['__main__'] >>> sys.modules['pyshell'] >>> sys.modules['idlelib.pyshell'] >>> id(sys.modules['__main__']) 4551072712 >>> id(sys.modules['pyshell']) 4551072712 >>> id(sys.modules['idlelib.pyshell']) 4551072712 So this has __main__ and idlelib.pyshell the same module from the PEP499 patch and pyshell also the same from your sys.modules assignment. Test 3, with the pyshell.py sys.modules assignment commented out: [~/src/cpython-cs@github(git:PEP499-cs)]fleet*> PYTHONPATH=$PWD/Lib ./python.exe -i -m idlelib.pyshell Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'run' >>> sys.modules['__main__'] >>> sys.modules['pyshell'] Traceback (most recent call last): File "", line 1, in KeyError: 'pyshell' >>> sys.modules['idlelib.pyshell'] >>> id(sys.modules['__main__']) 4552379336 >>> id(sys.modules['idlelib.pyshell']) 4552379336 Here we've got __main__ and idlelib.pyshell the same module and no 'pyshell' in sys.modules. I don't think I understand your "relative import" scenario. Cheers, Cameron Simpson ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] BDFL-Delegate appointments for several PEPs
On 25Mar2019 03:52, Terry Reedy wrote:
On 3/25/2019 12:27 AM, Cameron Simpson wrote:
I was thinking about IDLE and its tangled web of circular inports,
but I am now convinced that this change will not affect it. Indeed,
idlelib/pyshell.py already implements idea of the proposal, ending
with
if __name__ == "__main__":
sys.modules['pyshell'] = sys.modules['__main__']
main()
After more investigation, I realized that to stop having duplicate
modulue:
1. The alias should be 'idlelib.pyshell', not 'pyshell', at least
when imports are all absolute.
The PEP499 patch effectively uses __main__.__spec__.name for the name
of the alias. Does that simplify your issue?
[...]
What about (untested):
if __name__ == '__main__':
if __spec__.name not in sys.modules:
sys.modules[__spec__.name] = sys.modules['__main__']
When I start pyshell in my master repository directory on windows with
python -m idlelib.pyshell
__spec__.name is 'idlelib.pyshell, which I currently hard-coded.
When I start with what should be equivalent
python f:/dev/3x/lib/idlelib/pyshell.py
__spec__ is None and __spec__.name an attribute error.
Um, yes. I presume that since no "import" has been done, there's no
import spec (.__spec__).
Clearly the above needs to accomodate this, possibly with a fallback
guess. Is sniffing the end components of __file__ at all sane? ending in
idlelib/pyshell.py or pyshell.py? Or is that just getting baroque?
I don't think these are strictly the same from some kind of purist
viewpoint: the path might be anything - _is_ it reasonable to suppose
that it has a module name (== importable/finding through the import
path)?
If I run python f:/dev/3x/lib/idlelib/pyshell.py, the PEP patch would
have to notice that pyshell is a module within idlelib and alias
'__main__' to 'idlelib.pyshell', not 'pyshell'. Would the same be
true if within-package import were all relative?
I think so because we're using .__spec__.name, which I though was
post import name resolution.
You must be doing something different when __spec__ is None ;-). I
tested the patch and it does not raise AttributeError with the command
above.
Indeed. I may have fudged a bit when I said "The PEP499 patch
effectively uses __main__.__spec__.name". It modifies runpy.py's
_run_module_as_main function, and that is called for the "python -m
module_name" invocation, so it can get the module spec because it has a
module name.
So the patch doesn't have to cope with __spec__ being None.
As you say, __spec__ is None for "python path/to/file.py" so __spec__
isn't any use there. Apologies.
[...]
Test 3, with the pyshell.py sys.modules assignment commented out:
[~/src/cpython-cs@github(git:PEP499-cs)]fleet*> PYTHONPATH=$PWD/Lib
./python.exe -i -m idlelib.pyshell
[...]
>>> sys.modules['__main__']
>>> sys.modules['pyshell']
Traceback (most recent call last):
File "", line 1, in
KeyError: 'pyshell'
>>> sys.modules['idlelib.pyshell']
>>> id(sys.modules['__main__'])
4552379336
>>> id(sys.modules['idlelib.pyshell'])
4552379336
Here we've got __main__ and idlelib.pyshell the same module and no
'pyshell' in sys.modules.
I don't think I understand your "relative import" scenario.
If files other that pyshell used relative 'import ./pyshell' instead
of absolute 'import idlelib.pyshell', would the sys.modules key still
be 'idlelib.pyshell' or 'pyshell'? Which is to ask, would the alias
needed to avoid a second pyshell module still be 'idlelib.pyshell' or
'pyshell'?
Ok.
As I understand it Python 3 imports are absolute: without a leading dot
a name is absolute, so "import pyshell" should install
sys.module['pyshell'] _provided_ that 'pyshell' can be found in the
module search path.
Conversely, an "import .pyshell" is an import relative to the current
module's package name, equivalent to an import of the absolute path
"package.name.pyshell", for whatever the package name is. So (a) you can
only import '.pyshell' from within a package containing a 'pyshell.py'
file and (b) you can't import import '.pyshell' if you're not in a
package.
I stuffed a "test2.py" into the local idlelib like this:
import sys
print("running", __file__, __name__)
print(repr(sorted(sys.modules)))
print(repr(sys.paht))
from pyshell import idle_showwarning
print(repr(sorted(sys.modules)))
and fiddled with the "from pyshell import idle_showwarn
Re: [Python-Dev] Strange umask(?)/st_mode issue
On 29Mar2019 19:30, Steve Dower wrote:
On 29Mar.2019 1731, Nathaniel Smith wrote:
That does sound strange.
Oh good, it's not just me :)
How easily can you reproduce it? That majorly
effects how I would try to debug something like this...
No idea. Looking at the builds on Pipelines again, it seems that all the
builds since about 10am PDT have been failing, so perhaps they rolled
out something that changed how umask works? I don't have any idea
whether that's possible though.
If you're able to get an strace of a failed run then that would probably
tell us a *lot*.
I'll need help with that :) But if it's just commands in the
.azure-pipelines/posix-steps.yml file then sending a PR with them added
should do it.
Can you get a branch into your pipeline? Then you could just hack the
tarfile test with something quick and dirty like:
pid = os.getpid()
system("strace -p %d 2>/path/to/strace.out &" % pid)
time.sleep(2) # get strace heaps of time to start
just before the tarfile open. A ghastly hack but it would get you
debugging info. You could even decide to remove the strace.out file if
the umask issue doesn't show, if it is erratic (can't see why it would
be though).
Cheers,
Cameron Simpson
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Strange umask(?)/st_mode issue
On 01Apr2019 09:12, Steve Dower wrote: On 30Mar2019 1130, Gregory P. Smith wrote: I wouldn't expect it to be the case in a CI environment but I believe a umask can be overridden if the filesystem is mounted and configured with acls set? (oh, hah, Ivan just said the same thing) Yep, it appears this is the case. The Pipelines team got back to me and it seems to be a known issue - the workaround they gave me was to run "sudo setfacl -Rb /home/vsts" at the start, so I've merged that in for now (to master and 3.7). Could that be done _without_ sudo to just the local directory containing the test tar file? If that works then you don't need any nasty privileged sudo use (which will just break on platforms without sudo anyway). Cheers, Cameron Simpson ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Strange umask(?)/st_mode issue
On 01Apr2019 15:44, Steve Dower wrote: On 01Apr2019 1535, Cameron Simpson wrote: On 01Apr2019 09:12, Steve Dower wrote: On 30Mar2019 1130, Gregory P. Smith wrote: I wouldn't expect it to be the case in a CI environment but I believe a umask can be overridden if the filesystem is mounted and configured with acls set? (oh, hah, Ivan just said the same thing) Yep, it appears this is the case. The Pipelines team got back to me and it seems to be a known issue - the workaround they gave me was to run "sudo setfacl -Rb /home/vsts" at the start, so I've merged that in for now (to master and 3.7). Could that be done _without_ sudo to just the local directory containing the test tar file? If that works then you don't need any nasty privileged sudo use (which will just break on platforms without sudo anyway). I tried something similar to that and it didn't work. My guess is it's to do with the actual mount point? (I also tried without sudo at first, and when I didn't work, I tried it with sudo. I hear that's how to decide whether you need it or not ;) ) In any case, it only applies to the Azure Pipelines build definition, so there aren't any other platforms where it'll be used. Ok then. Curious: is the sudo now in the build setup? I'm just thinking that this isn't a tarfile specific fix but a "get correct POSIX umask semantics" fix, so it should apply to the entire environment. Or am I naive? Cheers, Cameron Simpson ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Overloading comparison operator for lists
This belongs on python-ideas, not python-dev. I've directed replies to this message there. Comments below. On 26May2019 21:52, Montana Burr wrote: NumPy arrays have this awesome feature, where array == 3 does an element-wise comparison and returns a list. For example: np.array([1,2,3,4,5])==3 returns [False,False,True,False,False] It would be cool if Python had similar functionality for lists. map(lamdba item: item==3, [1,2,3,4,5]) I'm not sure this rates extra Python features. Personally I'm -1 on this suggestion because == traditionally returns a Boolean, NumPy notwithstanding. Your example above doesn't return a Boolean. If that is not possible, perhaps we could consider allowing developers to overload operators on built-in types within the context of a project or module. For example, an overload in one module would have no effect on the same operator in a different module (such as any Python standard modules.) This is usally done by overloading dunder methods on classes. if you class subclasses a builtin eg int or list then the instances get the special behaviour. Additionally, let's then give the developers the option to explicitly import an overload from other modules. So, people could develop a module with the purpose of providing overloads that make complete sense within a certain context. If you go the subclass route you could do this with a mixin class (a class providing methods but little else, intended to be part of the MRO of a subclass). Cheers, Cameron Simpson ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: Is "%zd" format is portable now?
On 01Aug2019 15:14, Victor Stinner wrote: Is it supported on macOS, FreeBSD, AIX, Android, etc.? Looks like it is supported on MacOS 10.11.6 (El Capitan) here. "man 3 snprintf" lists the "z" modifier and says: The z modifier, when applied to a d or i conversion, indicates that the argument is of a signed type equivalent in size to a size_t. I know this is only one data point of many. Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/FB4JYXZE4KFN5JRQMB2FCRPTBJOWZEJL/
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
On 17Nov2020 09:55, Mark Shannon wrote:
>I'm wondering why
>```
>x = "value"
>try:
>1/0
>except Exception as x:
>pass
>```
>
>does not restore "value" to x after
>the `except` block.
Because the except is not a new scope. So it is the same "x".
Here:
https://docs.python.org/3/reference/compound_stmts.html#try
it says:
When an exception has been assigned using as target, it is cleared
at the end of the except clause. This is as if
except E as N:
foo
was translated to
except E as N:
try:
foo
finally:
del N
This means the exception must be assigned to a different name to be
able to refer to it after the except clause. Exceptions are cleared
because with the traceback attached to them, they form a reference
cycle with the stack frame, keeping all locals in that frame alive
until the next garbage collection occurs.
>Here's an example of restoring the value of the variable after the
>`except` block:
>
>>>> def f(x):
>... try:
>... 1/0
>... except Exception as x:
>... pass
>... return x
>...
>>>> f("hi")
>'hi'
In the Python 3.8.5 I don't see this:
Python 3.8.5 (default, Jul 21 2020, 10:48:26)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> def f(x):
... try:
... 1/0
... except Exception as x:
... pass
... return x
...
>>> f(3)
Traceback (most recent call last):
File "", line 1, in
File "", line 6, in f
UnboundLocalError: local variable 'x' referenced before assignment
and the same outside a function.
Cheers,
Cameron Simpson
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/CTCU2D2EAEVCHNNZK2LROZM5J4IW76JF/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
On 18Nov2020 08:34, Steven D'Aprano wrote:
>As far as shadowing other variables, if someone has so much code in
>their function, or at the top level, that they are at risk of
>inadvertantly shadowing variables, they have far more serious problems
>than the use of the conventional "e for exception".
>
>To start with, what else are they using "e" for? Surely it would be more
>important to use a better name *there* rather than change the exception
>variable.
Aye, that was my thought too. Avoiding a collision is so easy that
adding a magic special scope to Python seems overkill.
I'm also against lots of scopes. They cause pain.
I wrote a post in (I thought) a Python discussion about adding more
scopes citing what I consider a horrible flaw in Go's scope design which
shadows the common "err" return from various calls (Go doesn't use
exceptions), and provided a code example of how easy it was to
accidentally break one's error handling because of that design.
Annoyingly, I cannot find that post now. But basicly the if-statement
introduces a scrope (which starts at the test, before the {...}
brackets) while shadows an earlier variable, and on exiting the scrope
the common "err" variable is False again, indicating no error. Really
irritating.
Cheers,
Cameron Simpson
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/OFIF36EACAW5F4OBY7H6JXFQRVGZKKN6/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Which thing is "Development Mode"
On 26Feb2021 18:34, Coyot Linden wrote:
>As of 3.7, there is now a python feature called Development Mode:
>
>https://docs.python.org/3/library/devmode.html#devmode
>
>Which has a confusingly similar and nearly identical name to a setuptools
>feature:
>
>https://packaging.python.org/guides/distributing-packages-using-setuptools/#working-in-development-mode
>
>It seems like a bit of bikeshedding could create a name for one of them
>that disambiguates them.
I disagree.
They're both aids to development, with effects suitable to their domains
(the interpreter itself for the former and the method of obtaining
packages in the latter). I think the names are very apt, and similar
because of their purpose.
Picking gratuitously different names just to have "different names"
seems pointless and confusing to me.
This isn't some shared namespace where names need to be different to
avoid conflicts such as package names or DNS domain names, but similar
scenarios ("I'm doing development") applied to different aspects of a
Python environment. Of course it is natural to use a phrase like
"development mode" for each.
Cheers,
Cameron Simpson
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/F4P6M2GUJN2QU2KZWNDXOINFY2BJHAAL/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission
On Fri, 26 Feb 2021 at 23:36, Jim J. Jewett wrote:
> Whenever I've used except Exception or stronger, it was a sanitary
> barrier around code that might well do unpredictable or even stupid
> things. Adding a new kind of exception that I hadn't predicted --
> including ExceptionGroup -- would certainly fit this description, and
> I want my driver loop to do what I told it.
This is my concern as well. I've got plenty of "except Exception" and
IMO none consititutes "abuse" - they're there for the same reason Jim
cites above: a barrier around _arbitrary_ callback code - so that the
outer control facility continues to roll on as expected.
All of these are handler frameworks for use by other code - they
inherently call an unknown and arbitrary thing at some point and expect
to catch an Exception raised and suitably handle it (log it, record it
for the caller to see later, whatever).
Now _all_ those handlers will need a special except* handler for
ExceptionGroup, because they DO NOT KNOW whether the called code might
raise it. And "except Exception", the longstanding recommended way to
catch "everything (well almost everything)" no longer works.
Now, I've long wanted something like MultiError or ExceptionGroup to
gather miltiple failures for raising together at a suitable
slightly-later point.
[...]
On 27Feb2021 19:06, Guido van Rossum wrote:
>Our goal for the PEP is that *unless* you're going to use APIs that are
>documented to raise ExceptionGroup, you won't have to use `except *` nor
>will you have to deal with ExceptionGroup otherwise.
But.. this isn't Java, where the (hah!) type annotations document the
exceptions it raises.
Jim again, catching Exception to protect calling code:
>> def safe_fn():
>> try:
>> do_something_funky()
>> return True
>> except Exception:
>> print("Well, that was a shame...")
>> return False
>>
>> then I am intending to guarantee that calling safe_fn() will never
>> raise an exception. [...]
Guido:
>The modification I proposed where we have both BaseExceptionGroup and
>ExceptionGroup will satisfy this need. It basically means that *unless* you
>are explicitly using an API that is documented to raise
>[Base]ExceptionGroup (such as a future variant of asyncio.gather()), you
>don't have to care about it. [...] that's why I am proposing to change the PEP
>so that your
>code will remain safe.
That would be welcome to me, too.
Cheers,
Cameron Simpson
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/IKGK5YEPUPPWCAQ6NQEVGMKA63KPFUJQ/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission
On 26Feb2021 02:44, Irit Katriel wrote: >On Fri, Feb 26, 2021 at 2:00 AM Guido van Rossum wrote: >> OT: Is ExceptionGroup *really* immutable in the current >> implementation? As >> long as the 'errors' field is a list, I think one could mutate the list >> directly. > >It's not, but we were going to make it an immutable tuple. Could you say why? Other than wanting to discourage mutation happy code getting out there? The reason I ask is that the scenario which comes to my mind is something like: except *OSError as e: AIUI "e" is an ExceptionGroup containing only OSErrors. So, one common thing in my own code is this: try: do something with a file except OSError as e: if e.errno == ENOENT: # file is missing, but that is ok # because we treat it like an empty file elif ... some other ok situation ... else: raise My natural inclination with an ExceptionGroup would be to winnow the OSErrors I'm handed, and push the _unhandled_ errors back into the original ExceptionGroup. That way, after the various except* clauses, a nonempty ExceptionGroup would remain with the unhandled errors, and it might perhaps be reraised then. Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/FZI5JRQKTSMF7BYSIM2PJTO4E6IHSPDA/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission
Error) for e in value_eg: if some_ValueError_we_understand: handle it else: unhandled.append(e) if eg: unhandled.append(eg) if unhandled: raise eg0.with_exceptions(unhandled) You'll note that "except*" is not useful in this pattern. However... If a subgroup had a reference to its parent this gets cleaner again: unhandled = [] eg0 = None try: . except* OSError as os_eg: eg0 = os_eg.__original__ # or __parent__ or something for e in os_eg: if an ok exception: handle it else: except* ValueError as value_eg: eg0 = os_eg.__original__ # or __parent__ or something for e in value_eg: if some_ValueError_we_understand: handle it else: unhandled.append(e) except* Exception as eg: eg0 = os_eg.__original__ # or __parent__ or something unhandled.extend(eg) if unhandled: raise eg0.with_exceptions(unhandled) Except that here I have no way to get "eg0", the original ExceptionGroup, for the final raise without the additional .__original__ attribute. Anyway, I'm strongly of the opinion that ExceptionGroups should look like containers, be iterable, be truthy/falsey based on empty/nonempty and that .split and .subgroup should return empty subgroups instead of None. Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/XLJCHKZMFMIHBGVSVMFPRQQG2SZVVHEJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission
On 27Feb2021 00:54, Irit Katriel wrote:
>On Sat, Feb 27, 2021 at 12:47 AM Jim J. Jewett wrote:
>> > Is this not allowed?
>>
>> >try:
>> >try:
>> >obj.func()# function that raises ExceptionGroups
>> >except AttributeError:
>> >logger.info("obj doesn't have a func")
>> >except *(AttributeError, SyntaxError):
>> >logger.info("func had some problems")
>>
>> Allowed, but probably in error ... no AttributeError will get through to
>> the except * unless it happened inside the except AttributeError handler.
>> Did you mean:
>
>If obj.func() raises an ExceptionGroup that contains AttributeError then
>"except AttributeError" doesn't catch it. So it will get through.
And I, for one, would expect that. And _want_ that: I want the code to
do what I said, not have some magic which silently/invisibly intercepts
ExceptionGroups which contain something buried deep in their subgroup
tree.
We already allow "deep" exceptions out, to be caught at an arbitrary
outer call stack level. I don't see why ExceptionGroups should be any
different.
I certainly do not want ExceptionGroup([AttributeError]) conflated with
AttributeError. That fills me with horror.
Cheers,
Cameron Simpson
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/CSJJSEEW2EY56JKMBCSD6R5SEJOF2ZVU/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission
On 28Feb2021 23:56, Irit Katriel wrote:
>If you go long, I go longer :)
:-)
>On Sun, Feb 28, 2021 at 10:51 PM Cameron Simpson wrote:
>> On 28Feb2021 10:40, Irit Katriel wrote:
>> >split() and subgroup() take care to preserve the correct metadata on
>> >all
>> >the internal nodes, and if you just use them you only make safe
>> operations.
>> >This is why I am hesitating to add iteration utilities to the API. Like we
>> >did, people will naturally try that first, and it's not the safest API.
>>
>> Wouldn't it be safe if the ExceptionGroup were immutable, as you plan?
>> Or have you another hazard in mind?
>
>Making them immutable won't help the metadata issue. split() and subgroup()
>copy the (context, cause traceback) from the original ExceptionGroups (root
>and internal nodes of the tree) to the result trees. If you DIY creating
>new ExceptionGroups you need to take care of that.
Ah, right. Yes.
The overflows into my request for a factory method to construct a "like"
ExceptionGroup with a new exception tree lower down.
>> But all that said, being able to iterable the subexceptions seems a
>> natural way to manage that:
>>
>> unhandled = []
>> try:
>> .
>> except *OSError as eg:
>> for e in eg:
>> if an ok exception:
>> handle it
>> else:
>> unhandled.append(e)
>> if unhandled:
>> raise ExceptionGroup("unhandled", unhandled)
>
>
>You just lost the metadata of eg. It has no context, cause and its
>traceback begins here.
Aye. Hence a wish, again lower down, for some reference to the source
ExceptionGroup and therefore a handy factory for making a new group with
the right metadata.
>And the exceptions contained in it, if they came
>from a deeper tree that you flattened into the list, now look like their
>traceback jumps straight to here from the place they were actually first
>inserted into an ExceptionGroup. This may well be an impossible code path.
Perhaps so. But it doesn't detract from how useful it is to iterate over
the inner exceptions. I see this as an argument for making it possible
to obtain the correct metadata, not against iteration itself. Even if
the iteration yielded some proxy or wrapper for the inner exception
instead of the naked exception itself.
>Here's an example:
[... flattened ExceptionGroup with uninformative tracebacks ...]
>>>> import traceback
>>>> def flatten(exc):
>... if isinstance(exc, ExceptionGroup):
>... for e in exc.errors:
>...yield from flatten(e)
>... else:
>... yield exc
[...]
>>>> traceback.print_exception(flat_h())
>Traceback (most recent call last):
> File "", line 3, in flat_h
>ExceptionGroup: flat_h
>
> Traceback (most recent call last):
> File "", line 3, in f
> ValueError: 42
>
>traceback.print_exception(h()) prints a reasonable traceback - h() called
>g() called f().
>
>But according to traceback.print_exception(flat_h()), flat_h() called
>f().
>
>You can preserve the metadata (and the nested structure with all its
>metadata) if you replace the last line with:
>raise eg.subgroup(lambda e: e in unhandled)
Ok. That works for me as my desired factory. Verbose maybe, but
workable. Um. It presumes exception equality will do - that feels
slightly error prone (including some similar but not unhandled
exceptions). But I can write something more pointed based on id().
>And for the part before that, iteration, Guido's pattern showed that you
>can roll it into the subgroup callback.
Aye.
>> There are some immediate shortcomings above. In particular, I have no
>> way of referencing the original ExceptionGroup without surprisingly
>> cumbersome:
>
>> try:
>> .
>> except ExceptionGroup as eg0:
>> unhandled = []
>> eg, os_eg = eg0.split(OSError)
>> if os_eg:
>> for e in os_eg:
>> if an ok exception:
>> handle it
>> else:
>> unhandled.append(e)
>> if eg:
>> eg, value_eg = eg.split(ValueError)
>> if value_eg:
>> for e in value_eg:
>> if some_ValueError_we_understand:
>> handle it
>> else:
>> unhandled.append(e)
>> if eg:
>> unhand
[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission
On 28Feb2021 20:05, Guido van Rossum wrote: >I'm trying to shorten this again... > >On Sun, Feb 28, 2021 at 5:54 PM Cameron Simpson wrote: >> Let's turn this on its head: >> - what specific harm comes from giving EGs container truthiness for >> size >> testing? > >For something that's not a pure container, this is an anti-pattern. No >other subclass of BaseException (that I know of) can ever be falsey, so >existing code would be forgiven to write "if e:" where "if e is not None:" >was meant. I that's how you feel, I'll withdraw my advocacy. I agree it's a difference. I'll outline my view below the objections below, but agree to disagree here - I'm not seeking to convince you any more. >> - what specific harm comes from returning an empty EG from split on >> no >> match instead of None? > >It would be an exception. What is raising an empty EG supposed to do? It'd be rather pointless. It assumes a little care on the person making it (well, using it where I'd asked subgroup to make it). I guess if there's a fall through in the except* which reraises, its a burden on the clause author to check such a contingency. But wouldn't such a check normally be written: if eg: raise on the presumption that None (falsey) means nothing to raise? And wouldn't the same code work if eg looks like a container in truthiness? >> - what specific harm comes from supporting iteration with a caveat >> about >> metadata in the docstring, and maybe a recommendation of subgroup? > >Iteration is such an easy-to-use API that it would become an attractive >nuisance -- people would use it regardless of whether it's the right tool >for the job. Perhaps. But an easy to use API is normally what one wants. For you, this is a downside because you hold that it is also usually a poor choice for this facility. Maybe my use case and habits are unusual. >> - if I wanted to override subgroup and split to not return None, is >> that >> even possible with the current ? i.e. can I make a clean metadata >> preserved EG from an empty list? For example: >> >> eg2 = eg.subgroup(lambda e: False) >> >> Does that get me None, or an empty group? If the latter, I can roll my >> own subclass with my desired features. If not, I can't AFAICT. > >It returns None. Writing a subclass that behaves differently than the base >class will just confuse users who aren't aware of your overrides. Yeah, a subclass might be misleading. Maybe all I need is a help function or class. Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/Z4BM5JR7XQOIXCLW6GCE5N6V5Q72XE7W/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: change of behaviour for '.' in sys.path between 3.10.0a7 and 3.10.0b1
On 03Jun2021 17:06, Bernat Gabor wrote: >On Thu, Jun 3, 2021 at 5:05 PM Larry Hastings >wrote: >> On 6/3/21 4:20 AM, Chris Johns wrote: >> Might be out of context here, but IMHO "." shouldn't be assumed to be >> the current directory anyway. >> >> As someone who has ported python to a system where it isn't, these >> assumptions tend to cause problems. >> >> That sounds miserable. What does "." signify on such a system, if not the >> current directory? > >The only other thing that would make sense is a folder in the current >working directory named ".", not? I can't speak for Chris Johns, and we're off topic for this thread. That said, I dislike "." in sys.path, particularly near the front. In fact my own "invoke this module's main" wrapper scripts strip it out. I remain personally of the opinion that it is a security nightmare, making imports _subject_ to the current directory (which might be anywhere, include a directory owned by a hostile user). So i dislike it as a default. I know Robin becker is testing specific behaviour - I don't oppose being _able_ to put "." in sys.path (though I think a concrete absolute path is a saner choice). So for Bernat and Larry: not systems where "." doesn't mean the working directory, but definitely in situations where you want a more secure loading of modules (i.e. only from where I expect, not destabilised by a cd). Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YMV4LUW5LYFKJYEIHVM5GTCC6POM6BYF/ Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-Dev] Semantics of __int__(), __index__()
On 03Apr2013 14:47, Hrvoje Niksic wrote: | On 04/03/2013 01:17 PM, Nick Coghlan wrote: | Why would one want to be absolutely sure of getting an int? So that arithmetic can be relied upon? If a subclass can override the add methods etc it can look like an int, be a subclass instance of an int, and yet not act like an int in all ways. If I go int(x), I personally want a real int out the end of it. -- Cameron Simpson Q: What's the difference between a psychotic and a neurotic? A: A psychotic doesn't believe that 2 + 2 = 4. A neurotic knows it's true, but it bothers him. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] relative import circular problem
On 03Apr2013 12:12, Greg Ewing wrote: | Kristján Valur Jónsson wrote: | >However, relative imports can _only_ be performed using the "from X import Y syntax" | | This seems like a legitimate complaint on its own, [...] | There are a couple of ways that this could be resolved. One | would be to use the name resulting from stripping off the | leading dots, so that |import .foo | would bind the module to the name 'foo'. +0 from me (I'd have been +0.5 but for Guido's explaination about "import X" generally letting you use "X" exactly as phrased in the import). | Another would be | to always require an 'as' clause in this case, so that you | would have to write' |import .foo as foo And a big +1 for this. Cheers, -- Cameron Simpson The Force. It surrounds us; It enfolds us; It gets us dates on Saturday Nights. - Obi Wan Kenobi, Famous Jedi Knight and Party Animal. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] casefolding in pathlib (PEP 428)
On 11Apr2013 14:11, Guido van Rossum wrote: | Some of my Dropbox colleagues just drew my attention to the occurrence | of case folding in pathlib.py. Basically, case folding as an approach | to comparing pathnames is fatally flawed. The issues include: | | - most OSes these days allow the mounting of both case-sensitive and | case-insensitive filesystems simultaneously | | - the case-folding algorithm on some filesystems is burned into the | disk when the disk is formatted | | - case folding requires domain knowledge, e.g. turkish dotless I | | - normalization is a mess, even on OSX, where it's better defined than elsewhere Yes, but what's the use case? Specificly, _why_ are you comparing pathnames? To my mind case folding is just one mode of filename conflict. Surely there are others (forbidden characters in some domains, like colons; names significant only to a certain number of characters; an so forth). Thus: what specific problem are you case-folding to address? On a personal basis I would normally address this kind of thing with stat(), avoiding any app knowledge about pathname rules: does this path exist, or are these paths referencing the same file? But of course that doesn't solve the wider issue with Dropbox, where the rules differ per platform and where work can take place disparately on separate hosts. Imagining Dropbox, I'd guess there's a file tree in the backing store. What is its policy? Does it allow multiple files differing only by case? I can imagine that would be bad when the tree is presented on a case insensitive platform (eg Windows, default MacOSX). Taking the view that DropBox should avoid that situation (in what are doubtless several forms), does Dropbox pre-emptively prevent making files with specific names based on what is already in the store, or resolve them to the same object (hard link locally, or simply and less confusingly and more portably, diverting opens to the existing name like a CI filesystem would)? What about offline? That suggests that the forbidden modes should known to the Dropbox app too. Is this the use case for comparing filenames instead of just doing a stat() to the local filesystem or to the remote backing store (via a virtual stat, as it were)? What does Dropbox do if the local app is disabled and a user runs riot in the Dropbox directory, making conflicting names: allowed by the local FS but conflicting in the backing store or on other hosts? What does Dropbox do if a user makes conflicting files independently on different hosts, and then syncs? I just feel you've got a name conflist issue to resolve (and how that's done is partly just policy), and pathlib which offers some facilities related to that kind of thing. But a mismatch between what you actually need to do and what pathlib offers. Fixing your problem isn't necessarily a bugfix for pathlib. I think we need to know the wider context. Cheers, -- Cameron Simpson I had a *bad* day. I had to subvert my principles and kowtow to an idiot. Television makes these daily sacrifices possible. It deadens the inner core of my being.- Martin Donovan, _Trust_ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] casefolding in pathlib (PEP 428)
On 11Apr2013 16:23, Guido van Rossum wrote: | On Thu, Apr 11, 2013 at 4:09 PM, Cameron Simpson wrote: | > On 11Apr2013 14:11, Guido van Rossum wrote: | > | Some of my Dropbox colleagues just drew my attention to the occurrence | > | of case folding in pathlib.py. Basically, case folding as an approach | > | to comparing pathnames is fatally flawed. [...] | > | > Yes, but what's the use case? Specificly, _why_ are you comparing pathnames? | | Um, this isn't about Dropbox. This is a warning against the inclusion | of any behavior depending on case folding in pathlib, based on | experience with case folding at Dropbox (both in the client and in the | server). Ah. That wasn't so apparent to me. I took you to have tripped over a specific problem that pathlib appeared to be missolving. I've always viewed path normalisation and its ilk as hazard prone and very context dependent, so I tend not to do it if I can help it. | You seem to be completely misunderstanding me. I am not asking for | help solving our problem. I am giving advice to avoid baking the wrong | solution to this class of problems into a new stdlib module. Ok, fine. [...snip lots of stuff now not relevant...] | My reasoning is as follows. If pathlib supports functionality for | checking whether two paths spelled differently point to the same file, | users are going to rely on that functionality. But if the | implementation is based on knowing how and when to case fold, it will | definitely have bugs. So I am proposing to either remove that | functionality, or to implement it by consulting the filesystem. Which | of course has its own set of issues, for example if the file doesn't | exist yet, but there are ways to deal with that too. Personally I'd be for removing it, or making the doco quite blunt about the many possible shortcomings of guessing whether two paths are the same thing without being able to stat() them. I'll back out now. Cheers, -- Cameron Simpson Having been erased, The document you're seeking Must now be retyped. - Haiku Error Messages http://www.salonmagazine.com/21st/chal/1998/02/10chal2.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] enum discussion: can someone please summarize open issues?
On 28Apr2013 19:46, Ethan Furman wrote:
| int, float, and bool all have object constructors that take the
| given string and return a matching instance; int /may/ return a
| pre-existing instance, bool /will/ return a pre-existing instance.
I think Guido's already pointed out this:
>>> bool('False')
True
| As I've stated before, 'bool's are the closest existing data type to
| enums, in that bools use the object constructor to convert the
| incoming parameter to an existing bool instance, of which there will
| only ever be two.
|
| bool(9) is bool('maybe') is bool(True) is True
|
| and similarly, Enum behavior /should be/ (in my opinion ;)
|
| Season.AUTUMN is Season('AUTUMN') is Season(3)
I think that would be _efficient_, but as an outside user I wouldn't
necessarily expect it unless I'd ready the spec very closely and
the spec was explicit about it.
Coming from (in the fairly distant past) a C background, I naively
think of enums as nicely named ordinals of some kind and expect to
compare them with ==, not "is".
If you want to guarantee "is", be my guest though.
I don't know how that might make things go with some hypothetical subclass
with many many and extensible enum values.
| Like it or not, when you write `class Season(Enum)` a new *type* is
| created called Season, and Season(x) should either return a Season
| instance that matches x or raise. Having a match and raising anyway
| just doesn't seem to me to be the Python Way.
I'm +1 on all of this.
[...]
| >I'm not sure whether flufl.enums support creating additional instances after
the event, but if it did, I would expect
| >that I could say Season('WET') to get a new instance. I am indifferent to
whether or not Season('AUTUMN') should return
| >the existing AUTUMN enum value.
|
| Adding more enumerators after the fact should not be supported;
| there is subclassing for that. Not worth actively preventing, but
| not encouraged.
I'd go a bit further here: I'd take this final sentence as being
-0 on preventing adding more enumerals(?), whereas I'm a solid -1
on preventing it. By all means not actively support it, but very
against doing things that make it hard for a subclass to support
it.
Just 2c,
--
Would you remember a one-line .sig? - Paul Thompson, thomp...@apple.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] stat module in C -- what to do with stat.py?
On 21Jun2013 21:39, Nick Coghlan wrote: | On 21 June 2013 17:25, Victor Stinner wrote: | > How do you plan to handle the following case in Python? | > | > "Looking in more detail: for the S_IFMT flags, OSX/Darwin/FreeBSD defines | > 0xe000 as S_IFWHT (whiteout), but Solaris defines it as | > S_IFPORT (event port)." | > | > We may keep the Python module if it is kept unchanged, but the Python | > and C modules should have the same public API (the C module should not | > have more features). | | I think it's OK to expose additional platform specific features in the | C version, and have them fail cleanly with the pure Python version | (rather than silently giving the wrong answer). What's not OK is for | the standard library to regress for other implementations just because | we added a C implementation for the benefit of CPython. That's exactly | the kind of thing PEP 399 says we *won't* do. +1 I'm all for the C module exposing any and all of the S_* macros for the local platform, and good with the python module (if used because the C module isn't present, or conceivably is compiled out because it is known broken on this platform) exposing only the portable stuff. At least you can detect "I don't know what to do" rather than ploughing on mistakenly. -- Cameron Simpson Ignorance is preferable to error; and he is less remote from the truth who believes nothing, than he who believes what is wrong. - Thomas Jefferson ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] stat module in C -- what to do with stat.py?
On 20Jun2013 08:35, Benjamin Peterson wrote: | 2013/6/20 Charles-François Natali : | > 2013/6/20 Thomas Wouters : | >> If the .py file is going to be wrong or incomplete, why would we want to | >> keep it -- or use it as fallback -- at all? If we're dead set on having a | >> .py file instead of requiring it to be part of the interpreter (whichever | >> that is, however it was built), it should be generated as part of the build | >> process. Personally, I don't see the value in it; other implementations will | >> need to do *something* special to use it anyway. | > | > That's exactly my rationale for pushing for removal. | | +1 to nixing it. -1 to nixing it. I think there should be a pure python reference implementation. If it exposes only the portable constants/macros or, better, has a machine generated _section_ for the local platform macros, all to the good; it would not lie. - A huge amount of code only needs to care about the portable stuff (is this a dir, is this a regular file, is it neither). Missing local constants won't break such code. - A reference implementation exposes algorithms and purpose in a common language (Python, of course; everyone caring about such an impl can already read Python:-). - A reference implementation provides a base for other implementations to use outright, or to build on. - A reference implementation provides something to test against for comparison of the common stuff. - The implementation cost is low; the ref implementation already exists! Getting rid of it seems mad. Pruning/fixing/adapting the nonportable bits might be good. Christian Heimes' test patch would make that much easier to identify. So, -1 on removal of stat.py. Cheers, -- Cameron Simpson I will not do it as a hack I will not do it for my friends I will not do it on a MacI will not write for Uncle Sam I will not do it on weekends I won't do ADA, Sam-I-Am - Gregory Bond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 446: Add new parameters to configure the inherance of files and for non-blocking sockets
First up I broadly like this. You might want to make clear that the "blocking" parameter refers only to the file creation calls (eg socket.socket) and not to the file descriptor itself, and is not to be confused with the UNIX O_NONBLOCK file descriptor flag (and whatever equivalent flag may apply on Windows). This is deducable from your PEP, but I was at first confused, and initially expected get_blocking/set_blocking functions in New Functions. On 04Jul2013 13:03, Victor Stinner wrote: | Other Changes | - | The ``subprocess.Popen`` class must clear the close-on-exec flag of file | descriptors of the ``pass_fds`` parameter. I would expect Popen and friends to need to both clear the flag to get the descriptors across the fork() call, and _possibly_ to set the flag again after the fork. Naively, I would expect the the flag to be as it was before the Popen call, after the call. This is not addressed. Cheers, -- Cameron Simpson Time is nature's way of keeping everything from happening at once. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 446: Add new parameters to configure the inherance of files and for non-blocking sockets
On 05Jul2013 10:41, I wrote: | On 04Jul2013 13:03, Victor Stinner wrote: | | Other Changes | | - | | The ``subprocess.Popen`` class must clear the close-on-exec flag of file | | descriptors of the ``pass_fds`` parameter. | | I would expect Popen and friends to need to both clear the flag to | get the descriptors across the fork() call, and _possibly_ to set | the flag again after the fork. Naively, I would expect the the flag | to be as it was before the Popen call, after the call. This is harder than I'd thought through. Descriptors go across a fork() anyway, and after an exec() there is nobody to change their state. In order for the parent to restore the prior close-on-exec state (if deemed the right thing to do) it would need to know when the exec was done in the child. You'd need something like an extra descriptor attached to a pipe marked close-on-exec so that the parent could see EOF on the read end after the exec, since the exec would close the write end. tractable, but baroque. So I'd have the PEP take a position on the post-exec() state of the passed file descriptors, and if state is not going to be restored it should outright say that passing an fd clears the close-on-exec flag. Which is it does, but I think the doco should be pretty overt about: - this side effect of clearing close-on-exec on these fds - that the file descriptor and its close-on-exec state is common between the parent and the child - that setting of close-on-exec on the fds in the parent could in principle comprimise the exec in the child if done too soon If you decide state should get restored, the implementation gets a bit fiddlier to detect the exec() in the child. I can't speak for what might be required on non-UNIX. Cheers, -- Cameron Simpson Agree, for Law is costly. -- Very good advice to litigious Persons, founded upon Reason and Experience; for many Times the Charges of a Suit exceed the Value of the Thing in Dispute. - Bailey's dictionary, 1736 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 446: Add new parameters to configure the inherance of files and for non-blocking sockets
On 05Jul2013 08:24, Victor Stinner wrote: | 2013/7/5 Cameron Simpson : | > You might want to make clear that the "blocking" parameter refers | > only to the file creation calls (eg socket.socket) and not to the | > file descriptor itself, and is not to be confused with the UNIX | > O_NONBLOCK file descriptor flag (and whatever equivalent flag may | > apply on Windows). | | The two following codes are the same: | | s = socket.socket(..., blocking=False) | and | s = socket.socket(...) | s.setblocking(False) | | Both set O_NONBLOCK flag (UNIX) Oh, how embarassing. | or clear HANDLE_FLAG_INHERIT (Windows) | on the socket (which is a file descriptor). | | socket.socket(..., blocking=False) cannot fail with EAGAIN or | EWOULDBLOCK (at least on Linux according to the manual page). Fair enough. | > This is deducable from your PEP, but I was at first confused, and | > initially expected get_blocking/set_blocking functions in New | > Functions. | | socket objects already have get_blocking() and set_blocking() methods. Ah, ok then. As far as it goes. Shouldn't there be a general purpose os.get_blocking() and os.set_blocking() functions that operate on any file descriptor, paralleling os.get_cloexec() and os.set_cloexec()? They're not socket specific operations in principle, merely commonly used with sockets. | If we supported the blocking flag on any file descriptor, it would | make sense to add such function to the os module. But I explained in | the "Add blocking parameter for file descriptors and Windows | overlapped I/O" section why I only want to set/clear the blocking file | on sockets. But WHY? I think socket file decriptors should be treated little differently from other file descriptors. Certainly a stream socket connection is outstanding like other unseekable file descriptors in many respects. Just "only wanting it on sockets" seems a bit special purpose. If you're going address O_NONBLOCK in this proposal in addition to the close-on-exec flag, it should be general purpose. Otherwise, I think it should be separated out into a separate proposal if you're proposing it just for sockets; make this proposal just close-on-exec and forget the blocking stuff for this specific PEP. | > I would expect Popen and friends to need to both clear the flag to | > get the descriptors across the fork() call, and _possibly_ to set | > the flag again after the fork. Naively, I would expect the the flag | > to be as it was before the Popen call, after the call. | | As explained in the "Inheritance of file descriptors" section, the | close-on-exec flag has no effect on fork: | | "The flag has no effect on fork(), all file descriptors are inherited | by the child process." Agreed, see my second post where I explained I'd misthought it. However, as stated there, I think the side effects should be fairly overtly stated in the docuemntation. | The close-on-exec flag is cleared after the fork and before the | exec(). Pseudo-code for Popen: | | pid = os.fork() | if pid: | return pid | # child process | for fd in pass_fds: os.set_cloexec(fd, False) | os.execv(...) Fine. No state restore is fine with me. I think it should be as clear as possible in the doco. Cheers, -- Cameron Simpson Experience is what you have got after you needed it. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 446: Add new parameters to configure the inherance of files and for non-blocking sockets
On 05Jul2013 19:03, Victor Stinner wrote: | 2013/7/5 Cameron Simpson : | > | Both set O_NONBLOCK flag (UNIX) | > | > Oh, how embarassing. | | You said that the PEP is not cristal clear. Do you have a suggestion | to make it more clear? | | Should I mention that the close-on-exec flag is O_CLOEXEC on UNIX, and | HANDLE_FLAG_INHERIT on Windows? (except that HANDLE_FLAG_INHERIT set | means inheritable, whereas O_CLOEXEC set means *not* inheritable) It wouldn't hurt. Otherwise, colour me mostly convinced. I've read your "Rejected Alternatives" more closely and Ulrich Drepper's article, though I think the article also supports adding a blocking (default True) parameter to open() and os.open(). If you try to change that default on a platform where it doesn't work, an exception should be raised. | The close-on-exec flag only affects inheritance of file descriptors at | the execv() syscall, not at fork(). And execv() can be called without | fork(). Yes. Please forget I mentioned fork(); it is only relevant if you were offering some facility to undo the addition of cloexec to a Popen passed file descriptor. Which you are not. | Popen must clear close-on-exec flag on files from its pass_fds | parameter for convinience. It would be surprising to not inherit a | file descriptor passed to Popen in pass_fds, don't you think so? Yes yes yes. | I don't understand. I already wrote "The flag has no effect on fork(), | all file descriptors are inherited by the child process" in the PEP. | It is not enough? Do you have a suggestion to explain it better? My concerns are probably as well bundled with concerns about users not realising the a file descriptor state after a fork is shared, and then being confused if the other process did a seek() etc. I think I'm just concerned about a naive caller using this scenario: # gather up some already open fds to pass fds = (5,6,7) P = Popen(, pass_fds=fds) # now restore the cloexec state from before for fd in fds: os.set_cloexec(fd) where the cleanup loop in the parent gets to run before the exec() in the child. We could just consider it one of the pitfalls of fork/exec situations in general an expect people being this fiddly to need to add some synchornisation. Please consider your argument won. Cheers, -- Cameron Simpson ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 446: Add new parameters to configure the inherance of files and for non-blocking sockets
On 06Jul2013 14:43, Victor Stinner wrote: | 2013/7/6 Cameron Simpson : | > Yes. Please forget I mentioned fork(); it is only relevant if you | > were offering some facility to undo the addition of cloexec to a | > Popen passed file descriptor. Which you are not. | | Oh... gotcha. I now understood your concern. | | There is a little "trick" here: at fork, file descriptors are | duplicated in the child process and almost all properties (open state, | flags, offset, etc.) are shared. There is one little exception: file | attributes are not shared, and there is only one file attribute: | O_CLOEXEC. Setting O_CLOEXEC in a child process does not affect the | flag in the parent process ;-) I will add a unit test to check this. That's news to me. Interesting. I can't find UNIX doco to support this, though I haven't finished looking. None of the POSIX, Linux or MacOSX fork() manual pages seem to describe this. Can you point me at doco for this? I thought file descriptors were a direct index into a per-process table of file handle structures, and did not think the latter got part copies, part shared over a fork. Also, is O_CLOEXEC really the only file attribute? So O_NONBLOCK is a flag and not an attribute (guessing the terminology distinguishes these)? Now you mention it, ISTR noticing that there were distinct ioctls to adjust flags and something else. | I modified the PEP to explain that, and I also mentioned the name of | the close-on-exec and blocking flags: | http://hg.python.org/peps/rev/425f831fddf7 Thanks. Your change says: On Windows, the close-on-exec flag is ``HANDLE_FLAG_INHERIT``. I feel it would be clearer to say: On Windows, the close-on-exec flag is the inverse of ``HANDLE_FLAG_INHERIT``. or On Windows, the close-on-exec flag implemented using ``HANDLE_FLAG_INHERIT``. (This latter is fine because the rest of the paragraph explains the meaning of HANDLE_FLAG_INHERIT.) The bare "is" in your current wording suggests to me that the truthiness (ick!) of close-on-exec is the same as for HANDLE_FLAG_INHERIT, which it isn't. -- Cameron Simpson I've always been a big Greenaway fan - I've seen and enjoyed "The Falls" for crying out loud.- Peter Alexander Merel ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 446: Add new parameters to configure the inherance of files and for non-blocking sockets
On 06Jul2013 11:23, Charles-François Natali wrote: | > I've read your "Rejected Alternatives" more closely and Ulrich | > Drepper's article, though I think the article also supports adding | > a blocking (default True) parameter to open() and os.open(). If you | > try to change that default on a platform where it doesn't work, an | > exception should be raised. | | Contrarily to close-on-exec, non-blocking only applies to a limited | type of files (e.g. it doesn't work for regular files, which represent | 90% of open() use cases). sockets, pipes, serial devices, ... And you can set it on anything. Just because some things don't block anyway isn't really a counter argument. Cheers, -- Cameron Simpson Hofstadter's Law: It always takes longer than you expect, even when you take into account Hofstadter's Law. - Douglas Hosfstadter, Godel, Escher, Bach: an Eternal Golden Braid ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 446: Add new parameters to configure the inherance of files and for non-blocking sockets
Sorry for the delay; work got in the way. On 07Jul2013 15:21, Victor Stinner wrote: | Ok, I think that the best consensus here is to add a new | os.set_blocking() function. The function would not be available on | Windows, but should be available on all UNIX platforms. Thanks. | See the diff: | http://hg.python.org/peps/rev/ce61588d244c This sentence: The flag is cleared in the child process before executing the program, the change does not change the flag in the parent process. needs a semicolon, not a comma. | @Charles-François and Cameron: Do you prefer to new version of the PEP? | http://www.python.org/dev/peps/pep-0446/ | (will be updated in a few minutes) I'm happy with it. Thank you, -- Cameron Simpson ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On 15Jul2013 09:48, Steven D'Aprano wrote: | On 14/07/13 21:09, Nick Coghlan wrote: | | >Slight adjustment to the proposed wording to ensure completely undocumented | >modules are also considered private: | | -1 on this adjustment. If somebody cannot be bothered writing a one-line doc string: | | "This module is private, don't touch." | | then they certainly shouldn't be allowed to use up a public name for a private module. I don't think we should be encouraging more private, undocumented modules. (Documentation is valuable even for private modules.) | | I'd go further, and say that no more private modules should be accepted for the std lib unless they have a leading underscore. I suppose for backwards compatibility reasons, we probably can't go through the std lib and rename private modules to make it clear they are private, but we don't have to accept new ones without the underscore. I disagree. A private module is a perfectly sane way to implement the internals of something, especially if it is subject to implementation change in the future. Clarification: is Nick classifying a module with docstrings by no content in the "modules" section of the Python doco as undocumented? That is what I would presume; I'd expect the code to be littered with docstrings anyway, but the module as a whole is not presented in the documentation and so should be private and not relied upon. Cheers, -- Cameron Simpson I sometimes wish that people would put a little more emphasis upon the observance of the law than they do upon its enforcement. - Calvin Coolidge ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)
On 25Jul2013 00:35, Antoine Pitrou wrote: | On Wed, 24 Jul 2013 15:25:50 -0700 | Guido van Rossum wrote: | > >> To reduce the need for 3rd party subprocess creation code, we should | > >> have better daemon creation code in the stdlib -- I wrote some damn | > >> robust code for this purpose in my previous job, but it never saw the | > >> light of day. | > > | > > What do you call "daemon"? An actual Unix-like daemon? | > | > Yeah, a background process with parent PID 1 and not associated with | > any terminal group. | | But is that relevant to the PEP? A daemon only uses fork(), not exec(). Not necessarily. I routinely run other commands in a daemonlike mode, without the command itself having "make myself a daemon" functionality. If I were writing the launcher in Python (and my "setsid" script is Python already), I would be doing all the daemon bits in Python including the fork, then execing the other tool. Cheers, -- If it can't be turned off, it's not a feature. - Karl Heuer ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Daemon creation code in the standard library (was: Inherance of file descriptor and handles on Windows (PEP 446))
On 25Jul2013 17:26, Antoine Pitrou wrote: | Le Thu, 25 Jul 2013 12:08:18 +1000, | Ben Finney a écrit : | > Work continues on the PEP 3143-compatible ‘python-daemon’, porting it | > to Python 3 and aiming for inclusion in the standard library. | | The PEP hasn't been formally accepted yet, however. | Skimming back through the archives, one sticking point was the default | value of the "umask" parameter. Setting the umask to 0 if the user | didn't ask for something else is a disaster, security-wise. I have to say, +10 here. I have always found the convention that daemons have a umask of 0 to be utterly bogus, because almost all library code relies on the umask to set default security policy for initial file permissions. Prone to rant on this at length if required... Cheers, -- Cameron Simpson Cordless hoses have been around for quite some time. They're called buckets. - Dan Prener ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Revert #12085 fix for __del__ attribute error message
[ I've replied to two messages here: Georg's and Glenn's (supporting MRAB's).
- Cameron
]
On 25Sep2013 08:22, Georg Brandl wrote:
| Am 24.09.2013 00:11, schrieb Greg Ewing:
| > How about something like "Uncaught exception in __del__
| > method ignored"? It explains fairly clearly what has
| > happened, and also indicates what do do about it --
| > catch it in the __del__ method.
|
| "Exception in __del__ caught and not propagated:"
| Georg
On 24Sep2013 09:33, Glenn Linderman wrote:
| [MRAB]:
| >> Why not just say something like "Cannot propagate exception..."; it's
| >> simpler than "Unpropagatable exception...". [...]
|
| First one I've heard that accurately and unambiguously and briefly
| describes the issue.
| +1
I'm strongly in favour of Georg's one ("Exception in __del__ caught and not
propagated").
Why?
It says simply and clearly what has happened.
It denotes the relevant context (__del__) in which it happened.
The reader can then decide to find out why that decision may have been made.
Why not MRAB's? ("Cannot propagate exception...")
While better than "Unpropagatable exception" and "unraisable" and
"unreraisable", it has the same flaw that I think underlies Antoine's
concerns: it suggests the reason there's an error printed instead
of further propagation is a _property of the exception_.
It doesn't say it outright, but as an outsider that is definitely
what I would at first infer.
So: a small +0.1 for "Cannot propagate exception..."
And: a big +2 for "Exception in __del__ caught and not propagated:".
Cheers,
--
Cameron Simpson
Let the bullet of your thought achieve the true and final path of its
trajectory before it reaches the ear of the listener, lest it plow out the
other side of his head. - Henry David Thoreau
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Rename contextlib.ignored() to contextlib.ignore().
On 11Oct2013 13:51, Mark Lawrence wrote: > On 11/10/2013 11:33, Eric V. Smith wrote: > >And Antoine has again taught me a new word: > >polysemic: having more than one meaning; having multiple meanings > > IMHO a poor word to use. I'm a middle aged Brit who's never heard > of it so people who have English as a second language have little or > no chance :( IMO a fine word. New to me, but perfectly clear and memorable once explained. People with English as a second language might do better than us if they've cottoned onto the patterns arising from the root languages, versus a native speaker basing things entirely on memory. Cheers, -- Cameron Simpson The engine purrs beneath me with a purpose, ready for the pleasure of my hand upon the throttle. - Peter Smith ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Rename contextlib.ignored() to contextlib.ignore().
On 12Oct2013 15:27, Raymond Hettinger wrote: > On Oct 12, 2013, at 1:41 PM, Ethan Furman wrote: > . break_on reads just fine to me, although leave_on is probably better. > > Sorry, these are both really bad. I agree. > I've user tested ignore() and had good results. > It makes the intention of the code very clear. I think it is good also. I would also be happy with: ignore_exception for greater explicitness or, perhaps better for Ethan's camp: discard_exception But we've got "ignore" in play already. Can't we accept that it is somewhat evocative though clearly not perfect for everyone, and move on? Cheers, -- Cameron Simpson Once a Junior Programmer interrupted a Great Guru of the Sun to ask a Question of no importance. The Great Guru replied in words which the Junior Programmer did not understand. The Junior Programmer sought to rephrase the Question, saying, "Stop me if I appear stupid." The great Guru, without speaking, reached over and pressed L1-A. The Junior Programmer achieved Enlightenment. - Jon Green ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __objclass__ documentation
On 12Oct2013 15:03, Ethan Furman wrote: > It was pointed in Issue16938[1] that __objclass__ is not documented anywhere. > > Is the following an appropriate description? (in Doc/reference/datamodel.rst > in user-defined functions) > +-+---+---+ > | :attr:`__objclass__`| The class this object belongs | | > | | to; useful when the object is | | > | | a descriptor, or a virtual or | | > | | dynamic class attribute, and | | > | | it's __class__ attribute does | | "its" please. Like "his", "her", "their". > | | not match the class it is | | > | | associated with, or it is not | | > | | in that class' ``__dict__``. | | > +-+---+---+ > [1] http://bugs.python.org/issue16938 I'd make this two sentences. Just turn the semicolon into a full stop, and commence "This is useful when". I had trouble with the end. How about: or it is not in __class__.__dict__ Do I misunderstand? Cheers, -- Cameron Simpson Trust the computer... the computer is your friend. - Richard Dominelli ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: containment and the empty container
Note: I know you understand all this, I'm not "explaining" how things
work below, I'm explaining how/why I think about how these work.
On 08Nov2021 13:43, Ethan Furman wrote:
>When is an empty container contained by a non-empty container?
[...]
>For example:
>
>{} in {1:'a', 'b':2] <-- TypeError because of hashability
>set() in {1, 2, 'a', 'b'} <-- ditto
Right. Also, the members are not dicts or sets, respectively.
>[] in ['a', 'b', 1, 2] <-- False
The members are not lists.
>'' in 'a1b2' <-- True
This is because "in" isn't measuring a setlike membership (I mean here,
"subset"). It is looking for a substring. Compare:
>>> 'abc' in 'abcdef'
True
>>> 'abc' in 'abxcdef'
False
So str is not a set, because of its sequential nature.
>SomeFlag.nothing in SomeFlag.something <-- ???
I would expect "true", myself.
>Personally, I have never had a use for '' in 'some string' being True,
[...]
>So, does Flag adhere to set theory, or is just happenstance that some
>operators work the same for both groups?
I would expect flags to be like sets. I've always thought of them that
way - independent presence/absence of things. They're not sequenced. (If
they're packed into ints there's some sequencing in the storage behind
the scenes, but that isn't part of my model the rest of the time.)
>Can we have `SomeFlag.nothing in SomeFlag.something` be `False`, or
>would that be too surprising?
I'd be surprised by this. I would rather a clean "subset" notion here.
I was going to digress about "<" vs "in". For sets, "<" means subset and
"in" means "element in set". That isn't exactly parallel to flags. What
if "SomeFlag.nothing < SomeFlag.something" meant a subset test? Would we
need "in" at all? Or is "<" out of the picture because FLags, or at
least IntFlags, might do numeric-like stuff with "<"?
Cheers,
Cameron Simpson
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/Y2D2CUBO3C76DFTQTFV2V4IN7WPS72HD/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: containment and the empty container
On 08Nov2021 23:32, MRAB wrote:
>On 2021-11-08 22:10, Cameron Simpson wrote:
>>>{} in {1:'a', 'b':2] <-- TypeError because of hashability
>>>set() in {1, 2, 'a', 'b'} <-- ditto
>>>[] in ['a', 'b', 1, 2] <-- False
>>
>>Right. Also, the members are not dicts or sets, respectively.
>>
>More precisely, none of the keys are an empty set.
Aye, for the examples. But I was highlighting that the set/lists members
are a different type from the left-hand thing (empty set/list). A
category error, to my mind.
>>>SomeFlag.nothing in SomeFlag.something <-- ???
>>
>>I would expect "true", myself.
>>
>I'm not so sure.
>
>A flag could be a member, but could a set of flags?
Probably one flavour should raise a TypeError then with this comparison.
I wouldn't want "member flag present in SomeFlag.something" and
"set-of-members present in SomeFlag.something" to both be valid, there
is so much scope for ambiguity/accidents there. I would rather punt the
"set-of-members present" off to "<"/"<=" if we want to say either (which
of course we should be able to say).
So: To me "SomeFlag.nothing" is a set of flags, empty. _Not_ an
individual flag (a member).
Which means that _if_ we had a subset/strict-subset test, I'd want
"SomeFlag.nothing in" to raise a TypeError, since we should use the
subset/strict-subset operator for that flavour.
Which means I'm backtracking to TypeError, not 'true". _Provided" this
is a "member in set" comparison.
Of course, I think resently we don't distinguish a flag combination name
(".nothing", ".red_and_blue"==0b11) from single flags overtly. If there
are trite, _fast_, way to test count_of_flags(flag_value)==1 ? i.e. can
"members" be automatically distinguished from flag combinations
(obviously such a definition would exclude a
combination-of-just-1-flag).
>>I was going to digress about "<" vs "in". For sets, "<" means subset
>>and
>>"in" means "element in set". That isn't exactly parallel to flags. What
>>if "SomeFlag.nothing < SomeFlag.something" meant a subset test? Would we
>>need "in" at all? Or is "<" out of the picture because FLags, or at
>>least IntFlags, might do numeric-like stuff with "<"?
>>
>Actually, '<' means _proper_ subset (not ==), '<=' means subset
>(possibly ==).
Point taken. I'll try to be more precise. Warning: I'll become more
wordy.
Cheers,
Cameron Simpson
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/5SQCDE7HMKQUTAQPZKAK2SWZZ22SN36D/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Starting a new thread
On 12May2022 20:17, Barney Stratford wrote: >It seems like the consensus is that this is a good idea, but it’s the >wrong good idea. Should I cancel the PR or should we try to make it >into a better good idea? Why not shift slightly? As remarked, having a function automatically spawn threads can be confusing, because spawning a thread has implications for both the thread code itself and for the person calling the function. The caller might find that confusing or complex. Personally, my approach has been a tiny shim function named "bg" in my personal kit, to make it easy to spawn a regular function in a thread: T = bg(func, args) # returns a running Thread An approach I've also seen is Celery's one of decorating a function with attributes which can dispatch it as a task, roughly: @task def regular_function(.) ... and it can be dispatched by saying regular_function.defer(..) and variations. Some downsides to decorators include: - only decorated functions can be kicked off "automatically as threads"; that could be a good thing too - the decorator wires in the dispatch mechanism: your decorator spawns a thread, the Celery @task queues to a Celery task queue, and so forth So my personal inclination is to provide an easy to use shim for the caller, not the function. Eg: from cs.threads import bg . T = bg(func, args.) or: from cs.threads import bg as bg_thread from cs.later import bg as bg_later . T = bg_thread(func, args..) # run in a Thread . R = bg_later(func, args...) # hand off to a Later, get a Result for collection .. bg = bg_later # specify the queuing system .. ... do stuff via bg(), the chosen queuing system ... or whatever other queuing system you might be using. The idea here is to make it easy to submit a function to any of several things rather than decorating the function itself to submit to a now-hardwired thing. Just things to consider Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BX6J7BKEXMNQFARDKMKDSKAS7VTOVPIP/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Switching to Discourse
On 15Jul2022 11:59, Ethan Furman wrote: >On 7/15/22 08:37, Petr Viktorin wrote: >> And that's exactly why I consume Discourse in mailing list mode, with >> client-side >> filtering in Thunderbird. > >How do you handle threading? I follow each (sub)thread through to >it's end, as it keeps a logical flow, but Discourse has everything >linear which means that as I read it the conversation keeps jumping >around, making it hard to follow. I use discourse in mailing list mode. It only looks unthreaded :-) What actually happens is that it has its own message-ids and mangles things, and treats some headers differently. Eg if I post to discourse, the copy of the message which comes to me from discourse has a discourse generated message-id, not my original message-id. I did a bit of digging here: https://discuss.python.org/t/why-is-the-result-after-this-function-is-called-like-this/14680/15 but have not got to submitting a bug report. Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ETQ2X5FDAJ6DHV5LGCB7WG4VR4JXP43U/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Switching to Discourse
On 18Jul2022 16:53, Joannah Nanjekye wrote: >I see I might have misunderstood, thinking a python-dev channel on discuss >was not as active as the mailing list. Understood. > >My original stand on preferring email stands though due to stable >standards. Several of us use the email mode in Discourse. It works quite well. For me, both python-dev and the PDO posts land in my "python" local folder. Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BMUHSEWGQQYU6BRSWTUQ5PHTHQYDZPSU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Switching to Discourse
On 21Jul2022 10:29, Stephen J. Turnbull wrote: >As long as Discourse provides the In-Reply-To header field, the current >threading algorithm would work reasonably well. Discourse does not do `In-Reply-To:` very well at all. Here's some headers from the _second_ post in the "Core dev sprint this year" thread: Message-ID: In-Reply-To: References: The first post has this: Message-ID: References: So at present Discourse's email implementation is buggy. I need to submit a bug report. In essense: The `References` and `In-Reply-To` headers cite a _nonexistent_ message-id which just denotes the thread number in the web forum. By contrast, the message-id itself at least is nice and unique. _However_, someone participating in "email mode" will of course send a message with its own distinct message-id from their source system, and that does not survive the email->discourse->email-out process. So your local copy of the message, if you keep one (I do) it will be a distinct duplicate message in your mail folder. I don't expect that to change. Anyway: - Discourse does provide `In-Reply-To` and `References` - they're bogus - they can be fixed (I'll submit a bug report, someone told me how to do that...) Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/I26HZ7LJLTDGCG2APQ7BYP3YKT4IMIYZ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Switching to Discourse
On 21Jul2022 13:25, Stephen J. Turnbull wrote: >Cameron Simpson writes: > > Discourse does not do `In-Reply-To:` very well at all. Here's some > > headers from the _second_ post in the "Core dev sprint this year" > > thread: > > > > Message-ID: > > > > In-Reply-To: > > References: > >I'm tempted to write something uncivil, but instead I'm gonna go hug a >puppy and weep. > > > So at present Discourse's email implementation is buggy. I need to > > submit a bug report. > >Thank you! Bug report: https://meta.discourse.org/t/discourse-email-messages-are-incorrectly-threaded/233499 >You may find it useful to cite RFC 5322, section 3.6.4, and emphasize >"unique" while mentioning the algorithm for populating References and >In-Reply-To presented there. I've pointed them at it. I didn't belabor the id generation since their ids seem ok. It's the referencing header which are broken. > > _However_, someone participating in "email mode" will of course send > > a > > message with its own distinct message-id from their source system, and > > that does not survive the email->discourse->email-out process. [...] > > I don't expect that to change. > >That's just plain obnoxious. Anybody who's in the CCs who >participates in "email mode" will get (practically speaking) >unfilterable duplicates, and (if there is offline discussion) a bogus >new thread. Well, I think a number of mailing lists startyed do this to support DKIM or DMARC or something, otherwise their message would amount to a forgery if what they received. Never dug into it much. It's annoying, but not nearly as annoying as broken threading. >I wonder if this goes all the way through to the backend database (ie, > the only id a message gets are its thread id, a timestamp, and some >way to ensure a total order in the case of equal timestamps), and the >only place in Discourse where the unique Message-ID appears is in the >outgoing message. In that case getting any sanity in Discourse email >could be very expensive for Discourse. Personally I don't care how "expensive" it is. The email mode is, to me, a _major_ feature of Discourse. I'm sure I'm not alone is hating forums which require me to go to them (or, equally bad, send some kind of summary of new things - I'm looking at you, Google Groups). That Discourse does quite a good job of letting people participate via the forum or email is very welcome. But it has to be done correctly. Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RIGD2MMFY66Z6OCGPACASD4ZAFKHCQUR/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Switching to Discourse
On 21Jul2022 15:42, Steven Barker wrote: >So last night I tried activating mailing list mode, and I'm not >remotely >satisfied with the experience so far. Where mailing lists are concerned, >I'm *only *subscribed to python-dev. Not python-users, not -ideas, not >-packaging (if that's still a thing). But Discourse's mailing list mode >sends me messages for all of those things in such a volume that it drowns >out any discussions on topics that would have shown up on python-dev (I >think one PEP discussion message came in overnight, compared to 20+ posts >on other tags). After the first two -users messages came in almost >immediately, I tried telling discourse to mute the tags I don't care about, >but it seems not to work at all. The page with the mailing list mode toggle >warns that it overrides other email settings, so I think I just get >everything regardless of other settings. Aye. I jointed meta.discourse.org yesterday to submit a bug report about the threading (which they acknowledge is a regession) and this morning I turned off mailing list mode. So: - I still use mailing list mode for discuss.python.org; I'm abstractly interested in everything, and I have an aggressive email filtering system - I'm not using email mode for meta.discourse.org; Gah However: mailing list mode _is_ the firehose (minus muted topics and categories - nb not tags). With it off you can elect to receive messages as email for several things: - when you're sent a personal message (always/when away/never) - when quoted, replied to, @ed or new activity in watched categories, tags, topics (always/when away/never) - there's an option for an activity summary (I set it to "weekly" for meta.discourse.org) So what you could do is watch the "Dev" category if you want the equivalent of just python-dev. I'm going to see how it plays out, but I expect that will let me get a tightly limited email feed which I can treat like any other email list. >If my only option is to be subscribed to a firehose of stuff I don't >care about, I'm going to disable mailing list mode and if python-dev dies, I'll >pretty much quit following Python's development. As mentioned, mailing list mode seems to be the firehose. The other "Emails" settings seem reasonably versatile to me on the face of it. Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ISHRSDGQYOJNXRNFZUCX5QU4WTV7X7CG/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Switching to Discourse
On 21Jul2022 17:46, Christopher Barker wrote: >OT: >Does anyone else find it very odd to call a communication system >“discord”? I think it is a refreshing level of honesty about what live chat is like. As in "discordant". Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZYCOWJM2BMIVNNSZGGAWXF6KPMYPLS6K/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Switching to Discourse
On 15Jul2022 13:26, Barry Warsaw wrote: To me, that’s the biggest negative of Discourse, and I definitely prefer threaded discussions. Unfortunately though, much like top posting , I think that horse is out of the barn, what with other forums like GitHub being linear. On Jul 15, 2022, at 11:59, Ethan Furman wrote: How do you handle threading? I follow each (sub)thread through to it's end, as it keeps a logical flow, but Discourse has everything linear which means that as I read it the conversation keeps jumping around, making it hard to follow. Threading on the Python Discourse should now be working correctly. This is the good work of Martin Brennan: https://meta.discourse.org/u/martin Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/LP6OV4FWIKGHTKWYQKXJF6MFQ62HUXUZ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Switching to Discourse
On 27Sep2022 11:14, Barry Warsaw wrote: Threading on the Python Discourse should now be working correctly. This is the good work of Martin Brennan: https://meta.discourse.org/u/martin I’m not sure what “working correctly” means. Do you have some examples on discuss.python.org where threading is used? Is this something that previous discussions get for free or only new replies? I’m not finding much information about this feature on the Discourse site. The email side of Discourse now correctly sets the In-Reply-To and References headers, so that the email side discussions no longer appear "flat" and threads can be read in order. Example View of a recent thread from my mailer: 28Sep2022 01:58 Stone Zhong via N ┌>discuss-ideas 5.7K 27Sep2022 19:21 Serhiy Storchak N │┌> discuss-ideas 6.1K 27Sep2022 19:16 Václav Brožík v N ├>discuss-ideas 7.2K 27Sep2022 18:57 Serhiy Storchak N ┌> discuss-ideas 6.1K 27Sep2022 17:07 Stone Zhong via N [Py] [Ideas] Improve dqeue discuss-ideas 7.5K (I sort my email a bit backwards.) This would have been a flat nontopologically ordered grouping a few days ago. Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/S3MAG3HKIEUEHYKKTPQA2YNJC6WWUBWB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: [CVE-2022-37454] SHA3 vulnerability and upcoming Python patches for 3.7 - 3.10
On 07Nov2022 12:26, Gregory P. Smith wrote: I personally didn't feel this one was urgent enough to ask anyone to spend time doing an emergency security release as triggering the crash requires someone sending a multi-gigabyte amount of data into a sha3 hash function in a single .update() method call. That seems like a rare code pattern. How many applications ever do that vs doing I/O in smaller chunks with more frequent .update() calls? As it happens I'm doing some work for a media archiving company and we're looking at recording checksums for archived files. I _may_ well be choosing to mmap a file and calling .update() on the mapping in one go. That said, that's (a) niche and (b) not even written yet. I think I'd still agree that this might be a nonurgent fix (haven't read the CVE properly yet). Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/AOBBVHKUAFXSY3D6T5OK53PFB44ZWY4N/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Switching to Discourse
On 09Dec2022 14:58, Simon Cross wrote: I can now pull from Discourse or GH when it’s convenient for me. It’s also much easier to disengage for a few days and catch up later. I have a question about how you handle multiple communities. I'm subscribed to ~30 python-dev style mailing lists across different projects. There is no way I can open up 30 Discourse sites each day. Mail brings everything into one place for me, and I have things setup so that new mail from python-dev style lists is separated from my general inbox. Are you not subscribed to a bunch of communities? Or is there some way you aggregate or visit each forum that works nicely? I'm not Barry, but I have Discourse's "mailing list mode" enabled for the Python Discourse forums. That delivers all posts to me as email, and my filter rules file almost everything into my "python" mail folder. Same for the matplotlib forum etc. Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/Q4CBEETK7LSRX566U4HH5UGPWR2Z2VET/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Switching to Discourse
I'm presuming we're talking about ways to bidirectionally mirror between mailman and a Discourse forum. On 10Dec2022 21:21, Stephen J. Turnbull wrote: Since Mailman (and its users!) expects messages to arrive asynchronously and handles that using threading information, as far as I can see the Mailman side is handled as well as it can be now that Discourse provides threading information, and you just subscribe Mailman to Discourse. That would work for receiving Discourse posts into the Mailman mailing list. And obviously we can just sign up Discourse to Mailman. Can we? If you're talking about posts from the mailing list appearing on Discourse, this isn't trivial. The latter direction may be harder, depending on whether Discourse can make sense of batches of messages being composed independently of its message flow. It (anecdotally, link lower down) looks like you can make a new Topic (thread) on Discourse if your user has a sufficient trust level (discourse users become more trusted over time). But the message author needs to be a Discourse user. So either all the individual mailman users need to be Discourse members _or_ the from address needs to be a list-wide id (eg the list submission address), but then the posts won't have the right author on the Discourse side. None of that is easy to fix - mailing lists essentially just forward messages, with some gatewaying of what messages they allow inbound depending on the list. By contrast, Discourse posts have authors which must be Discourse users. The fact that Discourse didn't provide threading information to email users in the past suggest that it has an alternative mechanism for organizing mail flows, and message IDs and Reference headers from email may not be so easily integrated. No, that stuff works ok now. For new Topics (threads) this post: https://discourse.gnome.org/t/email-use-of-discourse-how-to-start-a-new-thread/12324/4 sugggests that you can email to a specific category and I presume it would appear as a new Topic. I suspect any `In-Reply-To` or `References` would be ignored. For replies, the email goes to a post-specific address which is used to stitch the message into an existing Topic discussion thread. The outgoing `In-Reply-To` and `References` headers are made on the fly when Discourse sends the post as email to whichever users are set up to receive an email copy. The source `Message-ID` is preserved. I know nothing about Discourse internals, but I suspect that's going to be the difficult part if there is one. With a little luck that will be no problem. ;-) I know a little, but not a lot. In short: copying the Discourse stuff to mailman could be done by subscribing the mailman list to the Discourse forum. Letting _nonDiscourse_ users reply or post to Discourse is not trivial. Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YKYJA7Z2K5ZHH546JMJGOFPYJA7UI3G5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Switching to Discourse
On 11Dec2022 19:50, Tiziano Zito wrote: On Sat 10 Dec, 17:47 +0100, Baptiste Carvello wrote: There is a small catch though: unless I'm mistaken, Discourse won't let you subscribe to just a set of categories, so any filtering has to happen on the Mailman side. Well, it is actually possible to achieve what you want. I have set up Discourse in mailing-list mode [1]. By default muted categories are not included in the emails you get in mailing list mode. So, you just need to mute all categories you don't care about. It is a bit of work, but it needs to be done only once. Until a new category is made. To have an almost complete equivalent of the topics that were once discussed on python-dev, you can just mute every thing except the "Core Development" category. This is the setting I am using since a while and I am quite happy with it. You may want to unmute the "PEPs" category as well. Otherwise, that sounds like a workable approach. FYI, a recent post to the Ideas Discourse category has these relevant headers: X-Discourse-Post-Id: 77785 X-Discourse-Topic-Id: 19277 List-ID: Discussions on Python.org | Ideas List-Archive: https://discuss.python.org/t/expanding-asyncio-support-for-socket-apis/19277 You can filter on List-ID to select a category, which would avoid muting a lot of categories. Threading info is kept quite nicely, so I read the discourse mail notifications as if it were a mailing list and I almost do not see any difference. Text is sometimes a bit messy if people heavily use the discourse formatting capabilities, but this kind of posts are quite rare in my experience. Discourse formatting is MarkDown, which is at least pretty easy to read. This does not solve the problem of engaging actively in a discussion, of course, but at least for me it is OK to login to discourse if I have to post, given that 99.99% of the time I just want to read posts in my mail client. Yah. The tricky bit to my mind is that if the flow is just Discourse->Mailman then people replying on the mailing list do not get seen by the Discourse users. Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/IWFUVVH6J5TV7GZBPFJB55BVJEF6I54M/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Switching to Discourse
On 12Dec2022 01:05, Abdur-Rahmaan Janhangeer wrote: If only, fellow list colleagues, I could see only the topics I choose on Discourse. You can mute catgeories. I don't. The Discourse feels like python-list, python-dev, python-* combined. I feel cluttered. Aye. But I filter my inbound email, and can divert things as desired. On Discourse i miss the simplicity of a medialess, interactionless, botless, privilegeless, notificationless, badgeless platform. I seriously recommend you try the mailing list mode. I use it pretty happily, and only hop over to the web forum when someone posts a screenshot (which we actively discourage for code snippets) and occasional other rare situations. Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/5C64RFHGVCVBEYGNSRROZGK43HJ5MZVU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: python3 -bb and hash collisions
On 10Sep2019 10:42, Daniel Holth wrote: [...] I stopped using Python 3 after learning about str(bytes) by finding it in my corrupted database. [...] Could you outline how this happened to you? Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/47XIWHCI43C5CA3AG4PSOPAZM4KNC7ZS/
[Python-Dev] Re: python3 -bb and hash collisions
On 13Sep2019 09:31, Matt Billenstein wrote:
On Fri, Sep 13, 2019 at 08:37:26AM +1000, Cameron Simpson wrote:
On 10Sep2019 10:42, Daniel Holth wrote:
[...]
> I stopped using Python 3 after learning about str(bytes) by finding it
> in
> my corrupted database. [...]
Could you outline how this happened to you?
Not the OP, but I've actually seen something like this happen in postgres, but
it's postgres doing the adaptation of bytea into a text column, not python str
afaict:
conn = psycopg2.connect(...)
with conn.cursor() as cursor:
... cursor.execute('update note set notes=%s where id=%s returning notes',
('hi there', 'NwMVUksheafn'))
... cursor.fetchall()
... cursor.execute('update note set notes=%s where id=%s returning notes',
(b'hi there', 'NwMVUksheafn'))
... cursor.fetchall()
...
[{'notes': 'hi there'}]
[{'notes': '\\x6869207468657265'}]
We were storing the response of an api request from requests and had grabbed
response.content (bytes) instead of response.text (str). I was still able to
decode the original data from this bytes representation, so not ideal, but no
data lost.
I did wish this sorta thing had raised an error instead of doing what
it did.
Aye. Somewhere there's some Python taking the b'' and accepting it for
the notes= parameter, presumably in the postgres dbapi code. That isn't
a Python language bug to my eye. It could be some careless 2->3 adaption
I guess. I suspect it isn't postgres itself (or its C library) mangling
things, it would be accepting a C string or character buffer.
Still, I can see how this can quietly leak mojibake into your database.
Thanks,
Cameron Simpson
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/VCPZ6EHTXQLULVVOKJWUOLBSRB6EG2XO/
[Python-Dev] Re: small improvement idea for the CSV module
On 29Oct2019 21:37, Oz Tiram wrote: Quite a few tutorials show how to use namedtuple to gain memory saving and speed, over the DictReader. [...] Python's own documentation has got a recipe in the collections modules[1] Hence, I was wondering why not go the extra step and add a new class to the CSV module NamedTupleReader? This class would do a good service for Python's users, especially newcomers who are still not aware of modules like the collections module. Just for some context: It is often suggested that several things which are just documentation examples in modules like collections or itertools get promoted to presupplied functions/classes. It tends to fail. Personally I think at least some of these would be a good thing. (Disclaimer: I am not a core dev.) The tricky bit is the bikeshedding: what nice-but-superfluous features or corner cases should it support? I'd be for such things provided they lent themselve to extension i.e. "here's a basic implementation which is easily subclassable for extra features". The other thing is that there are probably examples of csv->namedtuple scattered throughout pypi, users might just need to find them. I know I've written such a thing for myself: https://pypi.org/project/cs.csvutils/ I entirely agree this would be easier to find and use in the stdlib. And mine is probably overfeatured and underclean for use in the stdlib. Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/TFZOWA6RNPCKHD36I64OKCHXXLU5HXKV/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes
On 20Mar2020 13:57, Eric Fahlgren wrote: On Fri, Mar 20, 2020 at 11:56 AM Dennis Sweeney wrote: If ``s`` is one these objects, and ``s`` has ``pre`` as a prefix, then ``s.cutprefix(pre)`` returns a copy of ``s`` in which that prefix has been removed. If ``s`` does not have ``pre`` as a prefix, an unchanged copy of ``s`` is returned. In summary, ``s.cutprefix(pre)`` is roughly equivalent to ``s[len(pre):] if s.startswith(pre) else s``. The second sentence above unambiguously states that cutprefix returns 'an unchanged *copy*', but the example contradicts that and shows that 'self' may be returned and not a copy. I think it should be reworded to explicitly allow the optimization of returning self. My versions of these (plain old functions) return self if unchanged, and are explicitly documented as doing so. This has the concrete advantage that one can test for nonremoval if the suffix with "is", which is very fast, instead of == which may not be. So one writes (assuming methods): prefix = cutsuffix(s, 'abc') if prefix is s: ... no change else: ... definitely changed, s != prefix also I am explicitly in favour of returning self if unchanged. Cheers, Cameron Simpson ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YJPASLMACS2Y7B2PIQVASW3EY5Q2EAL3/ Code of Conduct: http://python.org/psf/codeofconduct/
