Re: Two-Dimensional Expression Layout

[Lawrence D’Oliveiro]

On Sunday, August 21, 2016 at 6:49:19 PM UTC+12, Michael Selik wrote:

> Indeed it is, not sure why.

Moral: It helps to understand the code you’re criticizing, before you start 
criticizing, not after.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Two-Dimensional Expression Layout

[Michael Selik]

On Sun, Aug 21, 2016, 3:06 AM Lawrence D’Oliveiro 
wrote:

> On Sunday, August 21, 2016 at 6:49:19 PM UTC+12, Michael Selik wrote:
>
> > Indeed it is, not sure why.
>
> Moral: It helps to understand the code you’re criticizing, before you
> start criticizing, not after.
>

A true statement, but not my takeaway from this discussion.

>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The Joys Of Data-Driven Programming

[Rustom Mody]

On Sunday, August 21, 2016 at 5:13:28 AM UTC+5:30, Lawrence D’Oliveiro wrote:
> On Saturday, August 20, 2016 at 8:08:24 PM UTC+12, Marko Rauhamaa wrote:
> >
> > Lawrence D’Oliveiro:
> > 
> >> On Thursday, August 18, 2016 at 4:47:28 PM UTC+12, Marko Rauhamaa wrote:
> >>>
> >>> ... as a rule, I dislike rules. Rule languages tend to
> >>> grow out of all bounds, always remain deficient and have impenetrable,
> >>> ad-hoc semantics.
> >>
> >> That’s a very peculiar thing to say, considering that data-driven
> >> programming is a well-known technique for writing compact code.
> >>
> >> Less code, and in particular, less repetitive code => fewer bugs.
> > 
> > I can't relate to what you say.
> > 
> > All I can say is that I've encountered numerous bad cases of rule
> > systems, eg:
> > 
> >  - iptables
> > 
> >  - selinux policies
> > 
> >  - systemd unit files
> > 
> >  - asterisk
> > 
> >  - sendmail
> 
> I can’t relate to you, either. All those are spectacularly successful 
> examples of highly-configurable pieces of software.

Very interesting dialogue…
Theoretically Ive generally been aligned along the lines of Lawrence:
“Declarative is better than imperative”
[ Ive even espoused it as “data-orientation” in 
http://blog.languager.org/2012/10/functional-programming-lost-booty.html , 
a page that gets a lot of hits of late]

But seeing Marko’s list I am very uneasy (with my own default preferences)
Ive always taken it as an individual specific incidental case when one of these
instances does not fare well.
However seeing that list together like that I cant help wondering:
Is the philosophy of data-driven-ness itself somehow ill-conceived?

Some other examples:
1.
Aren’t makefiles data-driven? And if so should not much more declarative
attempts like jam have been correspondingly more successful?
https://www.perforce.com/resources/documentation/jam

[Personally Ive always believed that jam is better than make and is less used
for entirely historical reasons; something like half the world eoling with crlf 
and half with lf.  But maybe make is really a better design because more 
imperative?]

2. Basic computing theory shows that re-s and dfas are equivalent.  Which would 
one prefer to write/debug? [Thats not a rhetorical question]

3. Scheme — the language but also the informal culture around SICP — is where
data-driven programming got installed as a sacred dictum.
Scheme doesn't seem to be doing too well of late…
Coincidence??
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The Joys Of Data-Driven Programming

[Chris Angelico]

On Sun, Aug 21, 2016 at 9:53 PM, Rustom Mody  wrote:
> Some other examples:
> 1.
> Aren’t makefiles data-driven? And if so should not much more declarative
> attempts like jam have been correspondingly more successful?
> https://www.perforce.com/resources/documentation/jam
>
> [Personally Ive always believed that jam is better than make and is less used
> for entirely historical reasons; something like half the world eoling with 
> crlf
> and half with lf.  But maybe make is really a better design because more 
> imperative?]

The issues with makefiles are mainly to do with metaprogramming (plus
a few specific issues with the format itself, which don't apply to the
more general concept). There's a ton of magic to cope with makefiles
that try to rebuild themselves, plus all the extra tooling around the
outside (configure scripts etc), to try to move to a meta-level above
the makefile and thus enable metaprogramming of makefile contents.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The Joys Of Data-Driven Programming

[Rustom Mody]

On Sunday, August 21, 2016 at 5:51:36 PM UTC+5:30, Chris Angelico wrote:
> On Sun, Aug 21, 2016 at 9:53 PM, Rustom Mody  wrote:
> > Some other examples:
> > 1.
> > Aren’t makefiles data-driven? And if so should not much more declarative
> > attempts like jam have been correspondingly more successful?
> > https://www.perforce.com/resources/documentation/jam
> >
> > [Personally Ive always believed that jam is better than make and is less 
> > used
> > for entirely historical reasons; something like half the world eoling with 
> > crlf
> > and half with lf.  But maybe make is really a better design because more 
> > imperative?]
> 
> The issues with makefiles are mainly to do with metaprogramming (plus
> a few specific issues with the format itself, which don't apply to the
> more general concept). There's a ton of magic to cope with makefiles
> that try to rebuild themselves, plus all the extra tooling around the
> outside (configure scripts etc), to try to move to a meta-level above
> the makefile and thus enable metaprogramming of makefile contents.

Yeah… I guess thats the issue said differently:
viz. For a human to write/think in terms of rules is cleaner (more ‘hi-level’)
than writing out imperative actions.
However when it comes to generating — what you call metaprogramming — rule-based
systems suddenly stop being attractive, especially when they are the target
of the generator.
Standard example:
Programmers dont like to code in assembly
Compiler writers dont like to generate to anything else
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The Joys Of Data-Driven Programming

[Marko Rauhamaa]

Rustom Mody :
>> > All I can say is that I've encountered numerous bad cases of rule
>> > systems, eg:
>> > 
>> >  - iptables
>> > 
>> >  - selinux policies
>> > 
>> >  - systemd unit files
>> > 
>> >  - asterisk
>> > 
>> >  - sendmail
> [...]
> Some other examples:
> 1.
> Aren’t makefiles data-driven?

Yes, "make" should be added to my sin list.

> [Personally Ive always believed that jam is better than make and is
> less used for entirely historical reasons; something like half the
> world eoling with crlf and half with lf. But maybe make is really a
> better design because more imperative?]

Don't know jam, but can heartily recommend SCons.

> 3. Scheme — the language but also the informal culture around SICP —
> is where data-driven programming got installed as a sacred dictum.
> Scheme doesn't seem to be doing too well of late… Coincidence??

Lisp's "data is code and code is data" is a good principle. I have
successfully used it with Python as well.

However, Scheme's syntax-rules are infamous.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The Joys Of Data-Driven Programming

[Rustom Mody]

On Sunday, August 21, 2016 at 7:50:39 PM UTC+5:30, Marko Rauhamaa wrote:
> Rustom Mody :
> >> > All I can say is that I've encountered numerous bad cases of rule
> >> > systems, eg:
> >> > 
> >> >  - iptables
> >> > 
> >> >  - selinux policies
> >> > 
> >> >  - systemd unit files
> >> > 
> >> >  - asterisk
> >> > 
> >> >  - sendmail
> > [...]
> > Some other examples:
> > 1.
> > Aren’t makefiles data-driven?
> 
> Yes, "make" should be added to my sin list.

If you treat make as elaboration on this theme:

target : source
TAB Imperative Action

it belongs to your 'sin' pattern if you concentrate on line 1
Not if you concentrate on line 2

So maybe it finds the sweet spot between the two??
> 
> > [Personally Ive always believed that jam is better than make and is
> > less used for entirely historical reasons; something like half the
> > world eoling with crlf and half with lf. But maybe make is really a
> > better design because more imperative?]
> 
> Don't know jam, but can heartily recommend SCons.

Likely true at some point
Probably not now: http://stackoverflow.com/questions/2847730/c-build-systems
[ tup seems to be the rage; scons seems to have the worst performance]

> 
> > 3. Scheme — the language but also the informal culture around SICP —
> > is where data-driven programming got installed as a sacred dictum.
> > Scheme doesn't seem to be doing too well of late… Coincidence??
> 
> Lisp's "data is code and code is data" is a good principle. I have
> successfully used it with Python as well.

Heh Really?!?
I can tell you if a student of mine mixed up code and data like emacs-lisp
throwing customize *data* into user-config *code* (.emacs) he would be very 
sorry...

> 
> However, Scheme's syntax-rules are infamous.

No idea what you are talking about
Macros? Define-syntax's syntax? I-dont-like-all-those
Lots-of
Irritating
Single
Parentheses
?
Something else?

If you ask me Lisp failed because of out of kilter uncontrollable 
superiority-complex
-- 
https://mail.python.org/mailman/listinfo/python-list

PEP suggestion: Uniform way to indicate Python language version

[rocky]

The problem:

1. there are various code inspection tools that parse Python programs looking 
for style issues or whatnot. The deeper ones have to do a full parse of the 
python program. It would be helpful if there were a uniform way to indicate the 
Python language level used in Python source code.

2. I get a standalone python program that is not part of a package. vcprompt 
https://bitbucket.org/gward/vcprompt might be an example
It would be helpful if there were an easy way to know what language version of 
Python it assumes

Perl has something like called "use perl".  "use" is roughly equivalent to 
"import".

Possible solutions:

Do it the similar to "use perl". Here "perl" is a package that just tests the 
parameter given it. In Python such the code would look something like

File/module python30.py

   import sys
   assert sys.version >= (sys.version_info >= (3, 0))

The above works, but to reduce proliferation of packages it might be preferable 
to come up with some way to pass a version specification string similar to the 
specification strings allowed in setup.py

A metadata tag as a comment in a docstring or in a comment.
Preferably this would be given towards the top of the file to make it easier 
for tools to extract this information.

Thoughts?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The Joys Of Data-Driven Programming

[Tim Chase]

On 2016-08-21 04:53, Rustom Mody wrote:
> 2. Basic computing theory shows that re-s and dfas are equivalent.
> Which would one prefer to write/debug? [Thats not a rhetorical
> question]

I'm curious where REs and DFAs are shown to be equivalent (serious,
not trying to be snarky).  I can see with no trouble that all REs are
DFAs, but can't convince myself that *all* DFAs are REs.  Though this
might also depend on the RE engine.  Do you have links to resources
on such an "all REs have equivalent DFAs and all DFAs have equivalent
REs" logic/argument?  Or maybe my understanding of DFAs is flawed.

And as for which I prefer to write, for simple text-processing that
doesn't involve nesting, I generally prefer REs; but for more complex
processing, a custom DFA tends to expand a bit more cleanly in my
experience.

-tkc


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The Joys Of Data-Driven Programming

[Marko Rauhamaa]

Rustom Mody :

> On Sunday, August 21, 2016 at 7:50:39 PM UTC+5:30, Marko Rauhamaa wrote:
>> Don't know jam, but can heartily recommend SCons.
>
> Likely true at some point
> Probably not now: http://stackoverflow.com/questions/2847730/c-build-systems
> [ tup seems to be the rage; scons seems to have the worst performance]

If SCons performance becomes an issue for you, your source code
repository is far too large. Time to break it up into independent
components.

Maybe there are other builders as good as SCons, but the nice thing
about SCons is that the "rule language" is ordinary Python. That's key:
don't invent a new rule language; use a real programming language.

>> Lisp's "data is code and code is data" is a good principle. I have
>> successfully used it with Python as well.
>
> Heh Really?!?

Yes. I was the treasurer of some soccer teams. I recorded the financial
transactions by updating a simple Python program that looked like this:


:::
paid('2014-11-17', 'DOE JANE', '1274',
 'November', None,
 { FEE_INCOME : 150.00 })
owed('2014-11-30', 'November',
 { 150 : [ SOPHIA, EMMA, OLIVIA, AVA, ISABELLA, MIA, ZOE, LILY ] })
:::


The program produced web pages, spreadsheets, emails etc.

> I can tell you if a student of mine mixed up code and data like
> emacs-lisp throwing customize *data* into user-config *code* (.emacs)
> he would be very sorry...

Er, ok...


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP suggestion: Uniform way to indicate Python language version

[Vincent Vande Vyvre]

Le 21/08/2016 à 20:28, rocky a écrit :

The problem:

1. there are various code inspection tools that parse Python programs looking 
for style issues or whatnot. The deeper ones have to do a full parse of the 
python program. It would be helpful if there were a uniform way to indicate the 
Python language level used in Python source code.

2. I get a standalone python program that is not part of a package. vcprompt 
https://bitbucket.org/gward/vcprompt might be an example
It would be helpful if there were an easy way to know what language version of 
Python it assumes

Perl has something like called "use perl".  "use" is roughly equivalent to 
"import".

Possible solutions:

Do it the similar to "use perl". Here "perl" is a package that just tests the 
parameter given it. In Python such the code would look something like

File/module python30.py

import sys
assert sys.version >= (sys.version_info >= (3, 0))

The above works, but to reduce proliferation of packages it might be preferable 
to come up with some way to pass a version specification string similar to the 
specification strings allowed in setup.py

A metadata tag as a comment in a docstring or in a comment.
Preferably this would be given towards the top of the file to make it easier 
for tools to extract this information.

Thoughts?


I don't understand your assertion

Python 3.2.3 (default, Jun 18 2015, 21:46:42)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> assert sys.version >= (sys.version_info >= (3, 0))
Traceback (most recent call last):
  File "", line 1, in 
TypeError: unorderable types: str() >= bool()


What you means ?

--
Vincent V.V.
Oqapy  . python3-exiv2 
 . Qarte 
 . PaQager 

--
https://mail.python.org/mailman/listinfo/python-list

Re: Does This Scare You?

[Michael Torrie]

On 08/19/2016 05:42 PM, Lawrence D’Oliveiro wrote:
> Python 3.5.2+ (default, Aug  5 2016, 08:07:14) 
> [GCC 6.1.1 20160724] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> from pathlib import PureWindowsPath
> >>> PureWindowsPath("prn").is_reserved()
> True
> >>> PureWindowsPath("prn.doc").is_reserved()
> True
> >>> PureWindowsPath("com9.exe").is_reserved()
> True
> >>> PureWindowsPath("c:/my documents/prn.doc").is_reserved()
> True

Which part are you getting at?  That Windows treats certain filenames as
reserved (a known gotcha that has existed for decades) or that Python
allows you to test whether a path is valid in Windows?

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The Joys Of Data-Driven Programming

[Ben Bacarisse]

Tim Chase  writes:

> On 2016-08-21 04:53, Rustom Mody wrote:
>> 2. Basic computing theory shows that re-s and dfas are equivalent.
>> Which would one prefer to write/debug? [Thats not a rhetorical
>> question]
>
> I'm curious where REs and DFAs are shown to be equivalent (serious,
> not trying to be snarky).  I can see with no trouble that all REs are
> DFAs, but can't convince myself that *all* DFAs are REs.

All NFAs as well -- the don't have to be deterministic in either
direction.

> Though this
> might also depend on the RE engine.

Yes.  Due to their use in real systems, lots of RE engines have acquired
many bells and whistles.  Some bells so fancy that the resulting "REs"
can recognise whole new classes of language.  In this context, though,
"regular expression" means the expression form of regular grammars
(level 3 in the Chomsky hierarchy).

> Do you have links to resources
> on such an "all REs have equivalent DFAs and all DFAs have equivalent
> REs" logic/argument?  Or maybe my understanding of DFAs is flawed.

This is called Kleene's Theorem.  Searching for that will bring up all
sorts of proofs in various styles.  Here's one of the direction you are
asking about:

http://www.cs.may.ie/staff/jpower/Courses/Previous/parsing/node6.html

The other direction is covered all over the place because it it so often
used inside an RE matching engine to "compile" the RE.

> And as for which I prefer to write, for simple text-processing that
> doesn't involve nesting, I generally prefer REs; but for more complex
> processing, a custom DFA tends to expand a bit more cleanly in my
> experience.

You may be confusing DFAs and pushdown automata.  I say this because of
your reference to nesting.  PDAs can be thought of as either DFAs or
NFAs with a stack so they can recognise languages with nested structure.

-- 
Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: saving octet-stream png file

[Jon Ribbens]

On 2016-08-19, Larry Martell  wrote:
> fd.write(request.POST[key])

You could try:

  request.encoding = "iso-8859-1"
  fd.write(request.POST[key].encode("iso-8859-1"))

It's hacky and nasty and there might be a better "official" method
but I think it should work.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The Joys Of Data-Driven Programming

[Lawrence D’Oliveiro]

On Sunday, August 21, 2016 at 11:53:52 PM UTC+12, Rustom Mody wrote:
> However seeing that list together like that I cant help wondering:
> Is the philosophy of data-driven-ness itself somehow ill-conceived?

I don’t understand why. They are all examples of the success of the technique, 
and I can mention lots more.

> 2. Basic computing theory shows that re-s and dfas are equivalent.

And non-deterministic FSAs, too.

> Which would one prefer to write/debug?

That depends, not on their capabilities, but on how they are formulated.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The Joys Of Data-Driven Programming

[Lawrence D’Oliveiro]

On Monday, August 22, 2016 at 12:21:36 AM UTC+12, Chris Angelico wrote:
> The issues with makefiles are mainly to do with metaprogramming (plus
> a few specific issues with the format itself, which don't apply to the
> more general concept). There's a ton of magic to cope with makefiles
> that try to rebuild themselves, plus all the extra tooling around the
> outside (configure scripts etc), to try to move to a meta-level above
> the makefile and thus enable metaprogramming of makefile contents.

In other words, the limitations of the original Make concept itself. So you fix 
this by
a) As per Lampson, adding another level of indirection (e.g. GNU Autotools, 
CMake)
b) Trying to replace Make altogether (e.g. SCons, Jam, Ant/Maven/Gradle)

Which approach is enjoying more success? I seem to be coming across CMake a 
lot. It is popular because it can wrap, not just around Make, but other build 
systems as well, including platform-specific IDE-based ones like Microsoft 
Visual Studio and Apple XCode. In other words, it is a cross-platform build 
system.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The Joys Of Data-Driven Programming

[Lawrence D’Oliveiro]

On Monday, August 22, 2016 at 2:20:39 AM UTC+12, Marko Rauhamaa wrote:
> Lisp's "data is code and code is data" is a good principle. I have
> successfully used it with Python as well.

You think self-modifying code is a good idea?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The Joys Of Data-Driven Programming

[Lawrence D’Oliveiro]

On Monday, August 22, 2016 at 2:20:39 AM UTC+12, Marko Rauhamaa wrote:
> ... can heartily recommend SCons.

It’s Python 2 only, not Python 3.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The Joys Of Data-Driven Programming

[Marko Rauhamaa]

Lawrence D’Oliveiro :

> On Monday, August 22, 2016 at 2:20:39 AM UTC+12, Marko Rauhamaa wrote:
>> Lisp's "data is code and code is data" is a good principle. I have
>> successfully used it with Python as well.
>
> You think self-modifying code is a good idea?

Why change topics?


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Two-Dimensional Expression Layout

[Lawrence D’Oliveiro]

On Sunday, August 21, 2016 at 7:19:49 PM UTC+12, Michael Selik wrote:
>
> On Sun, Aug 21, 2016, 3:06 AM Lawrence D’Oliveiro wrote:
> 
>> On Sunday, August 21, 2016 at 6:49:19 PM UTC+12, Michael Selik wrote:
>>
>>> Indeed it is, not sure why.
>>
>> Moral: It helps to understand the code you’re criticizing, before you
>> start criticizing, not after.
> 
> A true statement, but not my takeaway from this discussion.

Really? Even after it has been pointed out to you?

I wonder what the name is for people like that...
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The Joys Of Data-Driven Programming

[Marko Rauhamaa]

Lawrence D’Oliveiro :

> On Monday, August 22, 2016 at 2:20:39 AM UTC+12, Marko Rauhamaa wrote:
>> ... can heartily recommend SCons.
>
> It’s Python 2 only, not Python 3.

And? SCons is very good, definitely beats make. It also illustrates the
use of a real programming language for special applications (as opposed
to ad hoc rules).

The flawed thinking behind rules is often expressed like this: We want
rules because not everybody is a programmer.

However, with rules, not even a programmer knows how to configure the
thing.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The Joys Of Data-Driven Programming

[breamoreboy]

On Sunday, August 21, 2016 at 11:18:49 PM UTC+1, Marko Rauhamaa wrote:
> Lawrence D’Oliveiro :
> 
> > On Monday, August 22, 2016 at 2:20:39 AM UTC+12, Marko Rauhamaa wrote:
> >> ... can heartily recommend SCons.
> >
> > It’s Python 2 only, not Python 3.
> 
> And? SCons is very good, definitely beats make. It also illustrates the
> use of a real programming language for special applications (as opposed
> to ad hoc rules).
> 
> The flawed thinking behind rules is often expressed like this: We want
> rules because not everybody is a programmer.
> 
> However, with rules, not even a programmer knows how to configure the
> thing.
> 
> 
> Marko

Some people clearly disagree 
http://catherinedevlin.blogspot.co.uk/2009/06/dont-need-no-stinking-rules-engine.html

Kindest regards.

Mark Lawrence.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Does This Scare You?

[eryk sun]

On Sun, Aug 21, 2016 at 8:03 PM, Michael Torrie  wrote:
> On 08/19/2016 05:42 PM, Lawrence D’Oliveiro wrote:
>> Python 3.5.2+ (default, Aug  5 2016, 08:07:14)
>> [GCC 6.1.1 20160724] on linux
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> from pathlib import PureWindowsPath
>> >>> PureWindowsPath("prn").is_reserved()
>> True
>> >>> PureWindowsPath("prn.doc").is_reserved()
>> True
>> >>> PureWindowsPath("com9.exe").is_reserved()
>> True
>> >>> PureWindowsPath("c:/my documents/prn.doc").is_reserved()
>> True
>
> Which part are you getting at?  That Windows treats certain filenames as
> reserved (a known gotcha that has existed for decades) or that Python
> allows you to test whether a path is valid in Windows?

To me it's scary that this check misses cases because it's trying to
be cross-platform instead of simply relying on GetFullPathName to do
the work. For example, it misses at least the following cases:

Optional trailing colon:

>>> pathlib.Path('C:/foo/NUL:').is_reserved()
False
>>> print(os.path._getfullpathname('C:/foo/NUL:'))
\\.\NUL

Trailing spaces:

>>> pathlib.Path('C:/foo/NUL  ').is_reserved()
False
>>> print(os.path._getfullpathname('C:/foo/NUL  '))
\\.\NUL

Trailing spaces followed by a file extension:

>>> pathlib.Path('C:/foo/NUL  .txt').is_reserved()
False
>>> print(os.path._getfullpathname('C:/foo/NUL  .txt'))
\\.\NUL

It's also a bit disappointing that the author of this function claims
in a comment that "foo/NUL" isn't reserved yet decides to "err on the
side of caution" (obviously not enough). Of course "foo/NUL" is
reserved:

>>> print(os.path._getfullpathname('foo/NUL'))
\\.\NUL

I think what happened is that the author tested by calling open() on a
non-existing path. DOS device names are only reserved for existing
directories, in order to return an error for an invalid path. The
difference is how RtlGetFullPathName_Ustr is called. When
GetFullPathName calls the latter function it doesn't care whether or
not the path is valid. On the other hand, RtlDosPathNameToNtPathName_*
calls RtlGetFullPathName_Ustr with a parameter to check for an invalid
path, which makes the path normalization fail. For example:

Existing directory:

>>> os.path.exists('C:/Temp')
True
>>> f = open('C:/Temp/NUL')

Query the device name:

>>> hFile = msvcrt.get_osfhandle(f.fileno())
>>> ntdll.NtQueryObject(hFile, 1, byref(name), sizeof(name), None)
0
>>> print(name.Buffer[:name.Length//2])
\Device\Null

(\\.\NUL, i.e. \GLOBAL??\NUL, is a symbolic link to NT's \Device\Null.)

Non-existing directory:

>>> os.path.exists('C:/Spam')
False
>>> g = open('C:/Spam/NUL')
Traceback (most recent call last):
  File "", line 1, in 
FileNotFoundError: [Errno 2] No such file or directory: 'C:/Spam/NUL'
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Does This Scare You?

[Chris Angelico]

On Mon, Aug 22, 2016 at 10:38 AM, eryk sun  wrote:
> To me it's scary that this check misses cases because it's trying to
> be cross-platform instead of simply relying on GetFullPathName to do
> the work.

"Trying to be" cross-platform? The point of these path modules is to
*be* cross-platform. I can't call Windows APIs on my Linux box (short
of messing around with VMs or Wine or something, which are
dependencies that Python doesn't need). So yes, these *are*
cross-platform, and that's not scary, that's deliberate design.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Does This Scare You?

[eryk sun]

On Mon, Aug 22, 2016 at 12:44 AM, Chris Angelico  wrote:
> "Trying to be" cross-platform? The point of these path modules is to
> *be* cross-platform. I can't call Windows APIs on my Linux box (short
> of messing around with VMs or Wine or something, which are
> dependencies that Python doesn't need). So yes, these *are*
> cross-platform, and that's not scary, that's deliberate design.

Sorry for the wording SNAFU there. Of course it's cross-platform. But
I think it's a dubious check. On Windows itself I would never try what
it's attempting to do. I would just call GetFullPathName to see if the
path changes to use the \\.\ device namespace. I doubt that the 3
examples I gave are the only places that this simplistic
implementation of 'is_reserved" fails. It's giving a dangerous
impression to the user that the path will be safe to use on Windows
when that's not the case.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Does This Scare You?

[Steve D'Aprano]

On Mon, 22 Aug 2016 10:38 am, eryk sun wrote:

> To me it's scary that this check misses cases because it's trying to
> be cross-platform instead of simply relying on GetFullPathName to do
> the work. For example, it misses at least the following cases:

Instead of shaking in your boots over a simple bug in a non-critical
library, how about reporting these cases on the bug tracker with an
explanation of the problem?



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Does This Scare You?

[Chris Angelico]

On Mon, Aug 22, 2016 at 11:08 AM, eryk sun  wrote:
> On Mon, Aug 22, 2016 at 12:44 AM, Chris Angelico  wrote:
>> "Trying to be" cross-platform? The point of these path modules is to
>> *be* cross-platform. I can't call Windows APIs on my Linux box (short
>> of messing around with VMs or Wine or something, which are
>> dependencies that Python doesn't need). So yes, these *are*
>> cross-platform, and that's not scary, that's deliberate design.
>
> Sorry for the wording SNAFU there. Of course it's cross-platform. But
> I think it's a dubious check. On Windows itself I would never try what
> it's attempting to do. I would just call GetFullPathName to see if the
> path changes to use the \\.\ device namespace. I doubt that the 3
> examples I gave are the only places that this simplistic
> implementation of 'is_reserved" fails. It's giving a dangerous
> impression to the user that the path will be safe to use on Windows
> when that's not the case.

Understood, and apology accepted. But I think it should be understood
that this will *not* call any Windows APIs. It's called
"PureWindowsPath" and is an example of a pure path:

https://docs.python.org/3/library/pathlib.html#pure-paths

They're specifically documented as not touching any file system, which
means that they are cross-platform and cannot be guaranteed to be
perfect. If you know you're running on Windows, use WindowsPath
instead (trying to do so on a non-Windows system will cause an
immediate exception), and then you can test your examples on that. It
may well already probe the file system in that situation, and if not,
you could put forward the feature suggestion that it should. But the
PurePath classes won't.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The Joys Of Data-Driven Programming

[Rustom Mody]

On Monday, August 22, 2016 at 3:48:49 AM UTC+5:30, Marko Rauhamaa wrote:
> Lawrence D’Oliveiro :
> 
> > On Monday, August 22, 2016 at 2:20:39 AM UTC+12, Marko Rauhamaa wrote:
> >> ... can heartily recommend SCons.
> >
> > It’s Python 2 only, not Python 3.
> 
> And? SCons is very good, definitely beats make. It also illustrates the
> use of a real programming language for special applications (as opposed
> to ad hoc rules).
> 
> The flawed thinking behind rules is often expressed like this: We want
> rules because not everybody is a programmer.
> 
> However, with rules, not even a programmer knows how to configure the
> thing.

The same thing said from flip side:
Programmers think programming (or is it programmering?) is kewl
Maybe… Maybe not…
All of us, even supposedly educated/intelligent, can suffer from the hammer-nail
effect with our hammer being general purpose programming (languages)

It occurs to me that a general-purpose rule system was created decades ago — 
prolog
That it did not take off maybe just a question of maturity and not something
intrinsically wrong with the concept

Personal Note: In a major curriculum revision of 1990 I canvassed vigorously
for using lisp instead of C,C++,Pascal etc to teach programming.
I was booed/shooed out.
One could say I was simply delirious
Or that I was some 25 years ahead of the time with Lisp-practicalized being
spelt ‘python’
There are so many things that we dont notice that make for bridging the gap 
between
being theoretically neat and practically usable:
- good docs
- runs OTB
- Or at least easy to install
- C interface 

That said, I dont like prolog — more imperative than classic imperative 
programming
And full proper logic ie full first order logic without abominations like
cut and with full quantifier generality is uncomputable

ie this may also be a pipe-dream.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Does This Scare You?

[Terry Reedy]

On 8/21/2016 9:08 PM, eryk sun wrote:

On Mon, Aug 22, 2016 at 12:44 AM, Chris Angelico  wrote:

"Trying to be" cross-platform? The point of these path modules is to
*be* cross-platform. I can't call Windows APIs on my Linux box (short
of messing around with VMs or Wine or something, which are
dependencies that Python doesn't need). So yes, these *are*
cross-platform, and that's not scary, that's deliberate design.


Sorry for the wording SNAFU there. Of course it's cross-platform. But
I think it's a dubious check. On Windows itself I would never try what
it's attempting to do. I would just call GetFullPathName to see if the
path changes to use the \\.\ device namespace. I doubt that the 3
examples I gave are the only places that this simplistic
implementation of 'is_reserved" fails. It's giving a dangerous
impression to the user that the path will be safe to use on Windows
when that's not the case.


Since the function is there, please consider opening an issue and 
submitting a patch to improve it.  Also, could the doc be improved to 
say that is_reserved() == False might be wrong?  It is possible to 
broaden the test so that too many names, rather than too, are marked as 
reserved?  Better to say "don't use" for a good name rather than 'use' 
for a reserved name.


--
Terry Jan Reedy


--
https://mail.python.org/mailman/listinfo/python-list

Re: PEP suggestion: Uniform way to indicate Python language version

[rocky]

The assertion should have been 

  assert sys.version >= (3, 0)

If we want to indicate the Python program supports language versions 3.0 and 
greater.

On Sunday, August 21, 2016 at 3:59:48 PM UTC-4, Vincent Vande Vyvre wrote:
> Le 21/08/2016 à 20:28, rocky a écrit :
> > The problem:
> >
> > 1. there are various code inspection tools that parse Python programs 
> > looking for style issues or whatnot. The deeper ones have to do a full 
> > parse of the python program. It would be helpful if there were a uniform 
> > way to indicate the Python language level used in Python source code.
> >
> > 2. I get a standalone python program that is not part of a package. 
> > vcprompt https://bitbucket.org/gward/vcprompt might be an example
> > It would be helpful if there were an easy way to know what language version 
> > of Python it assumes
> >
> > Perl has something like called "use perl".  "use" is roughly equivalent to 
> > "import".
> >
> > Possible solutions:
> >
> > Do it the similar to "use perl". Here "perl" is a package that just tests 
> > the parameter given it. In Python such the code would look something like
> >
> > File/module python30.py
> >
> > import sys
> > assert sys.version >= (sys.version_info >= (3, 0))
> >
> > The above works, but to reduce proliferation of packages it might be 
> > preferable to come up with some way to pass a version specification string 
> > similar to the specification strings allowed in setup.py
> >
> > A metadata tag as a comment in a docstring or in a comment.
> > Preferably this would be given towards the top of the file to make it 
> > easier for tools to extract this information.
> >
> > Thoughts?
> 
> I don't understand your assertion
> 
> Python 3.2.3 (default, Jun 18 2015, 21:46:42)
> [GCC 4.6.3] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> import sys
>  >>> assert sys.version >= (sys.version_info >= (3, 0))
> Traceback (most recent call last):
>File "", line 1, in 
> TypeError: unorderable types: str() >= bool()
> 
> 
> What you means ?
> 
> -- 
> Vincent V.V.
> Oqapy  . python3-exiv2 
>  . Qarte 
>  . PaQager 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP suggestion: Uniform way to indicate Python language version

[Ben Finney]

rocky  writes:

> The assertion should have been 
>
>   assert sys.version >= (3, 0)

Perhaps you haven't tried any of these examples before presenting them?

>>> import sys
>>> assert sys.version >= (3, 0)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: unorderable types: str() >= tuple()

Please make some working code before recommending it :-)

-- 
 \  “I was the kid next door's imaginary friend.” —Emo Philips |
  `\   |
_o__)  |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The Joys Of Data-Driven Programming

[Rustom Mody]

On Monday, August 22, 2016 at 12:24:30 AM UTC+5:30, Tim Chase wrote:
> On 2016-08-21 04:53, Rustom Mody wrote:
> > 2. Basic computing theory shows that re-s and dfas are equivalent.
> > Which would one prefer to write/debug? [Thats not a rhetorical
> > question]
> 
> I'm curious where REs and DFAs are shown to be equivalent (serious,
> not trying to be snarky).  I can see with no trouble that all REs are
> DFAs, but can't convince myself that *all* DFAs are REs.  Though this
> might also depend on the RE engine.  Do you have links to resources
> on such an "all REs have equivalent DFAs and all DFAs have equivalent
> REs" logic/argument?  Or maybe my understanding of DFAs is flawed.
> 
> And as for which I prefer to write, for simple text-processing that
> doesn't involve nesting, I generally prefer REs; but for more complex
> processing, a custom DFA tends to expand a bit more cleanly in my
> experience.
> 
> -tkc

A recent thread has some links on this:
 https://mail.python.org/pipermail/python-list/2016-July/712046.html

Some of the responses to that (Ben Bacarisse??) found better links than mine 
apparently

Also ragel is a tour-de-force that exploits this:
http://www.colm.net/open-source/ragel/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP suggestion: Uniform way to indicate Python language version

[rocky]

Sorry should have been: 

 assert sys.version_info >= (3,0)



On Sunday, August 21, 2016 at 11:30:11 PM UTC-4, Ben Finney wrote:
> rocky  writes:
> 
> > The assertion should have been 
> >
> >   assert sys.version >= (3, 0)
> 
> Perhaps you haven't tried any of these examples before presenting them?
> 
> >>> import sys
> >>> assert sys.version >= (3, 0)
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: unorderable types: str() >= tuple()
> 
> Please make some working code before recommending it :-)
> 
> -- 
>  \  “I was the kid next door's imaginary friend.” —Emo Philips |
>   `\   |
> _o__)  |
> Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Does This Scare You?

[eryk sun]

On Mon, Aug 22, 2016 at 1:26 AM, Chris Angelico  wrote:
> They're specifically documented as not touching any file system, which
> means that they are cross-platform and cannot be guaranteed to be
> perfect. If you know you're running on Windows, use WindowsPath
> instead (trying to do so on a non-Windows system will cause an
> immediate exception), and then you can test your examples on that. It
> may well already probe the file system in that situation, and if not,
> you could put forward the feature suggestion that it should. But the
> PurePath classes won't.

PureWindowsPath.is_reserved() delegates to
self._flavour.is_reserved(), and _WindowsFlavour.is_reserved() is the
common implementation. I don't like splitting this up between a pure
and possibly inaccurate version that can be used on Unix systems
versus a concrete and accurate version that can only be used on
Windows. It can certainly be done if people think that's a good idea.

Terry, I'll open an issue that includes the 3 examples I provided plus
anything else I can find. Some patterns may emerge for the handling of
space, dot, and colon that can be used to fix this in a general way.
It would help to consult a reverse-engineered implementation of
RtlGetFullPathName_Ustr  and RtlIsDosDeviceName_Ustr. I'll check the
ReactOS source code.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP suggestion: Uniform way to indicate Python language version

[Chris Angelico]

On Mon, Aug 22, 2016 at 1:37 PM, rocky  wrote:
> Sorry should have been:
>
>  assert sys.version_info >= (3,0)

The next question is: How common is code like this? I don't put
version checks in any of my modules. Adding magic comments would be of
value only if this sort of thing is common enough to need its own
syntax.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP suggestion: Uniform way to indicate Python language version

[Steven D'Aprano]

On Monday 22 August 2016 14:33, Chris Angelico wrote:

> On Mon, Aug 22, 2016 at 1:37 PM, rocky  wrote:
>> Sorry should have been:
>>
>>  assert sys.version_info >= (3,0)
> 
> The next question is: How common is code like this? I don't put
> version checks in any of my modules. Adding magic comments would be of
> value only if this sort of thing is common enough to need its own
> syntax.

Most of my code is supposed to work on Python 2.4 through the latest 3.x 
version. That means I have lots of version-specific code.

But most of it uses feature detection rather than version checks. So instead of 
writing:

if sys.version < '3':
def next(iterator):
return iterator.next()


I write this:

try:
next
except NameError:
def next(iterator):
return iterator.next()

or possibly this:


if not hasattr(builtins, 'next'):
def next(iterator):
return iterator.next()


But some features depend on new syntax, and you can't check for the 
availability of new syntax at runtime. (Well, you can, by clever use of exec(), 
but its too much trouble than its worth.) So very, very occasionally I have to 
do a version check:


if sys.version < '3':
import mymodule2 as mymodule
else:
import mymodule3 as mymodule


But... I don't understand what this proposal actually is. We already have a 
uniform way to indicate the Python language version: check sys.version, or 
sys.version_info, or sys.hexversion, whichever is more appropriate for your 
needs. Because this is just an ordinary comparison, you can then do whatever 
you like, no matter how outlandish or unusual:


if sys.version < '3':
try:
import mymodule3
except SyntaxError:
print("Yay, a syntax error!")
else:
print("Now that's weird, that shouldn't happen...")
del mymodule3
import mymodule2 as mymodule
elif config[RUN_OLD_VERSION]:
import mymodule2 as mymodule
else:
import mymodule3 as mymodule


Could somebody (the OP?) please explain what is the purpose of this proposal, 
what it does, how it works, and when would people use it?



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP suggestion: Uniform way to indicate Python language version

[Stefan Behnel]

Steven D'Aprano schrieb am 22.08.2016 um 07:35:
> if sys.version < '3':
> import mymodule2 as mymodule
> else:
> import mymodule3 as mymodule

This condition is going to fail when Python 30.0 comes out.

Stefan


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP suggestion: Uniform way to indicate Python language version

[Random832]

On Mon, Aug 22, 2016, at 01:35, Steven D'Aprano wrote:
> Could somebody (the OP?) please explain what is the purpose of this
> proposal, what it does, how it works, and when would people use it?

I think what he wants is a way for a module which uses features
(syntactic or otherwise, but I suppose especially syntactic features
since this can't as easily be done with a runtime check using existing
mechanisms) from a particular python version and which makes no
provision to run under earlier versions to fail with a message like
"This script requires Python 3.4 or later" rather than a mysterious
syntax error or worse a runtime error after the program has been running
for some time.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP suggestion: Uniform way to indicate Python language version

[Random832]

On Mon, Aug 22, 2016, at 02:03, Stefan Behnel wrote:
> Steven D'Aprano schrieb am 22.08.2016 um 07:35:
> > if sys.version < '3':
> > import mymodule2 as mymodule
> > else:
> > import mymodule3 as mymodule
> 
> This condition is going to fail when Python 30.0 comes out.

Er, won't it rather stop behaving as desired when Python 10.0 comes out,
and resume when 30.0 does?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP suggestion: Uniform way to indicate Python language version

[Stefan Behnel]

Stefan Behnel schrieb am 22.08.2016 um 08:03:
> Steven D'Aprano schrieb am 22.08.2016 um 07:35:
>> if sys.version < '3':
>> import mymodule2 as mymodule
>> else:
>> import mymodule3 as mymodule
> 
> This condition is going to fail when Python 30.0 comes out.

Oh, sorry - make that Python 10.0, that's way closer! See? I got it wrong
because I failed to understand your hugely obfuscated code! ;)

Stefan


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP suggestion: Uniform way to indicate Python language version

[Chris Angelico]

On Mon, Aug 22, 2016 at 3:35 PM, Steven D'Aprano
 wrote:
> Could somebody (the OP?) please explain what is the purpose of this proposal,
> what it does, how it works, and when would people use it?

My theory: A directive is actually parsed by the compiler, not
executed at run-time. So you could have a clean ImportError that says
that you need a particular version, rather than a SyntaxError because
you used "async def". But given that you can't recover from it without
the same "import mymodule2 as mymodule" trick anyway, there's not that
much difference.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP suggestion: Uniform way to indicate Python language version

[Steven D'Aprano]

On Monday 22 August 2016 16:03, Stefan Behnel wrote:

> Steven D'Aprano schrieb am 22.08.2016 um 07:35:
>> if sys.version < '3':
>> import mymodule2 as mymodule
>> else:
>> import mymodule3 as mymodule
> 
> This condition is going to fail when Python 30.0 comes out.


That will be sometime around the year 2257, based on a 18 month minor release 
cycle and 8 minor releases per major release. I'll deal with it around 2233 or 
so when Python 28.0 comes out.


All joking aside, if Python is still in active development by the time we reach 
Python 9.7 in roughly seventy years time, I'll be surprised.



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list