Re: RFC: Proposal: Deterministic Object Destruction

[bartc]

On 02/03/2018 08:15, Paul Rubin wrote:


If someone says "but
limited memory", consider that MicroPython runs on the BBC Micro:bit
board which has 16k of ram, and it uses gc.


The specs say it also has 256KB of flash memory (ie. 'ROM'), so I 
suspect much of the program code resides there.



--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Help me with the Python! ODE system.

[alenkabor129769]

I can not find an example with this function: 
https://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.RK45.html#scipy.integrate.RK45.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: RFC: Proposal: Deterministic Object Destruction

[Rick Johnson]

On Thursday, March 1, 2018 at 10:13:51 PM UTC-6, Steven D'Aprano wrote:
[...]
> And for the record, consider a tree of nodes, where each
> node points back at the root of the tree, which is also a
> node. So what does the root node point back at?

Finally! A practical solution is offered that answers the
challenge _directly_. And even though we had to wade through
a swamp of waist-deep toxic goo to get here -- i must say --
i am relieved.

Mr. D'Aprano, you win the gold.

Now. Let's get back to Deterministic Object Destruction, shall we?
-- 
https://mail.python.org/mailman/listinfo/python-list

APPLICATION NOT RUNNING.

[Faruq Bashir via Python-list]

I try to run an application with the latest version of python that is python 
3.6.4 (32-bit) ., instead of running the application it only shows feel free to 
mail python-list@python.org if you continue to encounter issues,Please help me 
out thanks.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: RFC: Proposal: Deterministic Object Destruction

[ooomzay]

On Friday, March 2, 2018 at 12:22:13 AM UTC, MRAB wrote:
> On 2018-03-01 23:38, ooomzay wrote:
[Snip]
> > PEP343 requires two new methods: __enter__ & __exit__.
> > RIAA requires no new methods.
> >
> > RIAA resources are invariant: If you have a reference to it you can use it.
> > PEP343 resources can not be invariant: To be robust the enter/exit state
> > must be tracked and checked. (assert self.handle in the example)
[Snip]
> > PEP343 requires specialised "with" syntax, RIAA requires no new syntax.
> > Furthermore, although src & dst objects are still accessible outside the
> > PEP343 "with" block they are not in a usable state (not invariant).
> > 
> > In the RIAA case the resources are guaranteed to be in a usable state as 
> > long
> > as any reference exists (invariant). References can also be safely
> > passed around.The resource will be freed/closed when the last man has 
> > finished
> > with it, even in the face of exceptions.
> > [snip]
> >
> What's the difference between 'RAIIFileAccess' and 'open'?

I listed all the differences I know of in my preceding post. (I have snipped 
the other stuff so they are easy to see).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: RFC: Proposal: Deterministic Object Destruction

[ooomzay]

On Friday, March 2, 2018 at 8:16:22 AM UTC, Paul Rubin wrote:[snip]
> controlling stuff like file handles
> with scopes (like with "with") is fine.

How does with work for non-trivial/composite objects that represent/reference 
multiple resources or even a hierarchy of such objects where all the resources 
held must be released in a timely fashion when finished with?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: RFC: Proposal: Deterministic Object Destruction (Posting On Python-List Prohibited)

[ooomzay]

On Friday, March 2, 2018 at 1:59:02 AM UTC, Lawrence D’Oliveiro wrote:
> On Friday, March 2, 2018 at 1:03:08 PM UTC+13, ooo...@gmail.com wrote:
> > On Thursday, March 1, 2018 at 11:51:50 PM UTC, Lawrence D’Oliveiro wrote:
> >> On Friday, March 2, 2018 at 12:39:01 PM UTC+13, ooo...@gmail.com wrote:
> >>>
> >>> class RAIIFileAccess():
> >>> '''File Access-like Resource using [RAII] idiom'''
> >>> ...
> >>> def __del__(self):
> >>> low_level_file_close(self.handle) # fictitious function
> >> 
> >> This may be OK for files opening for reading, not so good for writing.
> > 
> > Please could you explain the issue you perceive with writing?
> 
> Compare the difference in behaviour between
> 
> f = open("/dev/full", "w")
> f.write("junk")
> f.close()
> 
> and
> 
> f = open("/dev/full", "w")
> f.write("junk")
> f = None
> 
> Should there be a difference in behaviour?

Could you please be very explicit about why you think the PEP would be more 
suitable when reading a file than when writing a file? You have responded with 
another question that I just can't see the relevance of (yet).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: RFC: Proposal: Deterministic Object Destruction

[Chris Angelico]

On Sat, Mar 3, 2018 at 1:18 AM,   wrote:
> On Friday, March 2, 2018 at 8:16:22 AM UTC, Paul Rubin wrote:[snip]
>> controlling stuff like file handles
>> with scopes (like with "with") is fine.
>
> How does with work for non-trivial/composite objects that represent/reference 
> multiple resources or even a hierarchy of such objects where all the 
> resources held must be released in a timely fashion when finished with?
>

Can you give me an example that works with RAII but doesn't work in a
'with' statement?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: RFC: Proposal: Deterministic Object Destruction

[ooomzay]

On Friday, March 2, 2018 at 4:35:41 AM UTC, Steven D'Aprano wrote:
> On Thu, 01 Mar 2018 16:26:47 -0800, ooomzay wrote:
> 
> >> >> When does the destination file get closed?
> >> >
> >> > When you execute:-
> >> >
> >> >del dst
> >> >
> >> > or:-
> >> >
> >> >dst = something_else
> >> 
> >> What if you don't?
> > 
> > Then the resource will remain open until your script exits at which
> > point it is probably not very well defined exactly when or even if the
> > destructor/__del__ will be called.
> > 
> > I.e. Don't do this! Did you have some realistic case in mind or are you
> > just probing the behaviour?
> 
> 
> If you're going to *require* the programmer to explicitly del the 
> reference:
>
> f = open("file")
> text = f.read()
> del f

But I am not! On the contrary RAII frees the programmer from even having to 
remember to close the file. The poster asked what would happen if the resource 
was deliberately kept open by storing a reference at global scope. 

In practice CPython destroys it cleanly on exit - but I am not sure the 
language guarantees this - in any case RAII won't make things any worse in this 
respect. (Logfiles are a common example of such global resource.)

> then you might as well require them to explicitly close the file:
> 
> f = open("file")
> text = f.read()
> f.close()
> 
> which we know from many years experience is not satisfactory except for 
> the simplest scripts that don't need to care about resource management.
> That's the fatal flaw in RAII: 

We must be discussing a different RAII. That is the raison d'etre of RAII: RAII 
directly addresses this problem in an exception-safe way that does not burden 
the resource user at all.

> the problem is that the lifespan of the resource may 
> not be the same as the lifetime of the object. 
>
> Especially for files, the 
> problem is that the lifespan of resource (the time you are actually using 
> it) may be significantly less than the lifespan of the object holding 
> onto that resource. Since there's no way for the interpreter to know 
> whether or not you have finished with the resource, you have a choice:
> 
> - close the resource yourself (either explicitly with file.close(), 
>   or implicitly with a context manager);
> 
> - or keep the resource open indefinitely, until such eventual time
>   that the object is garbage collected and the resource closed.

Hence my PEP! It enables RAII. The interpreter merely has to call __del__ as 
soon as the object is no longer referenced (as does CPYthon).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Problem: Need galileo running on debian wheezy

[Cousin Stanley]

Gene Heskett wrote:

> 
> And the rock64 doesn't have wifi hardware 
> that I know of.
>  

  I did manage to get wifi working on my rock64
  using a usb wifi dongle ordered from their store  

  It was a bit fiddly to set up, somewhat shaky 
  connection-wise, and slower than I was comfortable with
  so I went with an ethernet connection which is quick 
  and stable  


> So, is galileo shown for armhf's ?

  The galileo package information 
  doesn't explicitly mention arm 

  However, I've had no problem 
  with any other python package
  on the rock64 here  

  dpkg-print-architecture indicates that it is arm64 
  and I pull the debian stretch packages and updates from 

o http://mirrors.kernel.org/debian/ stretch main non-free contrib
  
o http://mirrors.kernel.org/debian/ stretch-updates main contrib non-free

o http://ppa.launchpad.net/ayufan/rock64-ppa/ubuntu xenial main


> The reason for posting here is that galileo 
> is supposedly written in python, 
> but don't know if 2 or 3.

  The stretch package  depends  information for galileo
  indicates it is for python3  

  $ apt-cache show galileo

Depends: python3-requests (>= 2), python3-usb (>= 1.0.0~b2), python3:any 
(>= 3.4~)



-- 
Stanley C. Kitching
Human Being
Phoenix, Arizona

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: RFC: Proposal: Deterministic Object Destruction

[Paul Moore]

On 2 March 2018 at 15:09,   wrote:
> We must be discussing a different RAII. That is the raison d'etre of RAII: 
> RAII directly addresses this problem in an exception-safe way that does not 
> burden the resource user at all.

RAII works in C++ (where it was initially invented) because it's used
with stack-allocated variables that have clearly-defined and limited
scope. In my experience writing C++, nobody uses RAII with
heap-allocated variables - those require explicit allocation and
deallocation and so are equivalent to having an explicit "close()"
method in Python (or using __del__ in CPython as it currently exists).

Python doesn't have stack allocation, nor does it have a deterministic
order of deletion of objects when their last reference goes out of
scope (which can happen simultaneously for many objects):

class Tracker:
def __init__(self, n):
self.n = n
def __del__(self):
print("Deleting instance", self.n)

def f():
a = Tracker(1)
b = Tracker(2)

f()

The language doesn't guarantee that a is removed before b. Are you
proposing to make that change to the language as well?

Also Python only has function scope, so variables local to a
smaller-than-the-function block of code aren't possible. That's
something that is used in C++ a lot to limit the lifetime of resources
under RAII. How do you propose to address that (without needing
explicit del statements)? That's why the with statement exists, to
clearly define lifetimes smaller than "the enclosing function". Your
proposal doesn't offer any equivalent (other than an extra function).

Consider C++:

void fn() {
for (i = 0; i < 1; ++i) {
char name[100];
sprintf(name, "file%d.txt, i);
File f(name); // I don't think std::ofstream doesn't support RAII
f << "Some text";
}
}

Or (real Python):

def fn():
for i in range(1):
with open(f"file{i}.txt", "w") as f:
f.write("Some text")

How would you write this in your RAII style - without leaving 10,000
file descriptors open until the end of the function?

def loop_body(i):
f = open(f"file{i}.txt", "w")
f.write("Some text")
def fn():
for i in range(1):
loop_body(i)

That's both less efficient (function calls have a cost) and less
maintainable than the with-statement version.

Paul
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: RFC: Proposal: Deterministic Object Destruction

[ooomzay]

On Friday, March 2, 2018 at 2:43:09 PM UTC, Chris Angelico wrote:
> On Sat, Mar 3, 2018 at 1:18 AM,  ooomzay wrote:
> > On Friday, March 2, 2018 at 8:16:22 AM UTC, Paul Rubin wrote:[snip]
> >> controlling stuff like file handles
> >> with scopes (like with "with") is fine.
> >
> > How does with work for non-trivial/composite objects that 
> > represent/reference multiple resources or even a hierarchy of such objects 
> > where all the resources held must be released in a timely fashion when 
> > finished with?
> >
> 
> Can you give me an example that works with RAII but doesn't work in a
> 'with' statement?

My claim is about the relative elegance/pythonic nature of RAII w.r.t. 'with'. 
Probably with enough coding and no ommissions or mistakes 'with' could do 
achieve the same result that RAII does automatically without any 
burden on the user.

Consider the hierarchy/tree of resource-holding objects postulated above... 
Presumably __enter__, __exit__ (or functional equivalent) would have to be 
implemented at every layer breaking the invariance. Thus complicating and 
making less robust every object it touches.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Problem: Need galileo running on debian wheezy

[Gene Heskett]

On Friday 02 March 2018 10:27:57 Cousin Stanley wrote:

> Gene Heskett wrote:
> > 
> > And the rock64 doesn't have wifi hardware
> > that I know of.
> > 
>
>   I did manage to get wifi working on my rock64
>   using a usb wifi dongle ordered from their store 

fitbit says its BTLE, claims a 20 foot range.

>   It was a bit fiddly to set up, somewhat shaky
>   connection-wise, and slower than I was comfortable with
>   so I went with an ethernet connection which is quick
>   and stable 
>
> > So, is galileo shown for armhf's ?
>
>   The galileo package information
>   doesn't explicitly mention arm 
>
>   However, I've had no problem
>   with any other python package
>   on the rock64 here 
>
>   dpkg-print-architecture indicates that it is arm64
>   and I pull the debian stretch packages and updates from 
>
> o http://mirrors.kernel.org/debian/ stretch main non-free contrib
>
> o http://mirrors.kernel.org/debian/ stretch-updates main contrib
> non-free
>
> o http://ppa.launchpad.net/ayufan/rock64-ppa/ubuntu xenial main
>
So I have several choices, not all of which are arm64 based. I also have 
an old hp lappy I could put stretch on. And I found a BT dongle but no 
clue yet if compatible with the BTLE of the fitbit. More checking to see 
if it can pair. But I just now installed the rest of the BT toy's, but 
not checked for function yet. Might have to buy another dongle.  And/or 
find a place to put things in this midden heap. Too many years in the 
same house, but its paid off too.

Thanks.

> > The reason for posting here is that galileo
> > is supposedly written in python,
> > but don't know if 2 or 3.
>
>   The stretch package  depends  information for galileo
>   indicates it is for python3 
>
>   $ apt-cache show galileo
> 
> Depends: python3-requests (>= 2), python3-usb (>= 1.0.0~b2),
> python3:any (>= 3.4~) 
>
>
> --
> Stanley C. Kitching
> Human Being
> Phoenix, Arizona



-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: APPLICATION NOT RUNNING.

[Jason Friedman]

>
> I try to run an application with the latest version of python that is
> python 3.6.4 (32-bit) ., instead of running the application it only shows
> feel free to mail python-list@python.org if you continue to encounter
> issues,Please help me out thanks.
>

Hello, you might have more success if you restate your question.  Read
http://www.catb.org/esr/faqs/smart-questions.html#intro for information
about how to ask in a way that makes it more likely someone will help.

TL;DR:
Tell us the operating system.  Copy-and-paste into your question what you
typed.  Copy-and-paste into your question what the computer typed in
return.  Don't attach images.
-- 
https://mail.python.org/mailman/listinfo/python-list

Python 3.6 fails to install to non-standard directory under Linux

[bsferrazza]

I posted this to Stackoverflow and the original post can be seen here. I'll try 
and copy and past the contents below. Thank you for your help!

https://stackoverflow.com/questions/49074327/python-3-6-fails-to-install-to-non-standard-directory-under-linux

I have a completely insulated boostrapped toolchain+binaries setup (located 
under /home/tools) that has been able to build everything that I throw at it. 
That includes Python 2.7.14, which built without issue with a simple --prefix. 
The lone exception is Python 3.6.4. I had to edit the configure and setup.py 
script to patch some of the hardcoded paths that were set to standard /usr 
directories (the link to ncursesw was notable). It completely compiled and 
built correctly, except failed during the 'make install' installation of pip. 
The installation scripts create a pip-build-* directory under /tmp and the 
source of the failure seems to be centered around that. It appears when the 
directory is first created, it's made with the proper permission mask. But 
after failure, I can see it's only user writable (not even readable or 
executable).
***

d-w--- 4 myuser   eng  4096 Mar  1 00:56 pip-build-xh7onsny/

***
Here's the section of the strace where it first creates the pip-build-* 
directory.
***

[pid 19771] lstat("/home/myuser/.cache/pip", {st_mode=S_IFDIR|0700, 
st_size=4096, ...}) = 0
[pid 19771] geteuid()   = 10345
[pid 19771] access("/home/myuser/.cache/pip", W_OK) = 0
[pid 19771] mkdir("/tmp/pip-build-xh7onsny", 0700) = 0
[pid 19771] lstat("/tmp", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=28672, 
...}) = 0
[pid 19771] lstat("/tmp/pip-build-xh7onsny", {st_mode=S_IFDIR|0700, 
st_size=4096, ...}) = 0
[pid 19771] getcwd("/nfs/home/myuser/lfs/sources/Python-3.6.4"..., 1024) = 
46


Here's the end of the strace that shows where it Error'ed out.


[pid 19771] munmap(0x2ac5506c, 4096) = 0
[pid 19771] stat("/nfs/home/myuser/lfs/sources/Python-3.6.4/Lib/shutil.py", 
{st_mode=S_IFREG|0644, st_size=40227, ...}) = 0
[pid 19771] open("/nfs/home/myuser/lfs/sources/Python-3.6.4/Lib/shutil.py", 
O_RDONLY|0x8 /* O_??? */) = 4
[pid 19771] ioctl(4, FIOCLEX)   = 0
[pid 19771] fstat(4, {st_mode=S_IFREG|0644, st_size=40227, ...}) = 0
[pid 19771] ioctl(4, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff38a58460) = -1 ENOTTY 
(Inappropriate ioctl for device)
[pid 19771] lseek(4, 0, SEEK_CUR)   = 0
[pid 19771] read(4, "\"\"\"Utility functions for copying"..., 32768) = 32768
[pid 19771] lseek(4, 0, SEEK_CUR)   = 32768
[pid 19771] read(4, "e, extract_dir=None, format=None"..., 8192) = 7459
[pid 19771] read(4, "", 8192)   = 0
[pid 19771] close(4)= 0
[pid 19771] ioctl(2, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff38a59f40) = -1 EINVAL 
(Invalid argument)
[pid 19771] write(2, "Exception:\nTraceback (most recen"..., 1739Exception:
Traceback (most recent call last):
  File "/tmp/tmpojuc47gy/pip-9.0.1-py2.py3-none-any.whl/pip/basecommand.py", 
line 215, in main
status = self.run(options, args)
  File 
"/tmp/tmpojuc47gy/pip-9.0.1-py2.py3-none-any.whl/pip/commands/install.py", line 
385, in run
requirement_set.cleanup_files()
  File "/tmp/tmpojuc47gy/pip-9.0.1-py2.py3-none-any.whl/pip/utils/build.py", 
line 38, in __exit__
self.cleanup()
  File "/tmp/tmpojuc47gy/pip-9.0.1-py2.py3-none-any.whl/pip/utils/build.py", 
line 42, in cleanup
rmtree(self.name)
  File 
"/tmp/tmpojuc47gy/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/retrying.py", line 
49, in wrapped_f
return Retrying(*dargs, **dkw).call(f, *args, **kw)
  File 
"/tmp/tmpojuc47gy/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/retrying.py", line 
212, in call
raise attempt.get()
  File 
"/tmp/tmpojuc47gy/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/retrying.py", line 
247, in get
six.reraise(self.value[0], self.value[1], self.value[2])
  File "/tmp/tmpojuc47gy/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/six.py", 
line 686, in reraise
raise value
  File 
"/tmp/tmpojuc47gy/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/retrying.py", line 
200, in call
attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "/tmp/tmpojuc47gy/pip-9.0.1-py2.py3-none-any.whl/pip/utils/__init__.py", 
line 102, in rmtree
onerror=rmtree_errorhandler)
  File "/nfs/home/myuser/lfs/sources/Python-3.6.4/Lib/shutil.py", line 476, in 
rmtree
onerror(os.lstat, path, sys.exc_info())
  File "/nfs/home/myuser/lfs/sources/Python-3.6.4/Lib/shutil.py", line 474, in 
rmtree
fd = os.open(path, os.O_RDONLY)
PermissionError: [Errno 13] Permission denied: '/tmp/pip-build-xh7onsny'
) = 1739
[pid 19771] lstat("/tmp/tmpojuc47gy", {st_mode=S_IFDIR|0700, st_size=4096, 
...}) = 0
[pid 19771] open("/tmp/tmpojuc47gy", O_RDONLY|0x8 /* O_??? */) = 4
[pid 19771] ioctl(4, FIOCLEX)   = 0
[pid 19771] fstat(4, {st_mode=S_IFDIR|0700, st_size=4096, ...}) =

Re: Python 3.6 fails to install to non-standard directory under Linux

[bsferrazza]

Here's my configure I need to set ac_cv_fun_utimensat=no and 
ac_cv_func_futimens=no because presumably the file-system or kernel on my 
system doesn't support nanosecond timestamps. With these options, and patching 
the configure/setup.py files to remove references to /usr/lib/ncursesw and 
replace with my /home/tools/lib path, everything builds just fine. It's during 
the 'make install' phase of pip that it fails.

./configure --prefix=/home/tools \
  --enable-shared \
  --with-system-expat \
  --with-system-ffi \
  --with-ensurepip=install \
  ac_cv_func_utimensat=no \
  ac_cv_func_futimens=no
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to make Python run as fast (or faster) than Julia

[Python]

On Mon, Feb 26, 2018 at 09:57:06AM +, Steven D'Aprano wrote:
> Besides, if you want Python with no GIL so you can run threaded code, why 
> aren't you using IronPython or Jython?

But this is just another oversimplified argument.  In the real world
there rather often exist constraints which have nothing to do with
what is technically possible, but which make impractical or infeasible
what may be technically possible.

Python is often a preferred solution because it is often fantastic for
rapid implementation and maintainability.  The GIL's interference
with threaded code performance has, for me at least, on several
occasions been...  disappointing (perf costs of removing it aside)
because it gets in the way of choosing Python for such solutions.
Jython and IronPython are simply not feasible options for me, for
multiple reasons that have zero to do with their technical
suitability.

But back to the point of the thread, I think it's ironic that you're
basically making the benchmarks' argument:  Choose the tool which
suits the solution you need.  You argue we should use IronPython or
Jython when threads are needed, and the Julia benchmarks *imply* (but
never actually argue, AFAICT) that if you need a solution which relies
on heavy use of function calls, since Python will be slower than Julia,
you should therefore choose Julia instead.  But that's only a sensible
conclusion if all other things be equal, which of course they are not,
neither for Julia vs. Python, nor for Python vs. Jython or IronPython
(though likely the latter case is slightly closer to being true).

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3.6 fails to install to non-standard directory under Linux (Posting On Python-List Prohibited)

[bsferrazza]

On Friday, March 2, 2018 at 12:27:17 PM UTC-8, Lawrence D’Oliveiro wrote:
> On Saturday, March 3, 2018 at 6:42:05 AM UTC+13, bsfer...@avnera.com wrote:
> 
> > ...
> >   File "/nfs/home/myuser/lfs/sources/Python-3.6.4/Lib/shutil.py", line 476, 
> > in rmtree
> > ...
> 
> NFS trouble?
> 
> I have had builds give trouble when done on directories mounted via NFS, yet 
> work perfectly on local directories.


Hmm. That could very well be the case. I know the /home directory is NFS 
mounted, though not sure about /tmp.

I'm starting to get the impression that everything is actually OK, and that 
these errors are only on the cleanup after installation. Even with the meson 
install. This bugs.python.org/issue23346 bug got me in the right path. When I 
comment out the _use_fd_functions line that ends up setting it to True and 
replace it with a direct assignment to False, everything works ok. The rmtree 
function instead returns _rmtree_unsafe(path, onerror) when _use_fd_functions 
is false and that eliminates the error I was seeing.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to make Python run as fast (or faster) than Julia

[Chris Angelico]

On Sat, Mar 3, 2018 at 7:45 AM, Python  wrote:
> On Mon, Feb 26, 2018 at 09:57:06AM +, Steven D'Aprano wrote:
>> Besides, if you want Python with no GIL so you can run threaded code, why
>> aren't you using IronPython or Jython?
>
> But this is just another oversimplified argument.  In the real world
> there rather often exist constraints which have nothing to do with
> what is technically possible, but which make impractical or infeasible
> what may be technically possible.
>
> Python is often a preferred solution because it is often fantastic for
> rapid implementation and maintainability.  The GIL's interference
> with threaded code performance has, for me at least, on several
> occasions been...  disappointing (perf costs of removing it aside)
> because it gets in the way of choosing Python for such solutions.
> Jython and IronPython are simply not feasible options for me, for
> multiple reasons that have zero to do with their technical
> suitability.

Have you actually tried it and run into problems, or do you succumb to
the FUD and give up before implementing? I can think of only one time
when I tried to speed up a program by multithreading it and was unable
to due to locking. (And actually, it wasn't even the GIL that was the
problem - it was a non-thread-safe library I was calling on, and
*that* meant everything serialized.) "One time, people!" -- Wasabi

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: RFC: Proposal: Deterministic Object Destruction

[Steven D'Aprano]

On Fri, 02 Mar 2018 07:09:19 -0800, ooomzay wrote:

[...]
>> If you're going to *require* the programmer to explicitly del the
>> reference:
>>
>> f = open("file")
>> text = f.read()
>> del f
> 
> But I am not! On the contrary RAII frees the programmer from even having
> to remember to close the file. The poster asked what would happen if the
> resource was deliberately kept open by storing a reference at global
> scope.

You say that as if it were difficult to do, requiring the programmer to 
take extraordinary steps of heroic proportion. It doesn't.

It is unbelievably easy to store the reference at global scope, which 
means that the programmer needs to remember to close the file explicitly 
and RAII doesn't help.


> In practice CPython destroys it cleanly on exit - but I am not sure the
> language guarantees this

Typically the OS guarantees that any open files will be closed when the 
application exits. But that *absolutely does not* apply to other 
resources that may need closing, and Python at least doesn't guarantee to 
run __del__ during application shutdown as it may not be able to.

(It is better now than it used to be, but there are still scenarios where 
the resources needed to run __del__ are gone before __del__ is called.)


> - in any case RAII won't make things any worse
> in this respect. (Logfiles are a common example of such global
> resource.)

I didn't say RAII will make it worse, but neither will it make it better, 
nor  make "with" statements obsolete.

Your justification for requiring RAII is to ensure the timely closure of 
resources -- but to do that, you have to explicitly close or delete the 
resource. It simply isn't true that "RAII frees the programmer from even 
having to remember to close the file".


>> then you might as well require them to explicitly close the file:
>> 
>> f = open("file")
>> text = f.read()
>> f.close()
>> 
>> which we know from many years experience is not satisfactory except for
>> the simplest scripts that don't need to care about resource management.
>> That's the fatal flaw in RAII:
> 
> We must be discussing a different RAII. That is the raison d'etre of
> RAII: RAII directly addresses this problem in an exception-safe way that
> does not burden the resource user at all.

But as you said yourself, if the resource is held open in a global 
reference, it will stay open indefinitely. And remember, global in this 
context doesn't just mean the main module of your application, but 
*every* module you import.

I think you have just put your finger on the difference between what RAII 
*claims* to do and what it *actually* can do.



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Redundant features in python library, PyQt

[jladasky]

Python's standard library has (to take three examples) threads, processes, and  
datetime functions.  Meanwhile, PyQt has QThread, QProcess, and QDateTime.

Does this redundancy exist for C++ programmers who are programming Qt directly, 
and who may lack a standard C++ library with these features?  Are the Python 
wrappers simply provided for completeness, or do these libraries truly function 
differently?  When writing a PyQt application, how does one choose between the 
standard library and the PyQt library?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to make Python run as fast (or faster) than Julia

[Steven D'Aprano]

On Fri, 02 Mar 2018 14:45:55 -0600, Python wrote:

> On Mon, Feb 26, 2018 at 09:57:06AM +, Steven D'Aprano wrote:
>> Besides, if you want Python with no GIL so you can run threaded code,
>> why aren't you using IronPython or Jython?
> 
> But this is just another oversimplified argument.  In the real world
> there rather often exist constraints which have nothing to do with what
> is technically possible, but which make impractical or infeasible what
> may be technically possible.

Of course it is simplified. As is the argument that the GIL is the source 
of all evil, er, why Python threading is slow.

In reality, we may have good reasons for not using IronPython or Jython, 
such as the desire for the latest 3.x version, or because we rely on C 
external libraries that don't work under the CLR or JVM.


[...]
> But back to the point of the thread, I think it's ironic that you're
> basically making the benchmarks' argument:  Choose the tool which suits
> the solution you need.  You argue we should use IronPython or Jython
> when threads are needed, and the Julia benchmarks *imply* (but never
> actually argue, AFAICT) that if you need a solution which relies on
> heavy use of function calls, since Python will be slower than Julia, you
> should therefore choose Julia instead.

Indeed. While I haven't said so explicitly in this thread, I agree 
entirely with everything you say here. One of the things I'm very 
interested in is the possibility of writing number-crunching code in 
Julia and calling it from Python.

I haven't tried it yet, but it is on my bucket-list :-)


> But that's only a sensible
> conclusion if all other things be equal, which of course they are not,
> neither for Julia vs. Python, nor for Python vs. Jython or IronPython
> (though likely the latter case is slightly closer to being true).

There are always constraints on what language you might use. Foo-lang 
might be the best language for the job, but since I don't know the 
language (or even heard of it), I can't use it. You may be constrained by 
familiarity, shop politics, customer requirements, the availability of 
staff, personal preferences, legal requirements, etc.

But taking all of those things into consideration, the conclusion still 
fits: use the language that suits you and the job best. If that language 
is Python, then this is how you can get the best performance if and when 
needed.



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: RFC: Proposal: Deterministic Object Destruction

[Michael Torrie]

On 03/02/2018 08:36 AM, Paul Moore wrote:
> On 2 March 2018 at 15:09,   wrote:
>> We must be discussing a different RAII. That is the raison d'etre of RAII: 
>> RAII directly addresses this problem in an exception-safe way that does not 
>> burden the resource user at all.
> 
> RAII works in C++ (where it was initially invented) because it's used
> with stack-allocated variables that have clearly-defined and limited
> scope. In my experience writing C++, nobody uses RAII with
> heap-allocated variables - those require explicit allocation and
> deallocation and so are equivalent to having an explicit "close()"
> method in Python (or using __del__ in CPython as it currently exists).

Very good point.  I'm not sure the OP considered this when proposing
this idea.

That said, In C++, there are a number of ways of making heap-allocated
objects more RAII-like. These include smart pointers that do reference
counting and reference borrowing.  They work pretty well when used as
designed, but they do require some programmer understanding and
intervention to use correctly and leaks can and do happen.  Smart
pointers are pretty good hacks as far as they go, without a garbage
collector.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Redundant features in python library, PyQt

[Michael Torrie]

On 03/02/2018 04:15 PM, jlada...@itu.edu wrote:
> Python's standard library has (to take three examples) threads,
> processes, and  datetime functions. Meanwhile, PyQt has QThread,
> QProcess, and QDateTime.
> 
> Does this redundancy exist for C++ programmers who are programming Qt
> directly, and who may lack a standard C++ library with these
> features?  Are the Python wrappers simply provided for completeness,
> or do these libraries truly function differently?  When writing a
> PyQt application, how does one choose between the standard library
> and the PyQt library
Yes, when using Qt in C++, you'll be using QThread, QProcess, QDateTime,
etc, even though there are equivalents in the C++ standard library. This
is largely because Qt was created long before the C++ standard library
had many of these things.  But it's also because Qt's versions are
tightly integrated with the Qt object and event machinery.  A QThread
sets up the necessary message queues, for example, whereas a STL thread
will not.

When programming in PyQt, you will also want to use the Qt versions of
primitives when they exist. The primary reason is that the Qt versions
integrate with the Qt event-handling machinery.  As Qt is a C++ library,
it's necessary to wrap each and every Qt class with a Python class.
Thus PyQt often really is just C++ with a Python dialect. In other words
often times you'll be using C++ idioms and C++ data structures even
though Python's idioms and versions would be cleaner and more flexible.
This is the price you pay for using a C++ library that's wrapped using
automated tools for use in Python.  So you'll find api calls that return
a QString, for example, when a python string would be preferred.  Or a
QArray when a list would be better.

This is not just a PyQt thing. Any time you have thin wrappers around C
or C++ libraries, the underlying language idioms are going to bleed
through a bit into Python.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: RFC: Proposal: Deterministic Object Destruction

[Chris Angelico]

On Sat, Mar 3, 2018 at 11:58 AM, Michael Torrie  wrote:
> On 03/02/2018 08:36 AM, Paul Moore wrote:
>> On 2 March 2018 at 15:09,   wrote:
>>> We must be discussing a different RAII. That is the raison d'etre of RAII: 
>>> RAII directly addresses this problem in an exception-safe way that does not 
>>> burden the resource user at all.
>>
>> RAII works in C++ (where it was initially invented) because it's used
>> with stack-allocated variables that have clearly-defined and limited
>> scope. In my experience writing C++, nobody uses RAII with
>> heap-allocated variables - those require explicit allocation and
>> deallocation and so are equivalent to having an explicit "close()"
>> method in Python (or using __del__ in CPython as it currently exists).
>
> Very good point.  I'm not sure the OP considered this when proposing
> this idea.
>
> That said, In C++, there are a number of ways of making heap-allocated
> objects more RAII-like. These include smart pointers that do reference
> counting and reference borrowing.  They work pretty well when used as
> designed, but they do require some programmer understanding and
> intervention to use correctly and leaks can and do happen.  Smart
> pointers are pretty good hacks as far as they go, without a garbage
> collector.

Ref-counting smart pointers? They might save you the trouble of
calling (the equivalents of) Py_INCREF and Py_DECREF manually, but
they still don't solve reference loops, and they don't deal with the
termination cleanup problem. Smart pointers don't change heap
allocation semantics and don't remove the garbage collector; all they
do is change the way you do refcounting. (Which is not a bad thing,
btw. I've used smart pointers, and they can be very helpful. But
they're no panacea.)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: RFC: Proposal: Deterministic Object Destruction (Posting On Python-List Prohibited)

[Gregory Ewing]

Paul Rubin wrote:

So you want the programmer to put more head scratching into figuring out
which reference should be strong and which should be weak?


Also, sometimes weak references don't really solve the
problem, e.g. if you have a graph where you can't identify
any particular node as a "root" node, and you want the graph
to stay around as long as any of its nodes are referenced.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

I'm not seeing Paul Rubin's posts

[Steven D'Aprano]

I see people replying to Paul Rubin, but I'm not seeing his posts.

I've reading this through gmane.

Does anyone know what's going on?

Thanks,



Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: I'm not seeing Paul Rubin's posts

[Chris Angelico]

On Sat, Mar 3, 2018 at 4:42 PM, Steven D'Aprano
 wrote:
> I see people replying to Paul Rubin, but I'm not seeing his posts.
>
> I've reading this through gmane.
>
> Does anyone know what's going on?

I'm not seeing them either. Usually that means someone's been banned
from the mailing list, or else has self-banned by saying "don't
archive". Which, to me, seems like going out in public and talking to
people, and then saying "But that was off the record, so make sure you
forget that I said that".

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list