[Python-Dev] Re: When we remove 'U' mode of open()?

[Victor Stinner]

That sounds reasonable ;-)

Victor

On Thu, Apr 8, 2021 at 3:02 AM Inada Naoki  wrote:
>
> We are close to 3.10 beta and it is not ideal timing for removing.
> So my proposal is:
>
> * Remove 'U' in fileinput, because it makes my task little simpler.
> * Remove 'U' in other places in Python 3.11, after 3.10 branch is
> created (and master branch is renamed to main).
>
> On Thu, Apr 8, 2021 at 5:45 AM Brett Cannon  wrote:
> >
> >
> >
> > On Wed, Apr 7, 2021 at 10:01 AM Serhiy Storchaka  
> > wrote:
> >>
> >> 07.04.21 19:13, Victor Stinner пише:
> >> > Hi Inada-san,
> >> >
> >> > I'm +0 on removing again the flag, but I would prefer to not endorse
> >> > the responsibility. I am already responsible for enough incompatible
> >> > changes in Python 3.10 :-D
> >> >
> >> > Some context on this "U" open mode. The flag is accepted by many
> >> > functions opening files. It is deprecated (emit DeprecationWarning)
> >> > for 9 years (Python 3.3, 2012).
> >>
> >> It was silently deprecated before 3.3 (perhaps it was no-op since 3.0).
> >>
> >> I added DeprecationWarning with intention to remove this option in all
> >> functions accepting it. The only non-trivial support of the "U" mode was
> >> left in ZipFile.open(), and it was broken since beginning.
> >
> >
> > I think at this point the DeprecationWarning has definitely been on long 
> > enough, there was an explicit warning about it in Python 3.9, and 3.10 will 
> > be nearly 2 years removed from 2.7 reaching EOL which is the only place 
> > where "U" may still be used. So I think it's fine to drop "U" in 3.10.
> > ___
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-le...@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at 
> > https://mail.python.org/archives/list/python-dev@python.org/message/VTROKN5UOU3EN6F3OLX5RUK7TVETAXKB/
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> --
> Inada Naoki  
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/NYVORKRSH562UMAXXLSJOOW5ECBA3HC5/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/633HYC4O3TCOZARWXI6JCDFO2SBZHXRT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: How to attract developer attention to issue tracker items?

[pjfarley3]

Thanks Brett.  I guess I will just have to wait and see.

 

Peter

 

From: Brett Cannon  
Sent: Wednesday, April 7, 2021 4:37 PM
To: pjfarl...@earthlink.net
Cc: python-dev 
Subject: Re: [Python-Dev] How to attract developer attention to issue tracker 
items?

 

 

 

On Tue, Apr 6, 2021 at 11:22 PM mailto:pjfarl...@earthlink.net> > wrote:

Hi developers,

What should / shouldn't I do to attract any python developer response to
issue tracker items?  I am unsure of the proper procedure to follow so I am
asking here first.

 

What you're doing here is probably your best bet. Unfortunately the lack of 
response might stem from no core devs being interested in curses (but if any 
are then hopefully this email will spark their interest).

 

-Brett

 


I have filed two issues on the python issue tracker for python curses
issues:

1.  # 43715, a documentation update request (suggested additional
documentation file uploaded)
2.  # 43716, a possible ABI issue for the curses.pair_number() function
return value in a Windows environment

With respect to the documentation issue, I would appreciate any review of
whether the suggested documentation addition that I uploaded as a text file
is acceptable as-is or whether further refinement or correction is needed.

In the case of the possible ABI issue I am uncertain whether the issue is in
the python interface code or in the underlying curses implementation library
(windows-curses V2.2.0 using PDCurses).  I would appreciate it if anyone
knowledgeable of the python curses interfacing code could advise me whether
I need to pursue the issue with the third party library provider or with the
python development team.  At the moment all that I know is that the
"workaround" that I discovered corrects the issue under Windows but is not
necessary under linux / ncurses (in my testing I used Ubuntu 20.04 WSL2 to
confirm that fact).

TIA for your patience with my ignorance of the proper procedures and
channels.

My environment:

Windows 10 Pro 64 (latest version)
Python 3.8.9 (tags/v3.8.9:a743f81, Apr  6 2021, 14:02:34) [MSC v.1928 64 bit
(AMD64)]
Windows-curses V2.2.0

Peter
--

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7BQV7KOTMUJUDNYCWVAP64T6ZQYBLXDO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Help to Resolve issues with Pull request 25220

[Tony Flury via Python-Dev]

Terry, Barney,

I think I have done it right now - I have no idea what went wrong last 
time. I fixed it by recreating the changes from scratch, and 
scrupulously checking for the differences.


Thanks for your help - although ultimately I didn't try to rebase.

--


On 08/04/2021 01:32, Terry Reedy wrote:

On 4/7/2021 12:32 PM, Barney Gale wrote:
It looks like you’ve incorporated several other changes into your 
commit by mistake.
The PR definitely has too many changes unrelated to the issue.  I 
recognize a few of the changes as related to recent merges.


  My guess that the the issue-43737 branch was branched off of 
something other than master, which is usually a fatal mistake. The 
parent branch may also have been branched off of something other than 
master.


Tony, with rare exception, each branch should be branched directly off 
master, which means checking out master first each time.


There is probably some other bad sequence of events that could cause 
the issue.


 The simplest thing to do might be to re-create your git

branch and PR from scratch.


I agree with this.  That means updating the master branch, creating a 
new 43737 branch, make the 43737 changes, commit, and push.


Or, in case problem is elsewise, fix the merge conflict (below), 
merge, and push to your fork.  Update your master from upstream, 
change to branch, git merge upstream/master, commit to branch, push 
branch to fork, and see how things work.


If conflicting changes land while your PR is still open, you’ll 

need to

do something called a “rebase”.


Rebasing (perhaps done incorrectly) often makes things worse.  One 
fixes a merge conflix by editing the PR branch file(s) with the 
conflict and committing a change.


However, the merge conflict in expression.rst is this:

<<< issue-43737
.. productionlist::
   lambda_expr: "lambda" [`parameter_list`]: `expression`
   lambda_expr_nocond: "lambda" [`parameter_list`]: `expression_nocond`
===
.. productionlist:: python-grammar
   lambda_expr: "lambda" [`parameter_list`] ":" `expression`
>>> master

Adding a grammar production has nothing to do with the issue. This 
could be easily fixed by removing the lambda_expr_nocond line (and 
other junk if not editing in the local branch) but would not solve the 
more extensive issues.



___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/N6LJVPHGCECDYI7R45DLHT5JCMZGDHV2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 647 Accepted

[Federico Salerno]

I vaguely remember reading some discussion about Annotated and, for what 
little it's worth, I disagree with the consensus.


For now all that comes to mind is either stuffing the information in 
docstrings (which would be kind of an involution, seeing as that was the 
way for annotating type before type annotations), or changing the name 
of the type to something more obvious at a glance. Something like 
Typeguards["param", T], if arbitrary strings could be included, which 
would have to correspond to parameter names. This would give a few extra 
advantages:
- Multiple parameters could be annotated per function if needed (e.g. 
def my_function(param1, param2) -> Typeguards["param1", T, "param2", 
U]). An alternative syntax could be Typeguards[["param1", T], ["param2", U]]
- Functions with additional parameters that the function uses but which 
are not typeguarded could be used, and there would be no ambiguity as to 
which parameter is affected by the typeguard: def my_function(param1: 
Any, param2: bool) -> Typeguards["param1", T]


Personally, I like the idea of functions that /happen/ to restrict the 
type of a variable without it necessarily being their only or primary 
goal; this would enable those.


Possible alternative names: Ensures["param1", T], Ensure["param1", T], 
IsType["param1", T]. It's important to remember that jargon creates a 
potential hurdle to comprehension, so simpler language should, in my 
opinion, always be preferred.


Side-note: | is being introduced for union types in type annotations. 
Could it be an idea to add a & as well, to signal side-effects of the 
function that do not interfere with the return value? A function could 
then have a signature like:

- def my_function(param1, param2) -> bool & Typeguards["param1", T]
- def my_function(param1) -> int & Typeguards[T]  # Only one param, so 
"param1" is optional


I realise it would make signatures (optionally!) longer and arguably 
uglier, but the latter is mostly a matter of personal taster and of 
getting used to, which is always there when new features are introduced. 
I remember := being controversial, but it grew on me over time.


I hope there's at least something worth considering in the above ideas.


On 07/04/2021 21:59, gu...@python.org wrote:
But it isn't a "side effect". It is a distinct concept that is 
important to the type checker.


Note that in TypeScript this also doesn't look like a boolean -- it 
uses a unique syntax that has to be learned:


function isCustomer(partner: any): partner is Customer {
    . . .
}

Arguably the TS syntax is more easily intuited without looking it up, 
but TS has a certain freedom in its syntactic design that we don't 
have for Python: new *syntax* has to be added to the Python parser and 
can't be backported, whereas new *types* (like `TypeGuard[T]`) can 
easily be backported via typing_extensions.py.


We have really tried, but we did not come up with anything better than 
the current PEP.


FWIW you might be interested in Annotated (PEP 593), which can be used 
to indicate various attributes of a type annotation. Before you 
suggest that we adopt that instead of PEP 647, we considered that, and 
the consensus is that that's not what Annotated is for (it's intended 
for conveying information to tools *other* than the type checker, for 
example schema checkers etc.).


--
--Guido van Rossum (python.org/~guido )
/Pronouns: he/him //(why is my pronoun here?)/ 

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AMAXAPIZIHW2IATRQDQYWMKTRZAWS6W3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] NamedTemporaryFile and context managers

[Ethan Furman]

In issue14243 [1] there are two issues being tracked:

- the difference in opening shared files between posix and Windows
- the behavior of closing the underlying file in the middle of
  NamedTemporaryFile's context management

I'd like to address and get feedback on the context management issue.

```python
from tempfile import NamedTemporaryFile

with NamedTemporaryFile() as fp:
fp.write(b'some data')
fp = open(fp.name())
data = fp.read()

assert data == 'some_data'
```

Occasionally, it is desirable to close and reopen the temporary file in order to read the contents (there are OSes that 
cannot open a temp file for reading while it is still open for writing).  This would look like:


```python
from tempfile import NamedTemporaryFile

with NamedTemporaryFile() as fp:
fp.write(b'some data')
fp.close()  # Windows workaround
fp.open()
data = fp.read()

assert data == 'some_data'
```

The problem is that, even though `fp.open()` is still inside the context manager, the `close()` call deletes the file 
[2].  To handle this scenario, my proposal is two-fold:


1) stop using the TEMPFILE OS attribute so the OS doesn't delete the file on 
close
2) add `.open()` to NamedTemporaryFile

A possible side effect of (1) is that temp files may accumulate if the interpreter crashes, but given the 
file-management abilities in today's software that seems like a minor annoyance at most.


The backwards compatibility issue of (1) is that the file is no longer deleted after a manual `close()` -- but why one 
would call close() and then stay inside the CM, outside of testing, I cannot fathom.  [3]


So, opinions on modifying NamedTemporaryFile to not delete on close() if inside 
a CM, and add open() ?

--
~Ethan~


[1] https://bugs.python.org/issue14243
[2] plus, the `.open()` doesn't yet exist
[3] feel free to educate me  :-)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DZNEIRHZ7LVQNQRQ2JL5B4AOC3HH3B6O/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: NamedTemporaryFile and context managers

[Antoine Pitrou]

On Thu, 8 Apr 2021 13:31:26 -0700
Ethan Furman  wrote:
> 
> ```python
> from tempfile import NamedTemporaryFile
> 
> with NamedTemporaryFile() as fp:
>  fp.write(b'some data')
>  fp.close()  # Windows workaround
>  fp.open()
>  data = fp.read()
> 
> assert data == 'some_data'
> ```
> 
> The problem is that, even though `fp.open()` is still inside the context 
> manager, the `close()` call deletes the file 
> [2].  To handle this scenario, my proposal is two-fold:
> 
> 1) stop using the TEMPFILE OS attribute so the OS doesn't delete the file on 
> close
> 2) add `.open()` to NamedTemporaryFile

Instead, you could add a dedicated `.reopen()`?

Regards

Antoine.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HUGSC2FB6VW3VFKGDYKUYUKNJEWNPF5T/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: NamedTemporaryFile and context managers

[Ronald Oussoren via Python-Dev]

> On 8 Apr 2021, at 22:31, Ethan Furman  wrote:
> 
> In issue14243 [1] there are two issues being tracked:
> 
> - the difference in opening shared files between posix and Windows
> - the behavior of closing the underlying file in the middle of
>  NamedTemporaryFile's context management
> 
> I'd like to address and get feedback on the context management issue.
> 
> ```python
> from tempfile import NamedTemporaryFile
> 
> with NamedTemporaryFile() as fp:
>fp.write(b'some data’)
  fp.flush(). # needed to ensure data is actually in the file ;-)
>fp = open(fp.name())
>data = fp.read()
> 
> assert data == 'some_data'
> ```

I generally use a slightly different pattern for this:

with NamedTemporaryFile() as fp:
   fp.write(b'some data’)
   fp.flush()
   fp.seek(0)
   data = fp.read()

That is, reuse the same “fp” and just reset the stream.  An advantage of this 
approach is that you don’t need a named temporary file for this (and could even 
use a spooled one).   That said, I at times use this pattern with a named 
temporary file with a quick self-test for the file contents before handing of 
the file name to an external proces.


> 
> Occasionally, it is desirable to close and reopen the temporary file in order 
> to read the contents (there are OSes that cannot open a temp file for reading 
> while it is still open for writing).  This would look like:
> 
> ```python
> from tempfile import NamedTemporaryFile
> 
> with NamedTemporaryFile() as fp:
>fp.write(b'some data')
>fp.close()  # Windows workaround
>fp.open()
>data = fp.read()
> 
> assert data == 'some_data'
> ```
> 
> The problem is that, even though `fp.open()` is still inside the context 
> manager, the `close()` call deletes the file [2].  To handle this scenario, 
> my proposal is two-fold:
> 
> 1) stop using the TEMPFILE OS attribute so the OS doesn't delete the file on 
> close
> 2) add `.open()` to NamedTemporaryFile

I’ve never had the need for such an API, but must say that I barely use Windows 
and hence have not run into the “cannot open file for reading what it is still 
open for writing” issue.

> 
> A possible side effect of (1) is that temp files may accumulate if the 
> interpreter crashes, but given the file-management abilities in today's 
> software that seems like a minor annoyance at most.
> 
> The backwards compatibility issue of (1) is that the file is no longer 
> deleted after a manual `close()` -- but why one would call close() and then 
> stay inside the CM, outside of testing, I cannot fathom.  [3]
> 
> So, opinions on modifying NamedTemporaryFile to not delete on close() if 
> inside a CM, and add open() ?
> 
> --
> ~Ethan~
> 

Ronald

—

Twitter / micro.blog: @ronaldoussoren
Blog: https://blog.ronaldoussoren.net/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LOE7BKIHV66IG3J6LP3UAJSZL65AXWCK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: NamedTemporaryFile and context managers

[Ethan Furman]

On 4/8/21 1:43 PM, Antoine Pitrou wrote:

On Thu, 8 Apr 2021 13:31:26 -0700
Ethan Furman  wrote:


```python
from tempfile import NamedTemporaryFile

with NamedTemporaryFile() as fp:
  fp.write(b'some data')
  fp.close()  # Windows workaround
  fp.open()
  data = fp.read()

assert data == 'some_data'
```

The problem is that, even though `fp.open()` is still inside the context 
manager, the `close()` call deletes the file
[2].  To handle this scenario, my proposal is two-fold:

1) stop using the TEMPFILE OS attribute so the OS doesn't delete the file on 
close
2) add `.open()` to NamedTemporaryFile


Instead, you could add a dedicated `.reopen()`?


The main hurdle is that on Windows we let the OS manage the lifetime of the file, which means that it is deleted as soon 
as it is closed.  We would need to remove that branch and treat all NamedTemporaryFiles the same.


We could add reopen(), but since close() is already there... although I do like 
the name of `reopen`.

--
~Ethan~
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2DGIFUS6SU56MP6OWOEJPSWEDR2PL5CS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Request for reviewers: pathlib improvements

[Barney Gale]

Thanks so much to everyone who discussed and reviewed the code and made
suggestions.

The bulk of these patches have now landed. For those following along at
home, here’s a summary of the remaining PRs/bugs :

   - #18909  / bpo-39950
   : Add Path.hardlink_to() method that
   supersedes link_to()
  - Note: already under core review
   - #25271  / bpo-42998
   : Add ‘user’ parameter to Path.home()
   - #25240  / bpo-40107
   : Stop using os.open() to implement
   Path.open()
   - #25264  / bpo-43757
   : Make Path.resolve() use
   os.path.realpath() to resolve all symlinks in a path
  - Note: changes realpath() behaviour when symlink loops are
  encountered (!), and adds a keyword-only 'strict' param.
   - #25277  / bpo-39899
   : Follow-up fix for oversight in
   #18841 

I believe this is a complete account of the outstanding prep work. Once
these land, pathlib’s *path*, *flavour* and *accessor* abstractions will be
less leaky, which allows work on pathlib.AbstractPath to proceed.

Thanks again to Antoine Pitrou, Brett Cannon, Serhiy Storchaka, Steve Dower
and Eryk Sun for their invaluable contributions.

Barney

On Sat, 3 Apr 2021 at 03:02, Barney Gale  wrote:

> I’m working towards supporting subclassing pathlib.Path for things like
> zip/tar archives, iso files, S3, etc. This was first brought up a few years
> ago in bpo-24132  and discussed
> further on the python-ideas forum
> .
>
> Before I can approach the meat of the issue, I'd like to land a handful of
> backwards-compatible patches that tidy up the pathlib internals. These are:
>
>- #19220  / bpo-39924
>: Handle missing os functions more
>consistently in pathlib
>- #18838  / bpo-39895
>: Move pathlib.Path.touch()
>implementation into the path accessor
>- #19342  / bpo-40038
>: Remove partial support for
>preserving accessor when modifying a path
>- #18841  / bpo-39899
>: Make pathlib use
>os.path.expanduser() to expand home directories
>- #18834  / bpo-39659
>: Route calls from pathlib.Path to
>os.getcwd() via the path accessor
>- #18864  / bpo-39906
>: Add follow_symlinks parameter to
>pathlib.Path.stat() and chmod()
>- #24294  / bpo-42999
>: Expand and clarify
>pathlib.Path.link_to() documentation
>
> Could I please request review of these patches? A couple may already be
> ready to land.
>
> (I appreciate they’re a little dry. This is all prep work for the
> introduction of a shiny new class, tentatively named ‘AbstractPath’).
>
> Thanks!
>
> Barney
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XTXOUBCTLFO6AQBL66AM37QAVTXJ4YND/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: NamedTemporaryFile and context managers

[Rob Cliffe via Python-Dev]

Well this works:

from tempfile import NamedTemporaryFile
import os

with NamedTemporaryFile(delete=False) as fp:
  fp.write(b'some data')
  fp.close()
  with open(fp.name, 'rb') as fp2:
    data = fp2.read()
  os.remove(fp.name)
assert data == b'some data'

Of course it is theoretically possible that another app will do 
something to the temporary file between the "fp.close()" and the "open" 
on the next line, or between closing fp2 and the "os.remove".  Is this a 
concern?

Rob Cliffe

On 08/04/2021 22:42, Ethan Furman wrote:

On 4/8/21 1:43 PM, Antoine Pitrou wrote:

On Thu, 8 Apr 2021 13:31:26 -0700
Ethan Furman  wrote:


```python
from tempfile import NamedTemporaryFile

with NamedTemporaryFile() as fp:
  fp.write(b'some data')
  fp.close()  # Windows workaround
  fp.open()
  data = fp.read()

assert data == 'some_data'
```

The problem is that, even though `fp.open()` is still inside the 
context manager, the `close()` call deletes the file

[2].  To handle this scenario, my proposal is two-fold:

1) stop using the TEMPFILE OS attribute so the OS doesn't delete the 
file on close

2) add `.open()` to NamedTemporaryFile


Instead, you could add a dedicated `.reopen()`?


The main hurdle is that on Windows we let the OS manage the lifetime 
of the file, which means that it is deleted as soon as it is closed.  
We would need to remove that branch and treat all NamedTemporaryFiles 
the same.


We could add reopen(), but since close() is already there... although 
I do like the name of `reopen`.


--
~Ethan~
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2DGIFUS6SU56MP6OWOEJPSWEDR2PL5CS/

Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5E5PTFW2BXU33V7LGVKMRD5I2425OT6G/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: NamedTemporaryFile and context managers

[Eric V. Smith]

On 4/8/2021 4:43 PM, Antoine Pitrou wrote:

On Thu, 8 Apr 2021 13:31:26 -0700
Ethan Furman  wrote:

```python
from tempfile import NamedTemporaryFile

with NamedTemporaryFile() as fp:
  fp.write(b'some data')
  fp.close()  # Windows workaround
  fp.open()
  data = fp.read()

assert data == 'some_data'
```

The problem is that, even though `fp.open()` is still inside the context 
manager, the `close()` call deletes the file
[2].  To handle this scenario, my proposal is two-fold:

1) stop using the TEMPFILE OS attribute so the OS doesn't delete the file on 
close
2) add `.open()` to NamedTemporaryFile

Instead, you could add a dedicated `.reopen()`?


I think capturing the intent is important, rather than just putting 
.close() followed by .open(). Maybe a name that embodies "reopen for 
reading without deleting" would make the intent clear? Maybe .reopen() 
already captures that. Anyway, +1 for a method that combines the close/open.


Eric

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4SZVHI6FEHAWBMGI5KUFYUOTTGQQACN5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: NamedTemporaryFile and context managers

[Ivan Pozdeev via Python-Dev]

On 08.04.2021 23:31, Ethan Furman wrote:

In issue14243 [1] there are two issues being tracked:

- the difference in opening shared files between posix and Windows
- the behavior of closing the underlying file in the middle of
 NamedTemporaryFile's context management

I'd like to address and get feedback on the context management issue.

```python
from tempfile import NamedTemporaryFile

with NamedTemporaryFile() as fp:
   fp.write(b'some data')
   fp = open(fp.name())
   data = fp.read()

assert data == 'some_data'
```

Occasionally, it is desirable to close and reopen the temporary file in order to read the contents (there are OSes that cannot open a temp 
file for reading while it is still open for writing). This would look like:




What's the problem with `NamedTemporaryFile(mode='w+b')`? (it's actually the 
default!)

You can both read and write without reopening.
If you don't want to seek() between reading and writing, you can dup() the descriptor and wrap it with another file object: 
`fp2=open(os.dup(fp.file.fileno()))`




```python
from tempfile import NamedTemporaryFile

with NamedTemporaryFile() as fp:
   fp.write(b'some data')
   fp.close()  # Windows workaround
   fp.open()
   data = fp.read()

assert data == 'some_data'
```

The problem is that, even though `fp.open()` is still inside the context manager, the `close()` call deletes the file [2].  To handle this 
scenario, my proposal is two-fold:


1) stop using the TEMPFILE OS attribute so the OS doesn't delete the file on 
close
2) add `.open()` to NamedTemporaryFile

A possible side effect of (1) is that temp files may accumulate if the interpreter crashes, but given the file-management abilities in 
today's software that seems like a minor annoyance at most.


The backwards compatibility issue of (1) is that the file is no longer deleted after a manual `close()` -- but why one would call close() 
and then stay inside the CM, outside of testing, I cannot fathom.  [3]


So, opinions on modifying NamedTemporaryFile to not delete on close() if inside 
a CM, and add open() ?

--
~Ethan~


[1] https://bugs.python.org/issue14243
[2] plus, the `.open()` doesn't yet exist
[3] feel free to educate me  :-)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DZNEIRHZ7LVQNQRQ2JL5B4AOC3HH3B6O/
Code of Conduct: http://python.org/psf/codeofconduct/


--
Regards,
Ivan

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SW4LRGEXYER6HDE3WBIKSQHENQRZLMRP/
Code of Conduct: http://python.org/psf/codeofconduct/