[Python-Dev] Clarify the compatibility policy of lib2to3.

[Dong-hee Na]

Hi, 

Few days ago, I submitted a patch(https://github.com/python/cpython/pull/4417) 
which updates2to3 converts `operator.isCallable(obj)` to `isinstance(obj, 
collections.abc.Callable)`.

This was Serhiy Storchaka’s idea(https://bugs.python.org/issue32046) and I 
agree with his idea since `callable` is not available in all 3.x versions.

However, some people would like to clarify the policy of 2to3.
That means that 2to3 is compatible with the branch that is currently 
maintained? or it is converted to specific Python 3.X codes with the specific 
version of 2to3?

Cordially,
Dong-hee
—
Dong-hee Na
Chungnam National University | Computer Science & Engineering

Tel: +82 010-3353-9127
Email: donghee.n...@gmail.com
Linkedin: https://www.linkedin.com/in/dong-hee-na-2b713b49/
Github: https://github.com/corona10

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Remove formatter module

[Dong-hee Na]

A few days ago, Terry J. Reedy created the issue about the formatter module.
(https://bugs.python.org/issue42299)

The issue is mainly about testing code but also discussing the removal of
the module.
I noticed that the formatter module was deprecated in 3.4 and it was
originally scheduled to be removed in 3.6. But the removal was delayed
until 2.7 EOL.

Now 2.7 is EOL and there is no usage in stdlib.
So IMHO, it is good to remove the formatter module in 3.10

So before removing it, I would like to hear the concern.

Regards,
Dong-hee
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZEDIBBYCWI34GVOXDEUYXQY3LYXOFHA2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Remove formatter module

[Dong-hee Na]

Thanks everybody

I will merge this PR until tomorrow ;)

2020년 11월 24일 (화) 오후 10:08, Stéfane Fermigier 님이 작성:

>
>
> On Tue, Nov 24, 2020 at 12:51 PM Victor Stinner 
> wrote:
>
>>
>> I never used this module, I don't know what it is.
>>
>
> I've run a quick search on GitHub and the only meaningful reference I
> could find is the Grail browser (which had its last release, AFAICT, in
> 1999).
>
> http://grail.sourceforge.net/
>
>   S.
>
> --
> Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier
> - http://linkedin.com/in/sfermigier
> Founder & CEO, Abilian - Enterprise Social Software -
> http://www.abilian.com/
> Chairman, National Council for Free & Open Source Software (CNLL) -
> http://cnll.fr/
> Founder & Organiser, PyParis & PyData Paris - http://pyparis.org/ &
> http://pydata.fr/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RPKOFZ6NZIIIF665UBVD3PKD4XUMJMSN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 670: Convert macros to functions in the Python C API

[Dong-hee Na]

mplified code)::
>
> #ifdef COUNT_ALLOCS
> #  define _Py_INC_TPALLOCS(OP) inc_count(Py_TYPE(OP))
> #  define _Py_COUNT_ALLOCS_COMMA  ,
> #else
> #  define _Py_INC_TPALLOCS(OP)
> #  define _Py_COUNT_ALLOCS_COMMA
> #endif /* COUNT_ALLOCS */
>
> #define _Py_NewReference(op) (   \
> _Py_INC_TPALLOCS(op) _Py_COUNT_ALLOCS_COMMA  \
> Py_REFCNT(op) = 1)
>
> Python 3.8 function (simplified code)::
>
> static inline void _Py_NewReference(PyObject *op)
> {
> _Py_INC_TPALLOCS(op);
> Py_REFCNT(op) = 1;
> }
>
> PyObject_INIT()
> ---
>
> Example showing the usage of commas in a macro.
>
> Python 3.7 macro::
>
> #define PyObject_INIT(op, typeobj) \
> ( Py_TYPE(op) = (typeobj), _Py_NewReference((PyObject *)(op)),
> (op) )
>
> Python 3.8 function (simplified code)::
>
> static inline PyObject*
> _PyObject_INIT(PyObject *op, PyTypeObject *typeobj)
> {
> Py_TYPE(op) = typeobj;
> _Py_NewReference(op);
> return op;
> }
>
> #define PyObject_INIT(op, typeobj) \
> _PyObject_INIT(_PyObject_CAST(op), (typeobj))
>
> The function doesn't need the line continuation character. It has an
> explicit ``"return op;"`` rather than a surprising ``", (op)"`` at the
> end of the macro.  It uses one short statement per line, rather than a
> single long line. Inside the function, the *op* argument has a well
> defined type: ``PyObject*``.
>
>
> References
> ==
>
> * `bpo-45490 <https://bugs.python.org/issue45490>`_:
>   [meta][C API] Avoid C macro pitfalls and usage of static inline
>   functions (October 2021).
> * `What to do with unsafe macros
>   <https://discuss.python.org/t/what-to-do-with-unsafe-macros/7771>`_
>   (March 2021).
> * `bpo-43502 <https://bugs.python.org/issue43502>`_:
>   [C-API] Convert obvious unsafe macros to static inline functions
>   (March 2021).
>
>
> Copyright
> =
>
> This document is placed in the public domain or under the
> CC0-1.0-Universal license, whichever is more permissive.
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/2GN646CGWGTO6ZHHU7JTA5XWDF4ULM77/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Software Development Engineer at Line corp.

Tel: +82 10-3353-9127
Email: donghee.n...@gmail.com | donghee...@python.org |
donghee...@linecorp.com
Linkedin: https://www.linkedin.com/in/dong-hee-na-2b713b49/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JMT7TH37GMUKJNHVCZ7RXXPTPAXKQ2PG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Discussion about crash tolerance feature for gdbm module

[Dong-hee Na]

Hi folks,
>From gdbm 1.21, gdbm supports the crash tolerance feature.
see: https://www.gnu.org.ua/software/gdbm/manual/Crash-Tolerance.html

I would like to introduce this feature since python standard library is the
only gdbm binding library that is available for end-users.
And this is also effort for end-users to provide the latest new features.
If this feature is added following methods will be added.
  - 'x' flag will be added for extended gdbm format.
  - gdbm.gdbm_failure_atomic('path1', 'path2') API will be added for
snapshot paths.
  - gdbm.gdbm_latest_snapshot('path1', 'path2') API will be added for
getting latest valid snapshot path.

The above APIs will be used for people who need to recover their gdbm file
if disaster situations happen.
However, this feature is not yet decided to land to CPython.
and we have already discussed this issue at
https://bugs.python.org/issue45452
but I would like to hear other devs opinions.

I cc the original authors of this feature and Serhiy who thankfully already
participated in this discussion too :)

Warm Regards,
Dong-hee
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RFICESJKJW3KZSHYSU4R7UEPUUAJL2B3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Discussion about crash tolerance feature for gdbm module

[Dong-hee Na]

Hello Victor :)

> Do you ask us our opinion on the Python API that you propose? Or if
the whole feature is worth it?

Latter must be done first :)


> Can you please show me a short full example opening a database with
automatic snapshot recovery (with your proposed API)?

The following codes are demonstrating how the user sets the gdbm file to be
able to use the crash tolerance feature.

import dbm.gnu as dbm

db = dbm.open('x.db', 'nx') # For extension format
db.gdbm_failure_atomic('even_snapshot.bin', 'odd_snapshot.bin') # For
snapshot declaration
for k, v in zip('abcdef', 'ghijkl'):
db[k] = v

db.sync()
db.close()


The recovery task will be done separately because this situation is a
very special case.

import dbm.gnu as dbm

latest_snapshot = dbm.gdbm_latest_snapshot('even_snapshot.bin',
'odd_snapshot.bin')

db = dbm.open(latest_snapshot, 'r') # Open the latest valid snapshot

# Do what user want, os.rename(latest_snapshot, 'x.db') whatever.


Warm regards,

Dong-hee


2022년 1월 18일 (화) 오후 6:57, Victor Stinner 님이 작성:

> Hi Dong-hee,
>
> Can you please show me a short full example opening a database with
> automatic snapshop recovery (with your proposed API)?
>
> Do you ask us our opinion on the Python API that you propose? Or if
> the whole feature is worth it?
>
> Victor
>
> On Tue, Jan 18, 2022 at 2:41 AM Dong-hee Na  wrote:
> >
> > Hi folks,
> > From gdbm 1.21, gdbm supports the crash tolerance feature.
> > see: https://www.gnu.org.ua/software/gdbm/manual/Crash-Tolerance.html
> >
> > I would like to introduce this feature since python standard library is
> the only gdbm binding library that is available for end-users.
> > And this is also effort for end-users to provide the latest new features.
> > If this feature is added following methods will be added.
> >   - 'x' flag will be added for extended gdbm format.
> >   - gdbm.gdbm_failure_atomic('path1', 'path2') API will be added for
> snapshot paths.
> >   - gdbm.gdbm_latest_snapshot('path1', 'path2') API will be added for
> getting latest valid snapshot path.
> >
> > The above APIs will be used for people who need to recover their gdbm
> file if disaster situations happen.
> > However, this feature is not yet decided to land to CPython.
> > and we have already discussed this issue at
> https://bugs.python.org/issue45452
> > but I would like to hear other devs opinions.
> >
> > I cc the original authors of this feature and Serhiy who thankfully
> already participated in this discussion too :)
> >
> > Warm Regards,
> > Dong-hee
> > ___
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-le...@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/RFICESJKJW3KZSHYSU4R7UEPUUAJL2B3/
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> --
> Night gathers, and now my watch begins. It shall not end until my death.
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YFDEPBDWRF35DWFSA4ZE6N2FOCLAUO2U/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Discussion about crash tolerance feature for gdbm module

[Dong-hee Na]

I exchanged a mail for investigating details.

Before getting started, please remind the following fact.
- The latest snapshot is always valid, even if corruption has not occurred.

> Why is a separated script needed? / A single script cannot automatically
detect a corrupted database and load the latest snapshot?

So, the author said a separate script is never needed,
if the user reads from the latest snapshot file, it will always be
recovered.
so the user code will be like this.

import dbm.gnu as dbm

# skip check code that all files are exists, origin, even_snapshot, odd_snapshot


if origin is None:

db = dbm.open(origin, 'nx') # For extension format

db.gdbm_failure_atomic(even_snapshot, odd_snapshot) # For snapshot
declaration

else:

latest_snapshot = dbm.gdbm_latest_snapshot(even_snapshot odd_snapshot)

  db = dbm.open(latest_snapshot, 'r') # Open the latest valid snapshot

for k, v in zip('abcdef', 'ghijkl'):
db[k] = v

db.sync()
db.close()


> How is different from simply copying the whole database file?


Under the hood, the gdbm crash-tolerance mechanism *does* (logically) copy
the whole database file, but it does so efficiently, using "reflink"
copies, so the amount of physical storage resources used is minimal.

You may good to read this paper:
https://dl.acm.org/doi/pdf/10.1145/3487019.3487353


Warm Regards,

Dong-hee


2022년 1월 18일 (화) 오후 10:54, Victor Stinner 님이 작성:

> How does someone know if a database is corrupted? Why is a separated
> script needed?
>
> A single script cannot automatically detect a corrupted database and
> load the latest snapshot?
>
> How is different from simply copying the whole database file?
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JB5JKWEYQI5A44W4J5JTECZ73YAIF3Y3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Discussion about crash tolerance feature for gdbm module

[Dong-hee Na]

For more readable code:
https://gist.github.com/corona10/d4fe0b6367ea6865e37b4369a7d60912

2022년 1월 21일 (금) 오후 12:50, Dong-hee Na 님이 작성:

> I exchanged a mail for investigating details.
>
> Before getting started, please remind the following fact.
> - The latest snapshot is always valid, even if corruption has not occurred.
>
> > Why is a separated script needed? / A single script cannot automatically
> detect a corrupted database and load the latest snapshot?
>
> So, the author said a separate script is never needed,
> if the user reads from the latest snapshot file, it will always be
> recovered.
> so the user code will be like this.
>
> import dbm.gnu as dbm
>
> # skip check code that all files are exists, origin, even_snapshot, 
> odd_snapshot
>
>
> if origin is None:
>
> db = dbm.open(origin, 'nx') # For extension format
>
> db.gdbm_failure_atomic(even_snapshot, odd_snapshot) # For snapshot 
> declaration
>
> else:
>
> latest_snapshot = dbm.gdbm_latest_snapshot(even_snapshot odd_snapshot)
>
>   db = dbm.open(latest_snapshot, 'r') # Open the latest valid snapshot
>
> for k, v in zip('abcdef', 'ghijkl'):
> db[k] = v
>
> db.sync()
> db.close()
>
>
> > How is different from simply copying the whole database file?
>
>
> Under the hood, the gdbm crash-tolerance mechanism *does* (logically) copy
> the whole database file, but it does so efficiently, using "reflink"
> copies, so the amount of physical storage resources used is minimal.
>
> You may good to read this paper: 
> https://dl.acm.org/doi/pdf/10.1145/3487019.3487353
>
>
> Warm Regards,
>
> Dong-hee
>
>
> 2022년 1월 18일 (화) 오후 10:54, Victor Stinner 님이 작성:
>
>> How does someone know if a database is corrupted? Why is a separated
>> script needed?
>>
>> A single script cannot automatically detect a corrupted database and
>> load the latest snapshot?
>>
>> How is different from simply copying the whole database file?
>>
>> Victor
>> --
>> Night gathers, and now my watch begins. It shall not end until my death.
>>
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SQVHJIMKH4GDACFWUVSY35ZVVKND6R2S/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Discussion about crash tolerance feature for gdbm module

[Dong-hee Na]

After discussion with Victor by using DM, I decided to provide
high-level API instead of low-level APIs.

- gdbm.open(filename, snapshots=(foo, bar))

will do everything at once.

Regards,
Dong-hee

2022년 1월 21일 (금) 오후 12:52, Dong-hee Na 님이 작성:

> For more readable code:
> https://gist.github.com/corona10/d4fe0b6367ea6865e37b4369a7d60912
>
> 2022년 1월 21일 (금) 오후 12:50, Dong-hee Na 님이 작성:
>
>> I exchanged a mail for investigating details.
>>
>> Before getting started, please remind the following fact.
>> - The latest snapshot is always valid, even if corruption has not
>> occurred.
>>
>> > Why is a separated script needed? / A single script cannot
>> automatically detect a corrupted database and load the latest snapshot?
>>
>> So, the author said a separate script is never needed,
>> if the user reads from the latest snapshot file, it will always be
>> recovered.
>> so the user code will be like this.
>>
>> import dbm.gnu as dbm
>>
>> # skip check code that all files are exists, origin, even_snapshot, 
>> odd_snapshot
>>
>>
>> if origin is None:
>>
>> db = dbm.open(origin, 'nx') # For extension format
>>
>> db.gdbm_failure_atomic(even_snapshot, odd_snapshot) # For snapshot 
>> declaration
>>
>> else:
>>
>> latest_snapshot = dbm.gdbm_latest_snapshot(even_snapshot odd_snapshot)
>>
>>   db = dbm.open(latest_snapshot, 'r') # Open the latest valid 
>> snapshot
>>
>> for k, v in zip('abcdef', 'ghijkl'):
>> db[k] = v
>>
>> db.sync()
>> db.close()
>>
>>
>> > How is different from simply copying the whole database file?
>>
>>
>> Under the hood, the gdbm crash-tolerance mechanism *does* (logically) copy
>> the whole database file, but it does so efficiently, using "reflink"
>> copies, so the amount of physical storage resources used is minimal.
>>
>> You may good to read this paper: 
>> https://dl.acm.org/doi/pdf/10.1145/3487019.3487353
>>
>>
>> Warm Regards,
>>
>> Dong-hee
>>
>>
>> 2022년 1월 18일 (화) 오후 10:54, Victor Stinner 님이 작성:
>>
>>> How does someone know if a database is corrupted? Why is a separated
>>> script needed?
>>>
>>> A single script cannot automatically detect a corrupted database and
>>> load the latest snapshot?
>>>
>>> How is different from simply copying the whole database file?
>>>
>>> Victor
>>> --
>>> Night gathers, and now my watch begins. It shall not end until my death.
>>>
>>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PFEG4XFU3WICTVCPV5IYKOAONNU2NLPZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python no longer leaks memory at exit

[Dong-hee Na]

Hey Victor

First of all, I would like to thank Victor for all his efforts to resolve
this issue.
And I want to thank the other contributors to this issue as well!
I am very happy to hear that we can close bpo-1635741.
This has been a long mission requiring patience and
I'm proud to finally be able to put an end to it.

Let's move forward to the next step!

Warm regards,
Dong-hee

2022년 1월 28일 (금) 오전 12:37, Victor Stinner 님이 작성:

> Hi,
>
> tl; dr Python no longer leaks memory at exit on the "python -c pass"
> command ;-)
>
>
> == Bug report ==
>
> In 2007, the bpo-1635741 issue was reported on SourceForge:
> "Interpreter seems to leak references after finalization".
>
> This bug is 15 years old. It saw the bugs migration from SourceForge
> to Roundup (bugs.python.org) in 2007, the code migration from
> Subversion to Mercurial in 2009, and the second code migration from
> Mercurial to Git  in 2016!
>
> Link to the main issue: https://bugs.python.org/issue1635741
>
> Some people reported similar issues seen by WinCRT debug (bpo-6741),
> MSVC Debug C Runtime (bpo-32026), GCC AddressSanitzer (bpo-26888), or
> LeakSanitizer and Valgrind (bpo-21387).
>
> In general, "leaking memory" at Python exit doesn't matter, since the
> operating system releases all memory when a process completes. It
> matters when Python is embedded in an application. For example, Python
> can be used by a plugin which is unloaded and reloaded multiple times.
> It also matters for sub-interpreters.
>
>
> == Tedious task ==
>
> In the last 3 years, many people helped fixing this old issue by
> converting static types to heap types, adding a module state to C
> extension modules, converting extensions to the multi-phase
> initialization API (PEP 489), fix many memory leaks, fix bugs, etc.
>
> When it became possible to cleanly unload an extension module (thanks
> for the multi-phase init), tests on sub-intepreters (which load and
> unload extension modules) showed many *old* reference leaks: all of
> them have been fixed! When a test is run in sub-interpreters, Python
> is able to detect leaks, whereas currently it doesn't check for memory
> leaks at Python exit.
>
>
> == Python 3.10 regressions ==
>
> During the Python 3.10 development, we identified and fixed 3 major
> regressions caused by this work:
>
> * Converting static types to heap types make them mutable: the
> Py_TPFLAGS_IMMUTABLETYPE flag was added; 68 types use it in Python
> 3.11 (bpo-43908).
>
> * It became possible to create uninitialized objects by instanciating
> types (which should not be instanciated directly): the
> Py_TPFLAGS_DISALLOW_INSTANTIATION was added; 41 types use it in Python
> 3.11 (bpo-43916). For example, you cannot create an instance of
> type(sys.flags).
>
> * Heap types must implement the full GC protocol: Py_TPFLAGS_HAVE_GC
> flag, traverse and clear functions. Otherwise, the GC is unable to
> break reference cycles, whereas a type contains (multiple) strong
> references to itself (in the MRO tuple and in methods). All heap types
> have been fixed to fully implement the GC protocol (bpo-40217).
>
>
> == PEPs ==
>
> The work relies on multiple PEPs:
>
> * PEP 489: Multi-phase extension module initialization
> * PEP 573: Module State Access from C Extension Methods
> * PEP 630: Isolating Extension Modules
>
>
> == Persons who helped fixing the issue ==
>
> Incomplete list of people who helped to fix this issue:
>
> * Christian Heimes (modules: symtable, _hashlib, ,_random, grp, pwd,
> _queue, spwd, _struct, gc, _posixshmem, _posixsubprocess, select,
> resource)
> * Dong-hee Na (modules: _statistics, itertools, _heapq, _collections,
> _uuid, math, _stat, syslog, errno, fcntl, mmap, _dbm, _gdbm, _lzma,
> faulthandler, _bisect)
> * Eric Snow (PEP 573)
> * Erlend Egeberg Aasland (modules: _sqlite3, _sre, _multibytecodec)
> * Hai Shi (modules: _json, _codecs, _crypt, _contextvars, _abc, _bz2,
> _locale, audioop, _ctypes_test, _weakref, _locale)
> * Marcel Plch (PEP 573)
> * Martin von Löwis (PEP 3121)
> * Mohamed Koubaa (modules: sha256, multiprocessing, _winapi, _blake2,
> _sha3, _signal, _sha1, _sha512, _md5, zlib, _opcode, _overlapped,
> _curses_panel, termios, _sha256, scproxy, cmath, _lsprof, unicodedata)
> * Nick Coghlan (PEP 489, PEP 573)
> * Paulo Henrique Silva (modules: time, operator, _functools)
> * Petr Viktorin (PEP 489, PEP 573, PEP 630)
> * Stefan Behnel (PEP 489)
> * Victor Stinner (modules _string, mashal,_imp, _warnings, _thread)
>
> The work was scatted into many sub-issues, it was hard for me to track
> all persons who contributed, sorry about that! I only searched for
> "New chang

[Python-Dev] Re: Increase of Spammy PRs and PR reviews

[Dong-hee Na]

If you feel bad impression, sorry about that.
The mentee who cc me is under mentoring period. Since tracking all of
mentee’s activity is impossible, I requested him to cc me.
This was for checking his labeling is valid or not.

Warm regards
Dong-hee
2022년 2월 1일 (화) 오전 5:35, Lrupert via Python-Dev 님이
작성:

> > Gaming the system doesn't end up working well in the end anyway. The
> first time the gamers try to get a job interview and can't explain how
> they'd do a code review—something GitHub says they've done hundreds or
> thousands of times—the whole thing will fail.
>
> Observably, it feels like they are doing this for core privileges (if they
> don't already exist, they are a member of the python org?). Every time I
> see one of those PRs (e.g add test for X, add delete redundant variables
> for Y), the author seem to be cc-ing their mentor. This gives a bad
> impression to others about their intentions (constant contribution of
> trivial / low quality stuff with little-to-no-gain to achieve a higher
> number of commits, since it is a visible metric).
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/JZBQ2UYTXDCHADW4LEPGPE5SFLRHW5E3/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OHVDUMMSGVFWIFJ3K46OWNY6OURK3XI6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: It's now time to deprecate the stdlib urllib module

[Dong-hee Na]

I am not an expert about pip,
but it will be not a problem about installing the pip module once CPython
removes urllib module from stdlib?

Warm regards,
Dong-hee

2022년 2월 6일 (일) 오후 11:13, Victor Stinner 님이 작성:

> Hi,
>
> I propose to deprecate the urllib module in Python 3.11. It would emit
> a DeprecationWarning which warn users, so users should consider better
> alternatives like urllib3 or httpx: well known modules, better
> maintained, more secure, support HTTP/2 (httpx), etc.
>
> I don't propose to schedule its removal. Let's discuss the removal in
> 1 or 2 years.
>
> --
>
> urllib has many abstraction to support a wide range of protocols with
> "handlers": HTTP, HTTPS, FTP, "local file", proxy, HTTP
> authentication, HTTP Cookie, etc. A simple HTTP request using Basic
> Authentication requires 10-20 lines of code, whereas it should be a
> single line.
>
> Users (me included) don't like urllib API which was too complicated
> for common tasks.
>
> --
>
> Unhappy users created multiple better alternatives to the stdlib urllib
> module.
>
> In 2008, the "urllib3" module was created to provide an API designed
> to be as simple as possible for the most common HTTP and HTTPS
> requests. Example:
>
>req = http.request('GET', 'http://httpbin.org/robots.txt').
>
> In 2011, the "requests" module based on urllib3 was created.
>
> In 2013, the "aiohttp" module based on asyncio was created.
>
> In 2015, new "httpx" module was created:
>
> req = httpx.get('https://www.example.org/')
>
> Not only httpx has a regular "synchronous" API (blocking function
> calls), but it also has an asynchronous API!
>
> Sadly, while HTTP/3 is being developed, it seems like in this list,
> httpx is the only HTTP client library supporting HTTP/2 currently :-(
>
> For HTTP/2, I also found the "httplib2" module.
>
> For HTTP/3, I found the "http3" and "aioquic" modules.
>
> --
>
> Let's come back to urllib:
>
> * It's API is too complicated
> * It doesn't support HTTP/2 nor HTTP/3
> * It's barely maintained: there are 121 open issues including 3 security
> issues!
>
> The 3 open security issues:
>
> * bpo-33661 open 2018;
> * bpo-36338 open in 2019;
> * bpo-45795 open in 2021.
>
> Usually, it's bad when you refer to an open security issue by its
> creation year :-(
>
> The urllib module has long history of security vulnerabilities. List
> of *fixed* vulnerabilities:
>
> * 2011 (bpo-11662):
> https://python-security.readthedocs.io/vuln/urllib-redirect.html
> * 2017 (bpo-30119):
>
> https://python-security.readthedocs.io/vuln/urllib-ftp-stream-injection.html
> * 2017 (bpo-30500):
> https://python-security.readthedocs.io/vuln/urllib-connects-wrong-host.html
> * 2019 (bpo-35907):
> https://python-security.readthedocs.io/vuln/urllib-local-file-scheme.html
> * 2019 (bpo-38826):
> https://python-security.readthedocs.io/vuln/urllib-basic-auth-regex.html
> * 2021 (bpo-42967):
>
> https://python-security.readthedocs.io/vuln/urllib-query-string-semicolon-separator.html
> * 2021 (bpo-43075):
> https://python-security.readthedocs.io/vuln/urllib-basic-auth-regex2.html
> * 2021 (bpo-44022):
> https://python-security.readthedocs.io/vuln/urllib-100-continue-loop.html
>
> urllib is a package made of 4 parts:
>
> * urllib.request for opening and reading URLs
> * urllib.error containing the exceptions raised by urllib.request
> * urllib.parse for parsing URLs
> * urllib.robotparser for parsing robots.txt files
>
> I propose to deprecate all of them. Maybe the deprecation can be
> different for each sub-module?
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/EZ6O7MOPZ4GA75MKTDO7LAELKXUHK2QS/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/E6GN2THYCNQ2Q3CGMSH7GRCDFOOFDDCQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Request to postpone some Python 3.9 incompatible changes to Python 3.10

[Dong-hee Na]

collections.abc import Sequence
> > except ImprotError:
> > # Python 2.7 doesn't have collections.abc
> > from collections import Sequence
> >
> > While if we remove collections.Sequence in 3.10, they will face this
> > decision early in 2021 and they will simply fix their code by adding
> > ".abc" at the proper places, not needing any more compatibility
> > layers. Of course, there will be projects that will still have
> > declared Python 2 support in 2021, but it will arguably not be that
> > many.
> >
> > While it's certainly tempting to have "more pure" code in the standard
> > library, maintaining the compatibility shims for one more release
> > isn't really that big of a maintenance burden, especially when
> > comparing with dozens (hundreds?) of third party libraries essentially
> > maintaining their own.
> >
> > An good example of a broken package is the nose project which is no
> > longer maintained (according to their website): the last release was
> > in 2015. It remains a very popular test runner. According to
> > libraries.io, it has with 3 million downloads per month, 41.7K
> > dependent repositories and 325 dependent packages. We patched nose in
> > Fedora to fix Python 3.5, 3.6, 3.8 and now 3.9 compatibility issues.
> > People installing nose from PyPI with "pip install" get the
> > unmaintained flavor which is currently completely broken on Python
> > 3.9.
> >
> > Someone should take over the nose project and maintain it again, or
> > every single project using nose should pick another tool (unittest,
> > nose2, pytest, whatever else). Both options will take a lot of time.
> >
> >
> > == Request to revert some incompatible changes ==
> >
> > See https://docs.python.org/dev/whatsnew/3.9.html#removed
> >
> > Incompatible changes which require "if : (...) else: (...)"
> > or "try:  except (...): ":
> >
> > * Removed tostring/fromstring methods in array.array and base64 modules
> > * Removed collections aliases to ABC classes
> > * Removed fractions.gcd() function (which is similar to math.gcd())
> > * Remove "U" mode of open(): having to use io.open() just for Python 2
> > makes the code uglier
> > * Removed old plistlib API: 2.7 doesn't have the new API
> >
> >
> > == Kept incompatible changes ==
> >
> > Ok-ish incompatible changes (mostly only affects compatiblity
> > libraries like six):
> >
> > * _dummy_thread and dummy_threading modules removed: broke six, nine
> > and future projects. six and nine are already fixed for 3.9.
> >
> >
> > OK incompatible changes (can be replaced by the same code on 2.7 and 3.9):
> >
> > * isAlive() method of threading.Thread has been removed:
> > Thread.is_alive() method is available in 2.7 and 3.9
> > * xml.etree.ElementTree.getchildren() and
> > xml.etree.ElementTree.getiterator() methods are removed from 3.9, but
> > list()/iter() works in 2.7 and 3.9
> >
> >
> > == Call to check for DeprecationWarning in your own projects ==
> >
> > You must pay attention to DeprecationWarning in Python 3.9: it will be
> > the last "compatibility layer" release, incompatible changes will be
> > reapplied to Python 3.10.
> >
> > For example, you can use the development mode to see
> > DeprecationWarning and ResourceWarning: use the "-X dev" command line
> > option or set the PYTHONDEVMODE=1 environment variable. Or you can use
> > the PYTHONWARNINGS=default environment variable to see
> > DeprecationWarning.
> >
> > You might even want to treat all warnings as errors to ensure that you
> > don't miss any when you run your test suite in your CI. You can use
> > PYTHONWARNINGS=error, and combine it with PYTHONDEVMODE=1.
> >
> > Warnings filters can be used to ignore warnings in third party code,
> > see the documentation:
> > https://docs.python.org/dev/library/warnings.html#the-warnings-filter
> >
> > -- Victor Stinner and Miro Hrončok for Fedora
>
>
>
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/AOEVRTYSL5JVT532ACLFRGR5H5G3WQTN/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Software Development Engineer at Kakao corp.

Tel: +82 010-3353-9127
Email: donghee.n...@gmail.com | denn...@kakaocorp.com
Linkedin: https://www.linkedin.com/in/dong-hee-na-2b713b49/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VMC3R5BEONBAXSLG3HGVKZVIW3YZQHFV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 620: Hide implementation details from the C API

[Dong-hee Na]

)
> #endif  // PY_VERSION_HEX < 0x030900A4
>
> Using this header file, ``Py_SET_TYPE()`` can be used on old Python
> versions as well.
>
> Developers can copy this file in their project, or even to only
> copy/paste the few functions needed by their C extension.
>
> **STATUS**: In Progress (implemented but not distributed by CPython yet)
>
> The ``pythoncapi_compat.h`` header file is currently developer at:
> https://github.com/pythoncapi/pythoncapi_compat
>
> Process to reduce the number of broken C extensions
> ===
>
> Process to reduce the number of broken C extensions when introducing C
> API incompatible changes listed in this PEP:
>
> * Estimate how many popular C extensions are affected by the
>   incompatible change.
> * Coordinate with maintainers of broken C extensions to prepare their
>   code for the future incompatible change.
> * Introduce the incompatible changes in Python. The documentation must
>   explain how to port existing code. It is recommended to merge such
>   changes at the beginning of a development cycle to have more time for
>   tests.
> * Changes which are the most likely to break a large number of C
>   extensions should be announced on the capi-sig mailing list to notify
>   C extensions maintainers to prepare their project for the next Python.
> * If the change breaks too many projects, reverting the change should be
>   discussed, taking in account the number of broken packages, their
>   importance in the Python community, and the importance of the change.
>
> The coordination usually means reporting issues to the projects, or even
> proposing changes. It does not require waiting for a new release including
> fixes for every broken project.
>
> Since more and more C extensions are written using Cython, rather
> directly using the C API, it is important to ensure that Cython is
> prepared in advance for incompatible changes. It gives more time for C
> extension maintainers to release a new version with code generated with
> the updated Cython (for C extensions distributing the code generated by
> Cython).
>
> Future incompatible changes can be announced by deprecating a function
> in the documentation and by annotating the function with
> ``Py_DEPRECATED()``. But making a structure opaque and preventing the
> usage of a macro as l-value cannot be deprecated with
> ``Py_DEPRECATED()``.
>
> The important part is coordination and finding a balance between CPython
> evolutions and backward compatibility. For example, breaking a random,
> old, obscure and unmaintained C extension on PyPI is less severe than
> breaking numpy.
>
> If a change is reverted, we move back to the coordination step to better
> prepare the change. Once more C extensions are ready, the incompatible
> change can be reconsidered.
>
>
> Version History
> ===
>
> * Version 3, June 2020: PEP rewritten from scratch. Python now
>   distributes a new ``pythoncapi_compat.h`` header and a process is
>   defined to reduce the number of broken C extensions when introducing C
>   API incompatible changes listed in this PEP.
> * Version 2, April 2020:
>   `PEP: Modify the C API to hide implementation details
>   
> <https://mail.python.org/archives/list/python-dev@python.org/thread/HKM774XKU7DPJNLUTYHUB5U6VR6EQMJF/#TKHNENOXP6H34E73XGFOL2KKXSM4Z6T2>`_.
> * Version 1, July 2017:
>   `PEP: Hide implementation details in the C API
>   
> <https://mail.python.org/archives/list/python-id...@python.org/thread/6XATDGWK4VBUQPRHCRLKQECTJIPBVNJQ/#HFBGCWVLSM47JEP6SO67MRFT7Y3EOC44>`_
>   sent to python-ideas
>
>
> Copyright
> =
>
> This document has been placed in the public domain.
>
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/EV7F7Z6PLPWJU7SD2UPFEYKYUWU4ZJXZ/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Software Development Engineer at Kakao corp.

Tel: +82 010-3353-9127
Email: donghee.n...@gmail.com | denn...@kakaocorp.com
Linkedin: https://www.linkedin.com/in/dong-hee-na-2b713b49/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4XU6YHC6K47VURWG3I2QXLZSNPGFSRNL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Add mimetypes.mimesniff

[Dong-hee Na]

Hi,

A few weeks ago, I suggested adding mimetypes.mimesniff on stdlib.
(https://bugs.python.org/issue40841,
https://github.com/python/cpython/pull/20720)

Detecting MIME types well is an important feature and we already have
mimetypes detection library but AFAIK it is not good enough.

Note that some of our stdlib module already use the sniffing algorithm
(e.g imghdr)

The question is how exactly the mime sniffing should be done in terms
of file formats and algorithm. Luckily, WHATWG published the standard
for mime sniffing, and I think we should follow it.
(https://mimesniff.spec.whatwg.org/)

So I created the issue on the bpo and implemented it.
I 'd like to listen to all your opinions :)

-- 
Software Development Engineer at Kakao corp.

Tel: +82 10-3353-9127
Email: donghee.n...@gmail.com | denn...@kakaocorp.com
Linkedin: https://www.linkedin.com/in/dong-hee-na-2b713b49/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/633E4OMXY6D2SN4OIJPEYWJSW7DFKYKK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Pay for PR review and merging for VxWorks RTOS

[Dong-hee Na]

> Buildbot is also not a problem.

The early issue from the core developer side, it would be helpful to
review if the Buildbot is supported by Wind River for VxWorks.

2020년 8월 6일 (목) 오후 1:16, Xin, Peixing 님이 작성:
>
> Hi, Stanley:
>
>
>
> Thanks for your comments.
>
>
>
> Almost all the patches are simple compatibility patch. But the subprocess 
> module is an exception. Like windows, VxWorks have no fork/exec API provided. 
> Instead, VxWorks uses rtpSpawn() to spawn a new process. So we have to 
> implement VxWorks’ own extension modules to support subprocess module. 
> However, this is not significant amount of codes, just around 500 lines of C 
> codes. Hopes any core dev could help to review and merge the simple 
> compatibility patches. If I can be granted the commit privileges, that will 
> be great. To ensure the quality, I will get the patches reviewed enough well 
> before merging.
>
> For long term support for VxWorks RTOS, I am an engineer from Wind River. 
> Wind River is willing to officially provide the support and I am willing to 
> be the maintainer. Buildbot is also not a problem. Once the patches have been 
> merging done, we will connect the buildbot for VxWorks.
>
>
>
> Thanks,
>
> Peixing
>
>
>
> From: Kyle Stanley 
> Sent: Thursday, August 6, 2020 9:27 AM
> To: Xin, Peixing 
> Cc: python-dev@python.org; Meng, Bin 
> Subject: Re: [Python-Dev] Pay for PR review and merging for VxWorks RTOS
>
>
>
> What exactly does the PR involve? Is it a relatively simple compatibility 
> patch or something that adds significant amounts of platform-specific code? 
> The former can be reviewed (and potentially merged) by any core dev 
> knowledgeable in the areas being changed, but the latter requires long-term 
> support for the platform.
>
>
>
> In order to provide support for a new platform that we don't presently have 
> support for in CPython, it requires the following:
>
>
>
> 1) An existing core developer willing to provide long-term support for the 
> platform. Alternatively, someone else can be granted commit privileges, for 
> the purpose of maintaining that platform and fixing any issues that come up 
> in that CI that are specific to it. However, this is done on a case-by-case 
> basis, as it still requires a decent amount of trust with that individual.
>
>
>
> 2) A stable buildbot provided, to test against new changes made to CPython.
>
>
>
> The purpose of the above is to ensure that any substantial platform-specific 
> code added to CPython receives proper care; this is opposed to a large 
> addition of new code for a platform that has nobody to take care of it when 
> issues come up. Especially as a group of mostly volunteers (and those paid by 
> their employers to contribute), the latter wouldn't be sustainable. I 
> personally think it would be fantastic to have official support for real-time 
> OSs in CPython, but in this ecosystem, it requires being able to place trust 
> in someone that's willing and adequately proficient in the related areas to 
> provide long-term support for them.
>
>
>
> For more details, see PEP 11: 
> https://www.python.org/dev/peps/pep-0011/#supporting-platforms.
>
>
>
>
>
> On Wed, Aug 5, 2020 at 7:44 AM Xin, Peixing  wrote:
>
> Hi, Python developers:
>
>
>
> Anyone interested in PR review and merging for VxWorks RTOS? We can give a 
> proper pay for that. Contact me if you have interest.
>
>
>
> Thanks,
>
> Peixing
>
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/HATTYX3BWEIPVVUL7FMSNSCFYOBPS6OD/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/TJM24KANKBKLQSFZIH42V4G4WLZ7MGO3/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Software Development Engineer at Kakao corp.

Tel: +82 010-3353-9127
Email: donghee.n...@gmail.com | denn...@kakaocorp.com
Linkedin: https://www.linkedin.com/in/dong-hee-na-2b713b49/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XE6YSMZME4LQMY47IOHV2HTRIIEKIRKW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python 4 FAQ

[Dong-hee Na]

Great idea +1

2020년 9월 16일 (수) 오후 1:13, Guido van Rossum 님이 작성:

> Good idea. I tweeted pretty much what you wrote.
> https://twitter.com/gvanrossum/status/1306082472443084801
>
> On Tue, Sep 15, 2020 at 7:58 PM Steven D'Aprano 
> wrote:
>
>> There's still a lot of community angst over the possibility that
>> some hypothetical Python 4 will be like the 2/3 transition. Some people
>> even imagine that the version after 3.9 could be that transition.
>>
>> I know we're not responsible for the ravings of people on social media,
>> but do you think we should do a FAQ? Something along the lines of:
>>
>> - the version after 3.9 will be 3.10, and in fact it already exists;
>>
>> - if there ever is a version 4, the transition from 3 to 4 will be more
>> like that from 1 to 2 rather than 2 to 3.
>>
>>
>>
>>
>> --
>> Steve
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/UPKGKSEOOA3KEZUNQROFDZW66TAHN5IS/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/J4GQYYTYAE3QGYI7AGRGZCCRMMB3MXY5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZP75QDQRSL52C4A74YOWKX5OPF6FXTCY/
Code of Conduct: http://python.org/psf/codeofconduct/