Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512
> On May 27, 2016, at 12:54 AM, Raymond Hettinger > wrote: > > >> On May 25, 2016, at 3:29 AM, Christian Heimes wrote: >> >> I have three hashing-related patches for Python 3.6 that are waiting for >> review. Altogether the three patches add ten new hash algorithms to the >> hashlib module: SHA3 (224, 256, 384, 512), SHAKE (SHA3 XOF 128, 256), >> BLAKE2 (blake2b, blake2s) and truncated SHA512 (224, 256). > > Do we really need ten? I don't think the standard library is the place to > offer all variants of hashing. And we should avoid getting in a cycle of > "this was just released by NIST" and "nobody uses that one anymore". Is any > one of them an emergent best practice (i.e. starting to be commonly used in > network protocols because it is better, faster, stronger, etc)? > > Your last message on https://bugs.python.org/issue16113 suggests that these > aren't essential and that there is room for debate about whether some of them > are standard-library worthy (i.e. we will have them around forever). > I think that adding sha3 here is a net positive. While there isn’t a huge amount of things using it today, that’s largely because it’s fairly new— It’s a NIST standard so it won’t be long until things are using it. It would be surprising to me to be able to use sha1 and sha2 from the standard library, but not sha3. SHAKE is really just SHA3 with some additional tweaks to the parameters. I think if you’re adding SHA3 it’s pretty easy to also add these, though I don’t think that it’s as important as adding SHA3 itself. BLAKE2 is an interesting one, because while SHA3 is a NIST standard (so it’s going to gain adoption because of that), BLAKE2 is at least as strong as SHA3 but is better in many ways, particularly in speed— it’s actually faster than MD5 while being as secure as SHA3. This one I think is a good one to have in the standard library as well because it is all around a really great hash and a lot of things are starting to be built on top of it. In particularly I’d like to use this in PyPI and pip- but I can’t unless it’s in the standard library. — Donald Stufft ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512
Le 27 mai 2016 12:05 PM, "Donald Stufft" a écrit : > BLAKE2 is an interesting one, because while SHA3 is a NIST standard (so it’s going to gain adoption because of that), BLAKE2 is at least as strong as SHA3 but is better in many ways, particularly in speed— it’s actually faster than MD5 while being as secure as SHA3. BLAKE2 was part of the SHA3 competition and it was in finalists. The SHA3 competition is interesting because each algorithm is deeply tested and analyzed by many teams all around the world. Obvious vulnerabilities are quickly found. The advantage of putting SHA3 and BLAKE2 in the stdlib is that they have a different design. I don't expect that two designs have the same vulnerabilities, but I'm not ax expert :-) SHA3 (Keccak) is based on a new sponge construction: https://en.m.wikipedia.org/wiki/SHA-3 BLAKE is based on ChaCha: https://en.m.wikipedia.org/wiki/BLAKE_(hash_function) https://en.m.wikipedia.org/wiki/Salsa20#ChaCha_variant Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512
On 27.05.2016 06:54, Raymond Hettinger wrote: > >> On May 25, 2016, at 3:29 AM, Christian Heimes wrote: >> >> I have three hashing-related patches for Python 3.6 that are waiting for >> review. Altogether the three patches add ten new hash algorithms to the >> hashlib module: SHA3 (224, 256, 384, 512), SHAKE (SHA3 XOF 128, 256), >> BLAKE2 (blake2b, blake2s) and truncated SHA512 (224, 256). > > Do we really need ten? I don't think the standard library is the place to > offer all variants of hashing. And we should avoid getting in a cycle of > "this was just released by NIST" and "nobody uses that one anymore". Is any > one of them an emergent best practice (i.e. starting to be commonly used in > network protocols because it is better, faster, stronger, etc)? > > Your last message on https://bugs.python.org/issue16113 suggests that these > aren't essential and that there is room for debate about whether some of them > are standard-library worthy (i.e. we will have them around forever). I can understand your eagerness to get this landed, since it's been 4 years since work started, but I think we should wait with the addition until OpenSSL has them: https://github.com/openssl/openssl/issues/439 The current patch is 1.2MB for SHA-3 - that's pretty heavy for just a few hash functions, which aren't in any wide spread use yet and probably won't be for quite a few years ahead. IMO, relying on OpenSSL is a better strategy than providing (and maintaining) our own compatibility versions. Until OpenSSL has them, people can use Björn's package: https://github.com/bjornedstrom/python-sha3 Perhaps you could join forces with Björn to create a standard SHA-3 standalone package on PyPI based on your two variants which we could recommend to people in the docs ?! -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, May 27 2016) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512
> On May 27, 2016, at 6:54 AM, M.-A. Lemburg wrote: > > IMO, relying on OpenSSL is a better strategy than providing > (and maintaining) our own compatibility versions. Until OpenSSL > has them, people can use Björn's package: Even now, hashlib doesn’t rely on OpenSSL if I recall, I mean it will use it if OpenSSL is available but otherwise it has internal implementations too. — Donald Stufft ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512
On 27.05.2016 13:03, Donald Stufft wrote: > >> On May 27, 2016, at 6:54 AM, M.-A. Lemburg wrote: >> >> IMO, relying on OpenSSL is a better strategy than providing >> (and maintaining) our own compatibility versions. Until OpenSSL >> has them, people can use Björn's package: > > Even now, hashlib doesn’t rely on OpenSSL if I recall, I mean it will > use it if OpenSSL is available but otherwise it has internal implementations > too. I know, but still don't think that's a good idea. It makes sense in case you don't want to carry around OpenSSL all the time, but how often does that happen nowadays ? BTW: If I recall correctly, those hash implementations predate the deeper support for OpenSSL we now have in Python. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, May 27 2016) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512
OpenSSL sucks. Python would only have to bundle a reference implementation of the new hash algorithm(s), and unlike TLS suites they tend to just work. BLAKE2 is important, since it removes the last objection to replacing MD5 - speed - that has made it hard for cryptography fans to convince MD5 users to upgrade. On Fri, May 27, 2016 at 7:13 AM M.-A. Lemburg wrote: > On 27.05.2016 13:03, Donald Stufft wrote: > > > >> On May 27, 2016, at 6:54 AM, M.-A. Lemburg wrote: > >> > >> IMO, relying on OpenSSL is a better strategy than providing > >> (and maintaining) our own compatibility versions. Until OpenSSL > >> has them, people can use Björn's package: > > > > Even now, hashlib doesn’t rely on OpenSSL if I recall, I mean it will > > use it if OpenSSL is available but otherwise it has internal > implementations > > too. > > I know, but still don't think that's a good idea. It makes > sense in case you don't want to carry around OpenSSL all the > time, but how often does that happen nowadays ? > > BTW: If I recall correctly, those hash implementations predate > the deeper support for OpenSSL we now have in Python. > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Experts (#1, May 27 2016) > >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ > >>> Python Database Interfaces ... http://products.egenix.com/ > >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ > > > ::: We implement business ideas - efficiently in both time and costs ::: > >eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg >Registered at Amtsgericht Duesseldorf: HRB 46611 >http://www.egenix.com/company/contact/ > http://www.malemburg.com/ > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/dholth%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] embedding - PyImport_AppendInittab() after Py_Initialize()
Could anyone please clarify whether it is correct in python3.5 to call
PyImport_AppendInittab() after Py_Initialize()
I found this case among the tests for boost.python
https://github.com/boostorg/python/blob/develop/test/exec.cpp
And this test doesn't work for me with python3.5.An error looks like:
exec.cpp(137): Python Error detected in function 'void __cdecl
check_pyerr(bool)'
Traceback (most recent call last):
File "", line 1, in
ImportError: 'embedded_hello' is not a built-in module
1 error detected.
After debugging I found out that root cause is in
importlib/_bootstrap.py which do the following:
if fullname not in sys.builtin_module_names:
raise ImportError('{!r} is not a built-in
module'.format(fullname), name=fullname)
but sys.builtin_module_names is the constant which initialized once in
Py_Initialize()
---
To summarize:
Is it a bug in python3.5 or original test inside boost.python should be
fixed ?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512
On 05/27/2016 11:31 AM, Daniel Holth wrote: > BLAKE2 is important, since it removes the last objection to replacing MD5 - speed - that has made it hard for cryptography fans to convince MD5 users to upgrade. I have had to stick to MD5 for performance reasons (2 seconds in MD5 or 9.6 seconds in SHA256, IIRC) in scenarios that did not require an SHA*. Having BLAKE2 around wouldn't be a necessity, but if it shipped with newer versions of Python eventually there would be a commit switching the underlying hash function. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512
>> , which aren't in any wide spread use yet and > probably won't be for quite a few years ahead. Anything added to the stdlib now will be in py3.6+, yes? Which won't be in widespread use for quite a few years yet, either. So if ( and that's a big if) it's possible to anticipate what will be in widespread use in a couple years, getting it in now would be a good thing. -CHB > > IMO, relying on OpenSSL is a better strategy than providing > (and maintaining) our own compatibility versions. Until OpenSSL > has them, people can use Björn's package: > > https://github.com/bjornedstrom/python-sha3 > > Perhaps you could join forces with Björn to create a standard > SHA-3 standalone package on PyPI based on your two variants > which we could recommend to people in the docs ?! > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Experts (#1, May 27 2016) Python Projects, Coaching and Consulting ... http://www.egenix.com/ Python Database Interfaces ... http://products.egenix.com/ Plone/Zope Database Interfaces ... http://zope.egenix.com/ > > > ::: We implement business ideas - efficiently in both time and costs ::: > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 >D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > http://www.egenix.com/company/contact/ > http://www.malemburg.com/ > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2016-05-20 - 2016-05-27) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open5529 (+25) closed 33364 (+42) total 38893 (+67) Open issues with patches: 2400 Issues opened (46) == #16858: tarfile silently hides errors http://bugs.python.org/issue16858 reopened by mmarkk #26915: Test identity first in membership operation of ItemsView, Valu http://bugs.python.org/issue26915 reopened by xiang.zhang #27067: Improve curses tests http://bugs.python.org/issue27067 reopened by martin.panter #27072: random.getrandbits is limited to 2**31-1 bits on 64-bit Window http://bugs.python.org/issue27072 opened by Steven.Barker #27073: redundant checks in long_add and long_sub http://bugs.python.org/issue27073 opened by Oren Milman #27074: Confusing text about __all__ in __init__.py in tutorial http://bugs.python.org/issue27074 opened by ztane #27078: Make f'' strings faster than .format: BUILD_STRING opcode? http://bugs.python.org/issue27078 opened by ztane #27079: Bugs in curses.ascii predicates http://bugs.python.org/issue27079 opened by serhiy.storchaka #27080: Implement the formatting part of PEP 515, '_' in numeric liter http://bugs.python.org/issue27080 opened by eric.smith #27081: Multiprocessing is not robust against sys.stderr changes invol http://bugs.python.org/issue27081 opened by ppperry #27083: PYTHONCASEOK is ignored on Windows http://bugs.python.org/issue27083 opened by eryksun #27084: Add dir_fd and follow_symlinks kwargs to os.listdir and os.sca http://bugs.python.org/issue27084 opened by abacabadabacaba #27085: Make it possible to select return type for os.listdir http://bugs.python.org/issue27085 opened by abacabadabacaba #27086: Add closefd argument to os.listdir http://bugs.python.org/issue27086 opened by abacabadabacaba #27088: doc: select: epoll.poll: incorrect timeout units, missing maxe http://bugs.python.org/issue27088 opened by fordsfords #27095: Simplify MAKE_FUNCTION http://bugs.python.org/issue27095 opened by Demur Rumed #27099: IDLE: turn builting extensions into regular modules http://bugs.python.org/issue27099 opened by terry.reedy #27100: Attempting to use class with both __enter__ & __exit__ undefin http://bugs.python.org/issue27100 opened by ellingtonjp #27101: Compilation of python (modules) for foreign target platform pr http://bugs.python.org/issue27101 opened by complement #27103: regrtest: capture stdout (-W) option is incompatible with refl http://bugs.python.org/issue27103 opened by haypo #27105: cgi.__all__ is incomplete http://bugs.python.org/issue27105 opened by Unit03 #27106: configparser.__all__ is incomplete http://bugs.python.org/issue27106 opened by Unit03 #27107: mailbox.__all__ list is incomplete http://bugs.python.org/issue27107 opened by Unit03 #27108: mimetypes.__all__ list is incomplete http://bugs.python.org/issue27108 opened by Unit03 #27109: plistlib.__all__ list is incomplete http://bugs.python.org/issue27109 opened by Unit03 #27110: smtpd.__all__ list is incomplete http://bugs.python.org/issue27110 opened by Unit03 #27111: redundant variables in long_add and long_sub http://bugs.python.org/issue27111 opened by Oren Milman #27112: tokenize.__all__ list is incomplete http://bugs.python.org/issue27112 opened by Unit03 #27113: sqlite3 connect parameter "check_same_thread" not documented http://bugs.python.org/issue27113 opened by Dave Sawyer #27115: IDLE/tkinter: in simpledialog, != [OK] click http://bugs.python.org/issue27115 opened by terry.reedy #27117: turtledemo does not work with IDLE's new dark theme. http://bugs.python.org/issue27117 opened by terry.reedy #27119: `compile` doesn't compile into an AST object as specified http://bugs.python.org/issue27119 opened by leewz #27121: imghdr does not support jpg files with Lavc bytes http://bugs.python.org/issue27121 opened by René Løwe Jacobsen #27122: Hang with contextlib.ExitStack and subprocess.Popen (regressio http://bugs.python.org/issue27122 opened by Valentin David #27123: Allow `install_headers` command to follow specific directory s http://bugs.python.org/issue27123 opened by sylvain.corlay #27124: binascii.a2b_hex raises binascii.Error and ValueError, not Typ http://bugs.python.org/issue27124 opened by Lennart Grahl #27125: Typo in Python 2 multiprocessing documentation http://bugs.python.org/issue27125 opened by phx #27126: Apple-supplied libsqlite3 on OS X is not fork safe; can cause http://bugs.python.org/issue27126 opened by evan.jo...@bluecore.com #27127: Never have GET_ITER not followed by FOR_ITER http://bugs.python.org/issue27127 opened by Demur Rumed #27128: Add _PyObject_FastCall() http://bugs.python.org/issue27128 opened by haypo #27129: Wordcode, part 2 http://bugs.python.org/issue27129 opened by serhiy.storchaka #27130: zlib: Overflo
Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512
On 27.05.2016 17:44, Chris Barker - NOAA Federal wrote: >>> , which aren't in any wide spread use yet and >> probably won't be for quite a few years ahead. > > Anything added to the stdlib now will be in py3.6+, yes? > > Which won't be in widespread use for quite a few years yet, either. > > So if ( and that's a big if) it's possible to anticipate what will be > in widespread use in a couple years, getting it in now would be a good > thing. You cut away the important part of what I said: "The current patch is 1.2MB for SHA-3 - that's pretty heavy for just a few hash functions, ..." If people want to use the hashes earlier, this is already possible via a separate package, so we're not delaying their use. It is clear that SHA-3 will get more traction in coming years (*), but I'm pretty sure that OpenSSL will have good implementations by the time people will actively start using the new hash algorithm and then hashlib will automatically make that available (hashlib uses the OpenSSL EVP abstraction, so will be able to use any new algorithms added to OpenSSL). However, if we add the reference implementation now, we'd then be left with 1.2MB unnecessary code in the stdlib. The question is not so much: is SHA-3 useful or not, it's whether we want to maintain this forever going forward or not. (*) People are just now starting to move from SHA-1 to SHA-2 and SHA-2 was standardized in 2001. Python received SHA-2 support in 2006. So there's plenty of time to decide :-) > -CHB > > > >> >> IMO, relying on OpenSSL is a better strategy than providing >> (and maintaining) our own compatibility versions. Until OpenSSL >> has them, people can use Björn's package: >> >> https://github.com/bjornedstrom/python-sha3 >> >> Perhaps you could join forces with Björn to create a standard >> SHA-3 standalone package on PyPI based on your two variants >> which we could recommend to people in the docs ?! >> >> -- >> Marc-Andre Lemburg >> eGenix.com >> >> Professional Python Services directly from the Experts (#1, May 27 2016) > Python Projects, Coaching and Consulting ... http://www.egenix.com/ > Python Database Interfaces ... http://products.egenix.com/ > Plone/Zope Database Interfaces ... http://zope.egenix.com/ >> >> >> ::: We implement business ideas - efficiently in both time and costs ::: >> >> eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 >>D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg >> Registered at Amtsgericht Duesseldorf: HRB 46611 >> http://www.egenix.com/company/contact/ >> http://www.malemburg.com/ >> >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, May 27 2016) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512
On Fri, May 27, 2016 at 9:35 AM, M.-A. Lemburg wrote: > > So if ( and that's a big if) it's possible to anticipate what will be > > in widespread use in a couple years, getting it in now would be a good > > thing. > > You cut away the important part of what I said: > "The current patch is 1.2MB for SHA-3 - that's pretty heavy for > just a few hash functions, ..." > > If people want to use the hashes earlier, this is already possible > via a separate package, so we're not delaying their use. > That's true for ANY addition to the stdlib -- it could always be made available in a third party lib. (unless you want to use it in another part of the stdlib...) > It is clear that SHA-3 will get more traction in coming years (*), > but I'm pretty sure that OpenSSL will have good implementations by > the time people will actively start using the new hash algorithm > and then hashlib will automatically make that available (hashlib > uses the OpenSSL EVP abstraction, so will be able to use any > new algorithms added to OpenSSL). > > However, if we add the reference implementation now, we'd then be > left with 1.2MB unnecessary code in the stdlib. > I'm probably showing my ignorance here, but couldn't we swap in the OpenSSL implementation when that becomes available? -CHB (*) People are just now starting to move from SHA-1 to SHA-2 > and SHA-2 was standardized in 2001. Python received SHA-2 support > in 2006. So there's plenty of time to decide :-) can't deny the history, nor the inertia -- but that doesn't make it a good thing... -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512
On 27.05.2016 18:41, Chris Barker wrote: > On Fri, May 27, 2016 at 9:35 AM, M.-A. Lemburg wrote: > >>> So if ( and that's a big if) it's possible to anticipate what will be >>> in widespread use in a couple years, getting it in now would be a good >>> thing. >> >> You cut away the important part of what I said: >> "The current patch is 1.2MB for SHA-3 - that's pretty heavy for >> just a few hash functions, ..." >> >> If people want to use the hashes earlier, this is already possible >> via a separate package, so we're not delaying their use. >> > > That's true for ANY addition to the stdlib -- it could always be made > available in a third party lib. (unless you want to use it in another part > of the stdlib...) Well, any addition for which someone already wrote a package, but yes... >> It is clear that SHA-3 will get more traction in coming years (*), >> but I'm pretty sure that OpenSSL will have good implementations by >> the time people will actively start using the new hash algorithm >> and then hashlib will automatically make that available (hashlib >> uses the OpenSSL EVP abstraction, so will be able to use any >> new algorithms added to OpenSSL). >> >> However, if we add the reference implementation now, we'd then be >> left with 1.2MB unnecessary code in the stdlib. >> > > I'm probably showing my ignorance here, but couldn't we swap in the OpenSSL > implementation when that becomes available? We could, but only if we don't expose separate interfaces for the hashes and not add them to hashlib. hashlib.algorithms hashlib.algorithms_guaranteed > -CHB > > > (*) People are just now starting to move from SHA-1 to SHA-2 >> and SHA-2 was standardized in 2001. Python received SHA-2 support >> in 2006. So there's plenty of time to decide :-) > > > can't deny the history, nor the inertia -- but that doesn't make it a good > thing... > > > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/mal%40egenix.com > -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, May 27 2016) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Adding NewType() to PEP 484
Hi, It has been proposed to enhance the typing module with a NewType function that allows to define simple unique types with almost zero runtime overhead. The PR containing actual implementation and PEP 484 update is here: https://github.com/python/typing/pull/226 Review comments are very welcome. Best regards, Ivan ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding NewType() to PEP 484
Patches to Python's stdlib should go through bugs.python.org so it isn't lost in email. On Fri, May 27, 2016, 12:00 Ivan Levkivskyi wrote: > Hi, > > It has been proposed to enhance the typing module with a NewType function > that allows to define simple unique types with almost zero runtime > overhead. > > The PR containing actual implementation and PEP 484 update is here: > https://github.com/python/typing/pull/226 > > Review comments are very welcome. > > Best regards, > Ivan > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding NewType() to PEP 484
Sorry, this is more meant to be the start of a discussion about the proposed feature. And typing.py has its own upstream repo (like asyncio). --Guido (mobile) On May 27, 2016 12:52 PM, "Brett Cannon" wrote: > Patches to Python's stdlib should go through bugs.python.org so it isn't > lost in email. > > On Fri, May 27, 2016, 12:00 Ivan Levkivskyi wrote: > >> Hi, >> >> It has been proposed to enhance the typing module with a NewType function >> that allows to define simple unique types with almost zero runtime >> overhead. >> >> The PR containing actual implementation and PEP 484 update is here: >> https://github.com/python/typing/pull/226 >> >> Review comments are very welcome. >> >> Best regards, >> Ivan >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/brett%40python.org >> > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org > > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512
Le vendredi 27 mai 2016, M.-A. Lemburg a écrit : > > The curent patch is 1.2MB for SHA-3 - that's pretty heavy for just > a few hash functions, which aren't in any wide spread use yet and > probably won't be for quite a few years ahead. Oh wow, it's so fat? Why is it so big? Can't we use a lighter version? Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512
On May 27, 2016 3:04 PM, "Victor Stinner" wrote: > > Le vendredi 27 mai 2016, M.-A. Lemburg a écrit : >> >> The curent patch is 1.2MB for SHA-3 - that's pretty heavy for just >> >> a few hash functions, which aren't in any wide spread use yet and >> probably won't be for quite a few years ahead. > > > Oh wow, it's so fat? Why is it so big? Can't we use a lighter version? > The stark majority of the patch is Lib/test/vectors/sha3_224.txt, which seems to be (as the file path implies) just test data. A whopping >1k LOC of really long hashes. > Victor > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com > -- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong. http://kirbyfan64.github.io/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512
On 27.05.2016 22:58, Ryan Gonzalez wrote: > On May 27, 2016 3:04 PM, "Victor Stinner" wrote: >> >> Le vendredi 27 mai 2016, M.-A. Lemburg a écrit : >>> >>> The current patch is 1.2MB for SHA-3 - that's pretty heavy for just >>> a few hash functions, which aren't in any wide spread use yet and >>> probably won't be for quite a few years ahead. >> >> >> Oh wow, it's so fat? Why is it so big? Can't we use a lighter version? >> > > The stark majority of the patch is Lib/test/vectors/sha3_224.txt, which > seems to be (as the file path implies) just test data. A whopping >1k LOC > of really long hashes. Right. There's about 1MB test data in the patch, but even without that data, the patch adds more than 6400 lines of code. If we add this now, there should at least be an exit strategy to remove the code again, when OpenSSL ships with the same code, IMO. Aside: BLAKE2 has already landed in OpenSSL 1.1.0: https://github.com/openssl/openssl/tree/master/crypto/blake2 -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, May 27 2016) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512
> On May 27, 2016, at 5:41 PM, M.-A. Lemburg wrote: > > If we add this now, there should at least be an exit strategy > to remove the code again, when OpenSSL ships with the same > code, IMO. I think it is a clear win to have the fallback implementations in cases where people either don’t have OpenSSL or don’t have a new enough OpenSSL for those implementations. Not having the fallback just makes it more difficult for people to rely on those hash functions. — Donald Stufft ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512
On 27.05.2016 23:46, Donald Stufft wrote: > >> On May 27, 2016, at 5:41 PM, M.-A. Lemburg wrote: >> >> If we add this now, there should at least be an exit strategy >> to remove the code again, when OpenSSL ships with the same >> code, IMO. > > I think it is a clear win to have the fallback implementations in cases where > people either don’t have OpenSSL or don’t have a new enough OpenSSL for those > implementations. Not having the fallback just makes it more difficult for > people to rely on those hash functions. This will only be needed once the stdlib itself starts requiring support for some of these hashes and for that we could add a pure Python implementation, eg. https://github.com/coruus/py-keccak In all other cases, you can simply add the support via a package such as Björn's or Christian's. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, May 27 2016) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512
On Fri, May 27, 2016 at 3:08 PM, M.-A. Lemburg wrote: > On 27.05.2016 23:46, Donald Stufft wrote: >> >>> On May 27, 2016, at 5:41 PM, M.-A. Lemburg wrote: >>> >>> If we add this now, there should at least be an exit strategy >>> to remove the code again, when OpenSSL ships with the same >>> code, IMO. >> >> I think it is a clear win to have the fallback implementations in cases >> where people either don’t have OpenSSL or don’t have a new enough OpenSSL >> for those implementations. Not having the fallback just makes it more >> difficult for people to rely on those hash functions. > > This will only be needed once the stdlib itself starts requiring > support for some of these hashes and for that we could add > a pure Python implementation, eg. > > https://github.com/coruus/py-keccak > > In all other cases, you can simply add the support via a > package such as Björn's or Christian's. SHA-3 and BLAKE are extremely widely accepted standards, our users will expect them, and they're significant improvements over all the current hashes in the algorithms_guaranteed list. If we demote them to second-class support (by making them only available in some builds, or using a slow pure Python implementation), then we'll be encouraging users to use inferior hashes. We shouldn't do this without a very good reason, and I don't see anything very convincing here... by all means drop the megabyte of test data, but why does it matter how many lines of code the algorithm is? No python developer will ever have to look at it -- hash code by its nature is *very* low maintenance (it either computes the right function or it doesn't, and the right answer never changes). And in unlikely case where some terrible unexpected bug is discovered then the only maintenance needed will be to delete the current impl and drop-in whatever the new fixed one is. So +1 to adding SHA-3 and BLAKE to algorithms_guaranteed. -n -- Nathaniel J. Smith -- https://vorpus.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding NewType() to PEP 484
Also -- the most important thing. :-) What to call these things? We're
pretty much settled on the semantics and how to create them (A =
NewType('A', int)) but what should we call types like A when we're
talking about them? "New types" sounds awkward.
On Fri, May 27, 2016 at 12:54 PM, Guido van Rossum wrote:
> Sorry, this is more meant to be the start of a discussion about the proposed
> feature. And typing.py has its own upstream repo (like asyncio).
>
> --Guido (mobile)
>
> On May 27, 2016 12:52 PM, "Brett Cannon" wrote:
>>
>> Patches to Python's stdlib should go through bugs.python.org so it isn't
>> lost in email.
>>
>>
>> On Fri, May 27, 2016, 12:00 Ivan Levkivskyi wrote:
>>>
>>> Hi,
>>>
>>> It has been proposed to enhance the typing module with a NewType function
>>> that allows to define simple unique types with almost zero runtime
>>> overhead.
>>>
>>> The PR containing actual implementation and PEP 484 update is here:
>>> https://github.com/python/typing/pull/226
>>>
>>> Review comments are very welcome.
>>>
>>> Best regards,
>>> Ivan
>>> ___
>>> Python-Dev mailing list
>>> Python-Dev@python.org
>>> https://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe:
>>> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>>
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>
--
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512
On 05/27/2016 07:52 PM, Nathaniel Smith wrote: If we demote them to second-class support (by making them only > available in some builds, or using a slow pure Python implementation), > then we'll be encouraging users to use inferior hashes. We shouldn't > do this without a very good reason. I agree. And I really think we shouldn't even ship pure Python implementations of these hashing algorithms. I am fairly confident that these algorithms would be prohibitively slow if written in pure Python. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding NewType() to PEP 484
On Fri, May 27, 2016 at 04:01:11PM -0700, Guido van Rossum wrote:
> Also -- the most important thing. :-) What to call these things? We're
> pretty much settled on the semantics and how to create them (A =
> NewType('A', int)) but what should we call types like A when we're
> talking about them? "New types" sounds awkward.
TypeAlias? Because A is an alias for int?
--
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding NewType() to PEP 484
Guido van Rossum wrote:
Also -- the most important thing. :-) What to call these things? We're
pretty much settled on the semantics and how to create them (A =
NewType('A', int)) but what should we call types like A when we're
talking about them? "New types" sounds awkward.
Fake types? Virtual types? Pseudo-types?
--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding NewType() to PEP 484
Steven D'Aprano wrote: TypeAlias? Because A is an alias for int? That suggests it's just another name for the same type, but it's not. It's a distinct type as far as the static type checker is concerned. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding NewType() to PEP 484
On Fri, May 27, 2016 at 6:59 PM, Greg Ewing wrote: > Steven D'Aprano wrote: > >> TypeAlias? Because A is an alias for int? > > > That suggests it's just another name for the same type, > but it's not. It's a distinct type as far as the static > type checker is concerned. We discussed this over dinner at PyCon, some ideas we came up with: - Dependent types, harking back to a similar concept in Ada (https://en.wikibooks.org/wiki/Ada_Programming/Type_System#Derived_types) which in that language is also spelled with "new". - New type - Distinguished Type - Distinguished Subtype - Distinguished Type Alias - Distinguished Alias - BoatyMcBoatType The nice thing about "distinguished" is that it's a relatively rare word so it is easy to remember or look up. Personally I'm still in favor of Derived type (but I'm more into ancient programming languages than most folks here). I could also live with Distinguished Type. The problem with Alias is that these types are *not* aliases (at least not to the type checker, which is where we need the terminology -- so we can talk about what the type checker does to these). > Fake types? Virtual types? Pseudo-types? I'm not keen on any of these. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
