Re: [Python-Dev] zipimport

2015-06-04 Thread Nick Coghlan
On 4 June 2015 at 12:10, Rose Ames  wrote:
> Sounds like I can just add a zipimporter to Lib/importlib/_bootstrap.py and
> remove zipimport.c entirely, is that right?

We moved things around a bit for 3.5, so this would be in
Lib/importlib/_bootstrap_external.py now.

My vague recollection is that the key complication we came up with was
how to make the "standard library in a zipfile" scenario keep working
(together with the completely frozen application case, where the app
and its modules are bundled together with the interpreter in one
binary). In that case, the zip format is just being used as an
archive, rather than for compression, so the current zlib dependency
shouldn't enter into it.

However, it likely makes sense to look at what would be involved in
making zipimport PEP 451 compatible *without* accounting for that
constraint, and then see what would need to be refactored/frozen/moved
to C to handle it after the simpler case is working.

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] speed.python.org (was: 2.7 is here until 2020, please don't call it a waste.)

2015-06-04 Thread M.-A. Lemburg
On 04.06.2015 04:08, Tetsuya Morimoto wrote:
>> If someone were to volunteer to set up and run speed.python.org, I think
> we could add some additional focus on performance regressions. Right now,
> we don't have any way of reliably and reproducibly testing Python
> performance.
> 
> I'm very interested in speed.python.org and feel regret that the project is
> standing still. I have a mind to contribute something ...

On 03.06.2015 18:59, Maciej Fijalkowski wrote:> On Wed, Jun 3, 2015 at 3:49 PM, 
R. David Murray
 I think we should look into getting speed.python.org up and
 running for both Python 2 and 3 branches:

  https://speed.python.org/

 What would it take to make that happen ?
>>>
>>> I guess ideal would be some cooperation from some of the cpython devs,
>>> so say someone can setup cpython buildbot
>>
>> What does "set up cpython buildbot" mean in this context?
>
> The way it works is dual - there is a program running the benchmarks
> (the runner) which is in the pypy case run by the pypy buildbot and
> the web side that reports stuff. So someone who has access to cpython
> buildbot would be useful.

Ok, so there's interest and we have at least a few people who are
willing to help.

Now we need someone to take the lead on this and form a small
project group to get everything implemented. Who would be up
to such a task ?

The speed project already has a mailing list, so you could use
that for organizing the details.

We could also create a PSF work group and assign a budget to it,
if that helps.

If you need help with all this, let me know.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 04 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] speed.python.org (was: 2.7 is here until 2020, please don't call it a waste.)

2015-06-04 Thread R. David Murray
On Thu, 04 Jun 2015 12:55:55 +0200, "M.-A. Lemburg"  wrote:
> On 04.06.2015 04:08, Tetsuya Morimoto wrote:
> >> If someone were to volunteer to set up and run speed.python.org, I think
> > we could add some additional focus on performance regressions. Right now,
> > we don't have any way of reliably and reproducibly testing Python
> > performance.
> > 
> > I'm very interested in speed.python.org and feel regret that the project is
> > standing still. I have a mind to contribute something ...
> 
> On 03.06.2015 18:59, Maciej Fijalkowski wrote:> On Wed, Jun 3, 2015 at 3:49 
> PM, R. David Murray
>  I think we should look into getting speed.python.org up and
>  running for both Python 2 and 3 branches:
> 
>   https://speed.python.org/
> 
>  What would it take to make that happen ?
> >>>
> >>> I guess ideal would be some cooperation from some of the cpython devs,
> >>> so say someone can setup cpython buildbot
> >>
> >> What does "set up cpython buildbot" mean in this context?
> >
> > The way it works is dual - there is a program running the benchmarks
> > (the runner) which is in the pypy case run by the pypy buildbot and
> > the web side that reports stuff. So someone who has access to cpython
> > buildbot would be useful.

(I don't seem to have gotten a copy of Maciej's message, at least not
yet.)

OK, so what you are saying is that speed.python.org will run a buildbot
slave so that when a change is committed to cPython, a speed run will be
triggered?  Is "the runner" a normal buildbot slave, or something
custom?  In the normal case the master controls what the slave
runs...but regardless, you'll need to let us know how the slave
invocation needs to be configured on the master.

> Ok, so there's interest and we have at least a few people who are
> willing to help.
> 
> Now we need someone to take the lead on this and form a small
> project group to get everything implemented. Who would be up
> to such a task ?
> 
> The speed project already has a mailing list, so you could use
> that for organizing the details.

If it's a low volume list I'm willing to sign up, but regardless I'm
willing to help with the buildbot setup on the CPython side.  (As soon
as my credential-update request gets through infrastructure, at least :)

--David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] speed.python.org (was: 2.7 is here until 2020, please don't call it a waste.)

2015-06-04 Thread Maciej Fijalkowski
On Thu, Jun 4, 2015 at 4:32 PM, R. David Murray  wrote:
> On Thu, 04 Jun 2015 12:55:55 +0200, "M.-A. Lemburg"  wrote:
>> On 04.06.2015 04:08, Tetsuya Morimoto wrote:
>> >> If someone were to volunteer to set up and run speed.python.org, I think
>> > we could add some additional focus on performance regressions. Right now,
>> > we don't have any way of reliably and reproducibly testing Python
>> > performance.
>> >
>> > I'm very interested in speed.python.org and feel regret that the project is
>> > standing still. I have a mind to contribute something ...
>>
>> On 03.06.2015 18:59, Maciej Fijalkowski wrote:> On Wed, Jun 3, 2015 at 3:49 
>> PM, R. David Murray
>>  I think we should look into getting speed.python.org up and
>>  running for both Python 2 and 3 branches:
>> 
>>   https://speed.python.org/
>> 
>>  What would it take to make that happen ?
>> >>>
>> >>> I guess ideal would be some cooperation from some of the cpython devs,
>> >>> so say someone can setup cpython buildbot
>> >>
>> >> What does "set up cpython buildbot" mean in this context?
>> >
>> > The way it works is dual - there is a program running the benchmarks
>> > (the runner) which is in the pypy case run by the pypy buildbot and
>> > the web side that reports stuff. So someone who has access to cpython
>> > buildbot would be useful.
>
> (I don't seem to have gotten a copy of Maciej's message, at least not
> yet.)
>
> OK, so what you are saying is that speed.python.org will run a buildbot
> slave so that when a change is committed to cPython, a speed run will be
> triggered?  Is "the runner" a normal buildbot slave, or something
> custom?  In the normal case the master controls what the slave
> runs...but regardless, you'll need to let us know how the slave
> invocation needs to be configured on the master.

Ideally nightly (benchmarks take a while). The setup for pypy looks like this:


https://bitbucket.org/pypy/buildbot/src/5fa1f1a4990f842dfbee416c4c2e2f6f75d451c4/bot2/pypybuildbot/builds.py?at=default#cl-734

so fairly easy. This already generates a json file that you can plot.
We can setup an upload automatically too.


>
>> Ok, so there's interest and we have at least a few people who are
>> willing to help.
>>
>> Now we need someone to take the lead on this and form a small
>> project group to get everything implemented. Who would be up
>> to such a task ?
>>
>> The speed project already has a mailing list, so you could use
>> that for organizing the details.
>
> If it's a low volume list I'm willing to sign up, but regardless I'm
> willing to help with the buildbot setup on the CPython side.  (As soon
> as my credential-update request gets through infrastructure, at least :)
>
> --David
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/fijall%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Mingw help

2015-06-04 Thread Steve Dower
If you have an interest in linking to the Windows builds of Python 2.7 and 3.5+ 
using mingw, please visit http://bugs.python.org/issue24385

Unless someone can provide me with the One True Way to generate a lib that will 
work for everyone, I'm going to replace the lib file with instructions on how 
to generate it with your own tools. I've done enough guessing and need someone 
who actually uses mingw to step up and help out.

Thanks,
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Unable to build regex module against Python 3.5 32-bit

2015-06-04 Thread MRAB

On 2015-05-27 09:25, Paul Moore wrote:

On 27 May 2015 at 09:10, Nick Coghlan  wrote:

The old distutils docs aren't gone, the top level links just moved to the
distutils package docs: https://docs.python.org/3/library/distutils.html

I kept them (with the same deep link URLs) because I know there's stuff in
there that isn't currently documented anywhere else. I moved them to a more
obscure location because there's also stuff in there that's thoroughly
outdated, and it's a non-trivial task to figure out which is which and move
the still useful stuff to a more appropriate home :)


Thanks.

Your plan worked perfectly, because I never knew they were there :-)

https://docs.python.org/3/install/index.html#older-versions-of-python-and-mingw
implies that the libpythonXY.a files are only needed in older versions
of Python/mingw. I don't know how true that is, although I do know
that mingw should be able to link directly to a DLL without needing a
lib file.

It would be interesting to know if MRAB's build process can use the
DLL, rather than requiring a lib file (or for that matter if distutils
works without the lib file!)


Steve Dower's post has prompted me to look again at building the regex
module for Python 3.5, 32-bit and 64-bit, using just Mingw64 and
linking against python32.dll. It works!

Earlier versions of Python, however, including Python 2.7, still seem
to want libpython??.a.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Unable to build regex module against Python 3.5 32-bit

2015-06-04 Thread MRAB

On 2015-06-05 01:37, MRAB wrote:

On 2015-05-27 09:25, Paul Moore wrote:

On 27 May 2015 at 09:10, Nick Coghlan  wrote:

The old distutils docs aren't gone, the top level links just moved to the
distutils package docs: https://docs.python.org/3/library/distutils.html

I kept them (with the same deep link URLs) because I know there's stuff in
there that isn't currently documented anywhere else. I moved them to a more
obscure location because there's also stuff in there that's thoroughly
outdated, and it's a non-trivial task to figure out which is which and move
the still useful stuff to a more appropriate home :)


Thanks.

Your plan worked perfectly, because I never knew they were there :-)

https://docs.python.org/3/install/index.html#older-versions-of-python-and-mingw
implies that the libpythonXY.a files are only needed in older versions
of Python/mingw. I don't know how true that is, although I do know
that mingw should be able to link directly to a DLL without needing a
lib file.

It would be interesting to know if MRAB's build process can use the
DLL, rather than requiring a lib file (or for that matter if distutils
works without the lib file!)


Steve Dower's post has prompted me to look again at building the regex
module for Python 3.5, 32-bit and 64-bit, using just Mingw64 and
linking against python32.dll. It works!

Earlier versions of Python, however, including Python 2.7, still seem
to want libpython??.a.


For reference, here's how I can build the regex module on Windows 8.1,
64-bit, using only MinGW64.

For Python 3.5, I can link against "python35.dll", but for earlier
versions, including Python 2.7, I need "libpython.a".

I have built regex module for all of the 16 supported versions of
Python (2.5-2.7, 3.1-3.5, 64-bit and 32-bit) and they have all passed
the tests.


rem For Python 3.5, 64-bit.
rem Can link against the Python DLL.

rem Compile
"C:\MinGW64\bin\gcc.exe" -mdll -m64 -DMS_WIN64 -O2 -Wall -Wsign-compare 
-Wconversion -I"C:\Python35-64\include" -c 
"D:\mrab-regex\source\_regex_unicode.c" -o 
"D:\mrab-regex\release\3.5-64\_regex_unicode.o"


"C:\MinGW64\bin\gcc.exe" -mdll -m64 -DMS_WIN64 -O2 -Wall -Wsign-compare 
-Wconversion -I"C:\Python35-64\include" -c 
"D:\mrab-regex\source\_regex.c" -o "D:\mrab-regex\release\3.5-64\_regex.o"


rem Link
"C:\MinGW64\bin\gcc.exe" -m64 -shared -s 
"D:\mrab-regex\release\3.5-64\_regex_unicode.o" 
"D:\mrab-regex\release\3.5-64\_regex.o" -L"C:\Python35" -lpython35 -o 
"D:\mrab-regex\release\3.5-64\_regex.pyd"



rem For Python 3.5, 32-bit.
rem Can link against the Python DLL.

rem Compile
"C:\MinGW64\bin\gcc.exe" -mdll -m32  -O2 -Wall -Wsign-compare 
-Wconversion -I"C:\Python35-32\include" -c 
"D:\mrab-regex\source\_regex_unicode.c" -o 
"D:\mrab-regex\release\3.5-32\_regex_unicode.o"


"C:\MinGW64\bin\gcc.exe" -mdll -m32  -O2 -Wall -Wsign-compare 
-Wconversion -I"C:\Python35-32\include" -c 
"D:\mrab-regex\source\_regex.c" -o "D:\mrab-regex\release\3.5-32\_regex.o"


rem Link
"C:\MinGW64\bin\gcc.exe" -m32 -shared -s 
"D:\mrab-regex\release\3.5-32\_regex_unicode.o" 
"D:\mrab-regex\release\3.5-32\_regex.o" -L"C:\Python35-32" -lpython35 -o 
"D:\mrab-regex\release\3.5-32\_regex.pyd"



rem For Python 3.4, 64-bit.
rem Need to link against the Python .a file.

rem Make libpython34.a
"C:\MinGW64\x86_64-w64-mingw32\bin\gendef.exe" - 
"C:\Windows\System32\python34.dll" >"C:\Python34-64\libs\libpython34.def"


"C:\MinGW64\bin\dlltool.exe" --dllname python34.dll --def 
"C:\Python34-64\libs\libpython34.def" --output-lib 
"C:\Python34-64\libs\libpython34.a"


rem Compile
"C:\MinGW64\bin\gcc.exe" -mdll -m64 -DMS_WIN64 -O2 -Wall -Wsign-compare 
-Wconversion -I"C:\Python34-64\include" -c 
"D:\mrab-regex\source\_regex_unicode.c" -o 
"D:\mrab-regex\release\3.4-64\_regex_unicode.o"


rem Link
"C:\MinGW64\bin\gcc.exe" -mdll -m64 -DMS_WIN64 -O2 -Wall -Wsign-compare 
-Wconversion -I"C:\Python34-64\include" -c 
"D:\mrab-regex\source\_regex.c" -o "D:\mrab-regex\release\3.4-64\_regex.o"


"C:\MinGW64\bin\gcc.exe" -m64 -shared -s 
"D:\mrab-regex\release\3.4-64\_regex_unicode.o" 
"D:\mrab-regex\release\3.4-64\_regex.o" -L"C:\Python34-64\libs" 
-lpython34 -o "D:\mrab-regex\release\3.4-64\_regex.pyd"



rem For Python 3.4, 32-bit.
rem Need to link against the Python .a file.

rem Make libpython34.a
"C:\MinGW64\x86_64-w64-mingw32\bin\gendef.exe" - 
"C:\Windows\SysWOW64\python34.dll" >"C:\Python34-32\libs\libpython34.def"


"C:\MinGW64\x86_64-w64-mingw32\bin\dlltool.exe" --as-flags=--32 -m i386 
--dllname python34.dll --def "C:\Python34-32\libs\libpython34.def" 
--output-lib "C:\Python34-32\libs\libpython34.a"


rem Compile
"C:\MinGW64\bin\gcc.exe" -mdll -m32  -O2 -Wall -Wsign-compare 
-Wconversion -I"C:\Python34-32\include" -c 
"D:\mrab-regex\source\_regex_unicode.c" -o 
"D:\mrab-regex\release\3.4-32\_regex_unicode.o"


"C:\MinGW64\bin\gcc.exe" -mdll -m32  -O2 -Wall -Wsign-compare 
-Wconversion -I"C:\Python34-32\include" -c 
"D:\mrab-regex\source\_regex.c" -o "D:\mrab-regex\releas

Re: [Python-Dev] Unable to build regex module against Python 3.5 32-bit

2015-06-04 Thread Mark Lawrence

On 05/06/2015 01:37, MRAB wrote:

Steve Dower's post has prompted me to look again at building the regex
module for Python 3.5, 32-bit and 64-bit, using just Mingw64 and
linking against python32.dll. It works!

Earlier versions of Python, however, including Python 2.7, still seem
to want libpython??.a.



This http://bugs.python.org/issue24385 should interest you.

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com