Re: [Cython] h5py build broken by 0.20b2

[Stefan Behnel]

Hi,

thanks for the report!

Julian Taylor, 12.01.2014 21:12:
> a h5py cython file fails to build with cython git head and 0.20b2.
> It works with older versions.
> This file fails
> 
> The offending file seems to be:
> https://github.com/h5py/h5py/blob/master/h5py/h5p.pyd
> 
> cython h5py/h5p.pyx
> Error compiling Cython file:
> 
> ...
> from _objects cimport ObjectID
> 
> # --- Base classes ---
> 
> cdef class PropID(ObjectID):
> """ Base class for all property lists """
>^
> 
> h5py/h5p.pxd:17:4: Executable statement not allowed here
> ...
> 
> it was introduced in cython around this commit:
> b6b5152f386ddae503674cc26200a547f3b4c8b0
> properly handle expressions at the beginning of func/class/etc. blocks
> 
> Is this an intentional change?

Well, yes, but apparently one with unintended side-effects. :)

I pushed a proposed quick-fix here, test case pending, so it would be great
if you or Yaroslav could give it a try.

https://github.com/cython/cython/commit/279efc20fb8fb0bbcba5922afd010bdd3424e3b5

Stefan

___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.20 beta 2

[Christoph Gohlke]

On 1/11/2014 11:53 PM, Robert Bradshaw wrote:

There was a lot of bugfixes and other good stuff on master, so I
pulled it in and am releasing another beta. You can find it at
http://cython.org/release/Cython-0.20b2.tar.gz . I don't anticipate
merging master again before the release, so this is nearly a release
candidate, but please let us know if you run into any regressions.

- Robert


Hello,

I have trouble running the tests (`runtests.py`) on Windows:

The script does not get beyond `runTest (__main__.EndToEndTest) 
End-to-end common_include_dir ...`.
Setting `nthreads = 0` in `tests/build/common_include_dir.srctree` gets 
around this but the test fails. The file also uses `ls` and `grep` which 
are not commonly found on Windows.


The file `tests/build/build_dir.srctree` uses `os.symlink()`, which is 
not available on Windows 
.


There are hundreds of test errors, most of kind `IOError: [Errno 24] Too 
many open files`.
The test output for 64 bit Python 2.7 with numpy 1.7.2 and msvc9 is at 
. 



Christoph
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.20 beta 2

[Stefan Behnel]

Hi Christoph,

thanks for testing!

Christoph Gohlke, 13.01.2014 09:46:
> I have trouble running the tests (`runtests.py`) on Windows:
> [...]
> There are hundreds of test errors, most of kind `IOError: [Errno 24] Too
> many open files`.

The tests start consistently failing at some point, which hints at a file
handle leak, either in the test runner or in the compiler.


> The test output for 64 bit Python 2.7 with numpy 1.7.2 and msvc9 is at
> .

Yes, lots of problems and warnings. Might take a bit to look through all of
them.

Regarding this one:

"""
compiling (cpp) and running final_method_T586 ... final_method_T586.c
final_method_T586.c(8) : fatal error C1083: Cannot open include file:
'pyconfig.h': No such file or directory
ERROR
"""

Is there no "pyconfig.h" amongst the CPython header files on Windows?

Stefan

___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

[Cython] remove timestamps from generated source files

[Julian Taylor]

Hi,
Cython currently places timestamps into all its generated C source files.
This may be ok for developers who normally only rebuild files they
changed anyway but it is a major annoyance for distribution packagers
who often have to rebuild software from scratch including re-cythonizing.
The reason this is annoying is that the adding of timestamps breaks
caching of compiler results e.g. with ccache. Due to the size of the
generated files recompiling the full sources normally takes a large
fraction of the build time of packages.

I'm not sure what purpose the timestamps really serve, the version
number also included should be sufficient information in order to deduce
its origin.

Would it be possible to change Cython to stop putting timestamps into
the source?
I would also be happy if the granularity is reduced from seconds to
hours/days or an non-default option to disable it (e.g. an environment
variable).


Additionally I think it might be useful to have a vendor id added to the
version number added to the source files. This would allow easier origin
tracking of files created with distribution patched versions of Cython.
E.g. if Debian patches 0.20 of Cython it puts in

 Generated by Cython 0.20 (Debian revision 3)

Where the vendor id is added via a Distribution patch of the package and
none is emitted for upstream builds.
(CC. Debian maintainer of Cython for comments)

Cheers,
Julian Taylor
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] h5py build broken by 0.20b2

[Julian Taylor]

On 13.01.2014 09:26, Stefan Behnel wrote:
> Hi,
> ...
>>
>> it was introduced in cython around this commit:
>> b6b5152f386ddae503674cc26200a547f3b4c8b0
>> properly handle expressions at the beginning of func/class/etc. blocks
>>
>> Is this an intentional change?
> 
> Well, yes, but apparently one with unintended side-effects. :)
> 
> I pushed a proposed quick-fix here, test case pending, so it would be great
> if you or Yaroslav could give it a try.
> 
> https://github.com/cython/cython/commit/279efc20fb8fb0bbcba5922afd010bdd3424e3b5
> 
> Stefan


No, I still get the same error message with cython master
dab03a3cb796646c80d2f which includes the fix attempt.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] remove timestamps from generated source files

[Robert Bradshaw]

The timestamp is in a comment which ccache can strip before looking up
items in the cache; are you sure this is an issue?

On Mon, Jan 13, 2014 at 10:39 AM, Julian Taylor
 wrote:
> Hi,
> Cython currently places timestamps into all its generated C source files.
> This may be ok for developers who normally only rebuild files they
> changed anyway but it is a major annoyance for distribution packagers
> who often have to rebuild software from scratch including re-cythonizing.
> The reason this is annoying is that the adding of timestamps breaks
> caching of compiler results e.g. with ccache. Due to the size of the
> generated files recompiling the full sources normally takes a large
> fraction of the build time of packages.
>
> I'm not sure what purpose the timestamps really serve, the version
> number also included should be sufficient information in order to deduce
> its origin.
>
> Would it be possible to change Cython to stop putting timestamps into
> the source?
> I would also be happy if the granularity is reduced from seconds to
> hours/days or an non-default option to disable it (e.g. an environment
> variable).
>
>
> Additionally I think it might be useful to have a vendor id added to the
> version number added to the source files. This would allow easier origin
> tracking of files created with distribution patched versions of Cython.
> E.g. if Debian patches 0.20 of Cython it puts in
>
>  Generated by Cython 0.20 (Debian revision 3)
>
> Where the vendor id is added via a Distribution patch of the package and
> none is emitted for upstream builds.
> (CC. Debian maintainer of Cython for comments)
>
> Cheers,
> Julian Taylor
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] remove timestamps from generated source files

[Julian Taylor]

On 13.01.2014 22:24, Robert Bradshaw wrote:
> The timestamp is in a comment which ccache can strip before looking up
> items in the cache; are you sure this is an issue?

well this is embarrassing, it indeed strips the comments...
The issue was setup.py used a compiler symlink not present in my ccache
folder. I draw wrong conclusions to quickly ...
Sorry for the noise.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] h5py build broken by 0.20b2

[Robert Bradshaw]

I've verified that Stefan's latest fix lets h5py compile.

On Mon, Jan 13, 2014 at 10:49 AM, Julian Taylor
 wrote:
> On 13.01.2014 09:26, Stefan Behnel wrote:
>> Hi,
>> ...
>>>
>>> it was introduced in cython around this commit:
>>> b6b5152f386ddae503674cc26200a547f3b4c8b0
>>> properly handle expressions at the beginning of func/class/etc. blocks
>>>
>>> Is this an intentional change?
>>
>> Well, yes, but apparently one with unintended side-effects. :)
>>
>> I pushed a proposed quick-fix here, test case pending, so it would be great
>> if you or Yaroslav could give it a try.
>>
>> https://github.com/cython/cython/commit/279efc20fb8fb0bbcba5922afd010bdd3424e3b5
>>
>> Stefan
>
>
> No, I still get the same error message with cython master
> dab03a3cb796646c80d2f which includes the fix attempt.
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.20 beta 2

[Robert Bradshaw]

I fixed one of the tests that was trying to test using symlink:

https://github.com/cython/cython/commit/f236077786203f5f393dab3a707b82a62b4f4155

Is the list of failures notably worse than 0.19.x?

- Robert



On Mon, Jan 13, 2014 at 2:24 AM, Stefan Behnel  wrote:
> Hi Christoph,
>
> thanks for testing!
>
> Christoph Gohlke, 13.01.2014 09:46:
>> I have trouble running the tests (`runtests.py`) on Windows:
>> [...]
>> There are hundreds of test errors, most of kind `IOError: [Errno 24] Too
>> many open files`.
>
> The tests start consistently failing at some point, which hints at a file
> handle leak, either in the test runner or in the compiler.
>
>
>> The test output for 64 bit Python 2.7 with numpy 1.7.2 and msvc9 is at
>> .
>
> Yes, lots of problems and warnings. Might take a bit to look through all of
> them.
>
> Regarding this one:
>
> """
> compiling (cpp) and running final_method_T586 ... final_method_T586.c
> final_method_T586.c(8) : fatal error C1083: Cannot open include file:
> 'pyconfig.h': No such file or directory
> ERROR
> """
>
> Is there no "pyconfig.h" amongst the CPython header files on Windows?
>
> Stefan
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.20 beta 2

[Robert Bradshaw]

On Sun, Jan 12, 2014 at 12:20 AM, Stefan Behnel  wrote:
> Robert Bradshaw, 12.01.2014 08:53:
>> There was a lot of bugfixes and other good stuff on master, so I
>> pulled it in and am releasing another beta. You can find it at
>> http://cython.org/release/Cython-0.20b2.tar.gz . I don't anticipate
>> merging master again before the release, so this is nearly a release
>> candidate, but please let us know if you run into any regressions.
>
> The only thing that is still a bit unclear to me is whether calling
> object.__new__() from extension types was a good idea:
>
> https://github.com/cython/cython/commit/ddaffbe78c06e580ca0d9cd334c7a28a4de5c40b
>
> It means that we are now running a couple of safety checks inside of
> CPython on object instantiation that validate the arguments being passed
> into __init__() and __new__(), because object() doesn't expect any.
>
> http://thread.gmane.org/gmane.comp.python.cython.user/10267/focus=10519
>
> If we want to play safe for the release, we could restrict this change only
> to abstract classes for now (for which we know that CPython will generate
> an error). That was the original intention of the change anyway.

Yes, I'm going to "play it safe" and include the restrict-to-abstract change.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel