Re: [Cython] Status
On 1/02/20 3:29 pm, John Skaller2 wrote: But the all hell breaks loose for pointers. Your hack only works for rvalues. Yes, using these it's possible for Cython to accept something that will later be rejected by the C compiler. It's not perfect, but it gets the job done most of the time. -- Greg ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Size of output
Robert Bradshaw schrieb am 01.02.20 um 00:49: > Taking a quick glance at an auto-generated file for an empty .pyx, we have > [...] > ~300 lines module setup code. Even for trivial modules, we still > declare and call functions for creating globals, preparing types, etc. > even if we don't have any globals, types, etc. :) > I agree there's some fat that could be trimmed there, but not sure > it'd be worth the effort. +1 In any non-trivial module where the size really starts to matter, the portions of unused code that we generate will probably make no difference at all. I'll also add that it can sometimes be very difficult to avoid generating unused code due to target (CPython x.y, PyPy, …) specific utility code that comes with its own dependencies, which then may or may not be used on a given target. Removing the final 0.5% of code that is "unused in some cases" is really not something I would want to invest days into, each time we make a change in Cython. Stefan ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Size of output
> given target. Removing the final 0.5% of code that is "unused in some > cases" is really not something I would want to invest days into, each time > we make a change in Cython. But its not 0.5%. My problem is trying to read the generated code. A possibility might be an option which puts the boilerplate in one file and the variant code in another that then #includes the boilerplate. So the code is the same, but the main file is much easier to inspect to see what got generated. — John Skaller skal...@internode.on.net ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Status
> On 1 Feb 2020, at 20:00, Greg Ewing wrote: > > On 1/02/20 3:29 pm, John Skaller2 wrote: >> But the all hell breaks loose for pointers. Your hack only >> works for rvalues. > > Yes, using these it's possible for Cython to accept something > that will later be rejected by the C compiler. It's not perfect, > but it gets the job done most of the time. My concern is that the C compiler wont reject it, the program will corrupt data or crash: int32_t *x=..; *x = 42; int64_t *x=..; *x = 42; The C compiler will overwrite 4 or 8 bytes. Which one matters. In an rvalue context it probably doesn’t matter, in an lvalue context it does. But probably this doesn’t happen in Cython. — John Skaller skal...@internode.on.net ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Size of output
John Skaller2 schrieb am 01.02.20 um 15:32: > My problem is trying to read the generated code. You might be looking for "cython -a". Stefan ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Status
John Skaller2 schrieb am 01.02.20 um 15:38: > On 1 Feb 2020, at 20:00, Greg Ewing wrote: >> On 1/02/20 3:29 pm, John Skaller2 wrote: >>> But the all hell breaks loose for pointers. Your hack only >>> works for rvalues. >> >> Yes, using these it's possible for Cython to accept something >> that will later be rejected by the C compiler. It's not perfect, >> but it gets the job done most of the time. > > My concern is that the C compiler wont reject it, the program > will corrupt data or crash: > > int32_t *x=..; *x = 42; > int64_t *x=..; *x = 42; > > The C compiler will overwrite 4 or 8 bytes. Which one matters. As stated before, the C compiler will see the correct definition of the types in the header files. And the generated C code will use the same type names that the user had in their source code. "int64_t" will not magically become "int" there. Stefan ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Size of output
> On 2 Feb 2020, at 02:23, Stefan Behnel wrote: > > John Skaller2 schrieb am 01.02.20 um 15:32: >> My problem is trying to read the generated code. > > You might be looking for "cython -a". OK. I see these: ~/felix>../cython/cython.py usage: cython.py [-h] [-V] [-l] [-I INCLUDE_PATH] [-o OUTPUT_FILE] [-t] [-f] [-v] [-p] [--cleanup GENERATE_CLEANUP_CODE] [-w WORKING_PATH] [--gdb] [--gdb-outdir GDB_OUTDIR] [-D] [-a] [--annotate-fullc] [--annotate-coverage ANNOTATE_COVERAGE_XML] [--line-directives] [-+] [--embed] [-2] [-3] [--3str] [--lenient] [--capi-reexport-cincludes] [--fast-fail] [-Werror] [-Wextra] [-X NAME=VALUE,...] [-E NAME=VALUE,...] [sources [sources ...]] cython.py: error: cython: Need at least one source file What does -a do? I couldn’t find complete docs on the compiler switches although some are explained in the user guide. — John Skaller skal...@internode.on.net ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Status
>> >> My concern is that the C compiler wont reject it, the program >> will corrupt data or crash: >> >> int32_t *x=..; *x = 42; >> int64_t *x=..; *x = 42; >> >> The C compiler will overwrite 4 or 8 bytes. Which one matters. > > As stated before, the C compiler will see the correct definition of the > types in the header files. And the generated C code will use the same type > names that the user had in their source code. "int64_t" will not magically > become "int" there. Ah. Ok, I get it now. Thanks. The “int” is just telling Cython the type has the same operations as “int”, i.e. +, - *, / etc. Like I said, a poor mans version of a “type class”. But, the compiler emits the symbols the user wrote when required. — John Skaller skal...@internode.on.net ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Size of output
Interesting it’s not documented well, it definitely should be. It corresponds to this compiler option: https://cython.readthedocs.io/en/latest/src/userguide/source_files_and_compilation.html#Cython.Compiler.Options.annotate When you turn that option on or pass -a or —annotate to the compiler, cython generates a sidecar html file you can open in a browswer. The document initially just contains the cython source code, with each line highlighted shades of yellow to indicate the amount of python overhead that line invokes. You can also click on each line to see the corresponding generated C. It’s very helpful to understand how changes in cython syntax or even compiler options can influence the performance and level of safety of the generated C code (usually in an inverse relationship). On Sat, Feb 1, 2020 at 8:41 AM John Skaller2 wrote: > > > > On 2 Feb 2020, at 02:23, Stefan Behnel wrote: > > > > John Skaller2 schrieb am 01.02.20 um 15:32: > >> My problem is trying to read the generated code. > > > > You might be looking for "cython -a". > > OK. I see these: > > ~/felix>../cython/cython.py > usage: cython.py [-h] [-V] [-l] [-I INCLUDE_PATH] [-o OUTPUT_FILE] [-t] > [-f] > [-v] [-p] [--cleanup GENERATE_CLEANUP_CODE] [-w > WORKING_PATH] > [--gdb] [--gdb-outdir GDB_OUTDIR] [-D] [-a] > [--annotate-fullc] > [--annotate-coverage ANNOTATE_COVERAGE_XML] > [--line-directives] [-+] [--embed] [-2] [-3] [--3str] > [--lenient] [--capi-reexport-cincludes] [--fast-fail] > [-Werror] [-Wextra] [-X NAME=VALUE,...] [-E > NAME=VALUE,...] > [sources [sources ...]] > cython.py: error: cython: Need at least one source file > > What does -a do? I couldn’t find complete docs on the compiler switches > although some are explained in the user guide. > > — > John Skaller > skal...@internode.on.net > > > > > > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Size of output
Isn't it? https://cython.readthedocs.io/en/latest/src/tutorial/cython_tutorial.html#primes: Jump down to the part where it has "annotate=True" and it describes it in some detail. Also this takes all of thirty-seconds to try out and the result is pretty self explanatory. -- Warm Regards Prakhar Goel ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Size of output
> On 2 Feb 2020, at 03:06, Prakhar Goel wrote: > > Isn't it? > https://cython.readthedocs.io/en/latest/src/tutorial/cython_tutorial.html#primes: > Jump down to the part where it has "annotate=True" and it describes it > in some detail. > > Also this takes all of thirty-seconds to try out and the result is > pretty self explanatory. Ah, I missed that. The options are documented. What’s missing is the heading. The documentation is under “Jupyter Notebook”, whatever that is. I may have skipped reading that since I have no idea what it is. After that there is a heading, Compiler Options, but that actually refers to options set in setup.py rather than command line switches. https://cython.readthedocs.io/en/latest/src/userguide/source_files_and_compilation.html#Cython.Compiler.Options.annotate In any case whilst useful it’s not really what I would have liked to see, which is a file with just the “yellow” code in it, and no boilerplate. Be good to suppress the refnanny stuff too. Is that possible? That seems to be for debugging, yes? — John Skaller skal...@internode.on.net ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Size of output
On Sat, Feb 1, 2020 at 6:33 AM John Skaller2 wrote: > > > given target. Removing the final 0.5% of code that is "unused in some > > cases" is really not something I would want to invest days into, each time > > we make a change in Cython. > > But its not 0.5%. My problem is trying to read the generated code. > > A possibility might be an option which puts the boilerplate in one file > and the variant code in another that then #includes the boilerplate. > So the code is the same, but the main file is much easier to inspect > to see what got generated. The primary target of the generated code is a C compiler, though we do try to generate code that's reasonably comprehensible for a human as well. However, there's often a tradeoff to be made in terms of conciseness and readability of the generated C code and that of the Cython codebase itself, and the latter is arguably more important. Also, as mentioned earlier, the generated code also makes heavy use of macros (including much of the refnanny stuff) so that various decisions can be deferred to C compile time. I'm also sure there's room for improvement, but that might not be the highest priority for where to spend limited time. Hopefully the -a option is useful. - Robert ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
