Re: How to easily identify that a FUNCTION_DECL is a lambda
On 07/18/2018 03:45 AM, Jason Merrill wrote: > On Mon, Jul 16, 2018 at 5:23 PM, Martin Liška wrote: >> For purpose of --coverage I would like to distinguish lambda functions >> among DECL_ARTIFICIAL functions. > > I'm curious, why? > > Jason > It's important for GCOV to report coverage for functions that are really present in a source file. Lambdas are such functions. On the other hand functions like _Z41__static_initialization_and_destruction_* or some implicit constructors (Centering<3>::Centering(Centering<3> const&)) should not be reported. It confuses a user. Note that both have a valid location and one can't distinguish them. That's why I would like to sort out lambdas from others. Martin
Re: [RFC] Adding Python as a possible language and it's usage
On Tue, Jul 17, 2018 at 2:49 PM Martin Liška wrote:
>
> Hi.
>
> I've recently touched AWK option generate machinery and it's quite unpleasant
> to make any adjustments. My question is simple: can we starting using a
> scripting
> language like Python and replace usage of the AWK scripts? It's probably
> question
> for Steering committee, but I would like to see feedback from community.
>
> There are some bulletins why I would like to replace current AWK scripts:
>
> 1) gcc/optc-save-gen.awk is full of copy&pasted code, due to lack of flags
> type classes multiple
> global variables are created (var_opt_char, var_opt_string, ...)
>
> 2) similar happens in gcc/opth-gen.awk
>
> 3) we do very many regex matches (mainly in gcc/opt-functions.awk), I believe
>we should come up with a structured option format that will make parsing
> and
>processing much simpler.
>
> 4) we can come up with new sanity checks of options:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81397
>
> 5) there are various targets that generate *.opt files, one example is ARM:
> gcc/config/arm/parsecpu.awk
>
> where transforms:
> ./gcc/config/arm/arm-cpus.in
>
> I guess having a well-defined structured format for *.opt files will make
> it easier to write generated opt files?
>
> I'm attaching a prototype that can transform optionlist into options-save.c
> that can be compiled and works.
>
> I'm looking forward to a feedback.
I guess we either need to document python as build requirement in
install.texi then,
it currently has
@item A POSIX or SVR4 awk
Necessary for creating some of the generated source files for GCC@.
If in doubt, use a recent GNU awk version, as some of the older ones
are broken. GNU awk version 3.1.5 is known to work.
alternatively we could handle the generated files like those we still
need flex for:
@item --enable-generated-files-in-srcdir
Neither the .c and .h files that are generated from Bison and flex nor the
info manuals and man pages that are built from the .texi files are present
in the SVN development tree. When building GCC from that development tree,
or from one of our snapshots, those generated files are placed in your
build directory, which allows for the source to be in a readonly
directory.
If you configure with @option{--enable-generated-files-in-srcdir} then those
generated files will go into the source directory. This is mainly intended
for generating release or prerelease tarballs of the GCC sources, since it
is not a requirement that the users of source releases to have flex, Bison,
or makeinfo.
We already conditionally require Perl for building for some targets so I wonder
if using perl would be better ...
Do we get rid of the AWK build requirement with your changes?
Richard.
> Martin
Re: [RFC] Adding Python as a possible language and it's usage
On 18/07/18 10:51, Richard Biener wrote:
> On Tue, Jul 17, 2018 at 2:49 PM Martin Liška wrote:
>>
>> Hi.
>>
>> I've recently touched AWK option generate machinery and it's quite unpleasant
>> to make any adjustments. My question is simple: can we starting using a
>> scripting
>> language like Python and replace usage of the AWK scripts? It's probably
>> question
>> for Steering committee, but I would like to see feedback from community.
>>
>> There are some bulletins why I would like to replace current AWK scripts:
>>
>> 1) gcc/optc-save-gen.awk is full of copy&pasted code, due to lack of flags
>> type classes multiple
>> global variables are created (var_opt_char, var_opt_string, ...)
>>
>> 2) similar happens in gcc/opth-gen.awk
>>
>> 3) we do very many regex matches (mainly in gcc/opt-functions.awk), I believe
>>we should come up with a structured option format that will make parsing
>> and
>>processing much simpler.
>>
>> 4) we can come up with new sanity checks of options:
>> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81397
>>
>> 5) there are various targets that generate *.opt files, one example is ARM:
>> gcc/config/arm/parsecpu.awk
>>
>> where transforms:
>> ./gcc/config/arm/arm-cpus.in
>>
>> I guess having a well-defined structured format for *.opt files will make
>> it easier to write generated opt files?
>>
>> I'm attaching a prototype that can transform optionlist into options-save.c
>> that can be compiled and works.
>>
>> I'm looking forward to a feedback.
>
> I guess we either need to document python as build requirement in
> install.texi then,
> it currently has
>
> @item A POSIX or SVR4 awk
>
> Necessary for creating some of the generated source files for GCC@.
> If in doubt, use a recent GNU awk version, as some of the older ones
> are broken. GNU awk version 3.1.5 is known to work.
>
> alternatively we could handle the generated files like those we still
> need flex for:
>
> @item --enable-generated-files-in-srcdir
> Neither the .c and .h files that are generated from Bison and flex nor the
> info manuals and man pages that are built from the .texi files are present
> in the SVN development tree. When building GCC from that development tree,
> or from one of our snapshots, those generated files are placed in your
> build directory, which allows for the source to be in a readonly
> directory.
>
> If you configure with @option{--enable-generated-files-in-srcdir} then those
> generated files will go into the source directory. This is mainly intended
> for generating release or prerelease tarballs of the GCC sources, since it
> is not a requirement that the users of source releases to have flex, Bison,
> or makeinfo.
>
> We already conditionally require Perl for building for some targets so I
> wonder
> if using perl would be better ...
>
> Do we get rid of the AWK build requirement with your changes?
>
Nope, the Arm port uses AWK for handling the CPU description tables. I
chose to use that specifically because it was already relied on for
other parts of the build system.
Please don't go down the Perl line, though...
R.
> Richard.
>
>> Martin
Re: [RFC] Adding Python as a possible language and it's usage
On Wed, 2018-07-18 at 11:51 +0200, Richard Biener wrote:
> On Tue, Jul 17, 2018 at 2:49 PM Martin Liška wrote:
> >
> > Hi.
> >
> > I've recently touched AWK option generate machinery and it's quite
> > unpleasant
> > to make any adjustments. My question is simple: can we starting
> > using a scripting
> > language like Python and replace usage of the AWK scripts? It's
> > probably question
> > for Steering committee, but I would like to see feedback from
> > community.
> >
> > There are some bulletins why I would like to replace current AWK
> > scripts:
> >
> > 1) gcc/optc-save-gen.awk is full of copy&pasted code, due to lack
> > of flags type classes multiple
> > global variables are created (var_opt_char, var_opt_string, ...)
> >
> > 2) similar happens in gcc/opth-gen.awk
> >
> > 3) we do very many regex matches (mainly in gcc/opt-functions.awk),
> > I believe
> >we should come up with a structured option format that will make
> > parsing and
> >processing much simpler.
> >
> > 4) we can come up with new sanity checks of options:
> > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81397
> >
> > 5) there are various targets that generate *.opt files, one example
> > is ARM:
> > gcc/config/arm/parsecpu.awk
> >
> > where transforms:
> > ./gcc/config/arm/arm-cpus.in
> >
> > I guess having a well-defined structured format for *.opt files
> > will make
> > it easier to write generated opt files?
> >
> > I'm attaching a prototype that can transform optionlist into
> > options-save.c
> > that can be compiled and works.
> >
> > I'm looking forward to a feedback.
>
> I guess we either need to document python as build requirement in
> install.texi then,
> it currently has
>
> @item A POSIX or SVR4 awk
>
> Necessary for creating some of the generated source files for GCC@.
> If in doubt, use a recent GNU awk version, as some of the older ones
> are broken. GNU awk version 3.1.5 is known to work.
>
> alternatively we could handle the generated files like those we still
> need flex for:
If we go down the "document Python as a build requirement" route, we
would need to decide on a minimum version, and what to do about Python
2 vs Python 3. We could restrict ourselves to the common subset of the
two languages, or to require Python 3 (or Python 2, I suppose).
If we want somewhat conservative minimum versions, one strategy might
be to require (Python 2.* (2.6 or later) OR Python 3 (3.3 or later)),
and code to the common subset of 2.6 and 3.3. Implicitly, this would
mean no 3rd-party modules; we'd be sticking to the Python standard
library.
Rationale:
Python 2.6 onwards is broadly compatible with Python 3.*. and is about
to be 10 years old. (IIRC it was the system python implementation in
RHEL 6). I'm guessing that many older systems have Python 2 installed,
but not Python 3, and anything we write is likely to be compatible with
even older Python 2.* implementations.
Python 3.3 reintroduced the 'u' prefix for unicode string literals (PEP
414), which makes it much easier to write scripts that work with both
2.* and 3.*. Python 3.3 is almost 6 years old.
(this is just a suggestion)
Dave
> @item --enable-generated-files-in-srcdir
> Neither the .c and .h files that are generated from Bison and flex
> nor the
> info manuals and man pages that are built from the .texi files are
> present
> in the SVN development tree. When building GCC from that development
> tree,
> or from one of our snapshots, those generated files are placed in
> your
> build directory, which allows for the source to be in a readonly
> directory.
>
> If you configure with @option{--enable-generated-files-in-srcdir}
> then those
> generated files will go into the source directory. This is mainly
> intended
> for generating release or prerelease tarballs of the GCC sources,
> since it
> is not a requirement that the users of source releases to have flex,
> Bison,
> or makeinfo.
>
> We already conditionally require Perl for building for some targets
> so I wonder
> if using perl would be better ...
>
> Do we get rid of the AWK build requirement with your changes?
>
> Richard.
>
> > Martin
Re: [RFC] Adding Python as a possible language and it's usage
On Wed, Jul 18, 2018 at 06:56:31AM -0400, David Malcolm wrote: > > alternatively we could handle the generated files like those we still > > need flex for: We can't, because unlike the flex output, the option handling is heavily target specific and storing in the tarball a collection of per-target specially generated results would be a nightmare. > Rationale: > > Python 2.6 onwards is broadly compatible with Python 3.*. and is about > to be 10 years old. (IIRC it was the system python implementation in > RHEL 6). I'm guessing that many older systems have Python 2 installed, > but not Python 3, and anything we write is likely to be compatible with > even older Python 2.* implementations. > > Python 3.3 reintroduced the 'u' prefix for unicode string literals (PEP > 414), which makes it much easier to write scripts that work with both > 2.* and 3.*. Python 3.3 is almost 6 years old. > > (this is just a suggestion) Then the question is also whether to use python2, python3 or python binaries. E.g. on some distros python without suffix generates ugly warnings and that already affects dg-extract-results.sh which just runs python -c ... rather than first looking for python2 or python3 and only falling back to python if those don't exist. Some other contrib/ scripts look only for python3. Jakub
Re: [RFC] Adding Python as a possible language and it's usage
On Wed, 18 Jul 2018 at 11:56, David Malcolm wrote: > Python 2.6 onwards is broadly compatible with Python 3.*. and is about > to be 10 years old. (IIRC it was the system python implementation in > RHEL 6). It is indeed. Without some regular testing with Python 2.6 it could be easy to introduce code that doesn't actually work on that old version. I did that recently, see PR 86112. This isn't an objection to using Python (I like it, and anyway I don't touch the parts of GCC that you're talking about using it for). Just a caution that trying to restrict yourself to a portable subset isn't always easy for casual users of a language (also a problem with C++98 vs C++11 vs C++14 as I'm sure many GCC devs are aware).
Re: [RFC] Adding Python as a possible language and it's usage
Jonathan Wakely : > On Wed, 18 Jul 2018 at 11:56, David Malcolm wrote: > > Python 2.6 onwards is broadly compatible with Python 3.*. and is about > > to be 10 years old. (IIRC it was the system python implementation in > > RHEL 6). > > It is indeed. Without some regular testing with Python 2.6 it could be > easy to introduce code that doesn't actually work on that old version. > I did that recently, see PR 86112. > > This isn't an objection to using Python (I like it, and anyway I don't > touch the parts of GCC that you're talking about using it for). Just a > caution that trying to restrict yourself to a portable subset isn't > always easy for casual users of a language (also a problem with C++98 > vs C++11 vs C++14 as I'm sure many GCC devs are aware). It's not very difficult to write "polyglot" Python that is indifferent to which version it runs under. I had to solve this problem for reposurgeon; techniques documented here... http://www.catb.org/esr/faqs/practical-python-porting/ -- http://www.catb.org/~esr/";>Eric S. Raymond My work is funded by the Internet Civil Engineering Institute: https://icei.org Please visit their site and donate: the civilization you save might be your own.
Re: [RFC] Adding Python as a possible language and it's usage
On Wed, 18 Jul 2018 at 13:06, Eric S. Raymond wrote: > > Jonathan Wakely : > > On Wed, 18 Jul 2018 at 11:56, David Malcolm wrote: > > > Python 2.6 onwards is broadly compatible with Python 3.*. and is about > > > to be 10 years old. (IIRC it was the system python implementation in > > > RHEL 6). > > > > It is indeed. Without some regular testing with Python 2.6 it could be > > easy to introduce code that doesn't actually work on that old version. > > I did that recently, see PR 86112. > > > > This isn't an objection to using Python (I like it, and anyway I don't > > touch the parts of GCC that you're talking about using it for). Just a > > caution that trying to restrict yourself to a portable subset isn't > > always easy for casual users of a language (also a problem with C++98 > > vs C++11 vs C++14 as I'm sure many GCC devs are aware). > > It's not very difficult to write "polyglot" Python that is indifferent > to which version it runs under. I had to solve this problem for > reposurgeon; techniques documented here... I don't see any mention of avoiding dict comprehensions (not supported until 2.7, so unusable on RHEL6/CentOS6 and SLES 11). I maintain it's easy to unwittingly use a feature (such as dict comprehensions) which works fine on your machine, but aren't supported by all versions you intend to support. Regular testing with the oldest version is needed to prevent that (which was the point I was making).
Re: How to easily identify that a FUNCTION_DECL is a lambda
On Wed, Jul 18, 2018 at 7:03 PM, Martin Liška wrote: > On 07/18/2018 03:45 AM, Jason Merrill wrote: >> On Mon, Jul 16, 2018 at 5:23 PM, Martin Liška wrote: >>> For purpose of --coverage I would like to distinguish lambda functions >>> among DECL_ARTIFICIAL functions. >> >> I'm curious, why? > > It's important for GCOV to report coverage for functions that are > really present in a source file. Lambdas are such functions. > On the other hand functions like _Z41__static_initialization_and_destruction_* > or some implicit constructors (Centering<3>::Centering(Centering<3> const&)) > should not be reported. It confuses a user. Note that both have a valid > location and one can't distinguish them. Those other functions may have *a* location, but only DECL_SOURCE_LOCATION; could we distinguish on that basis? Jason
Re: [RFC] Adding Python as a possible language and it's usage
On Wed, Jul 18, 2018, 7:15 AM Jonathan Wakely wrote: > On Wed, 18 Jul 2018 at 13:06, Eric S. Raymond wrote: > > > > Jonathan Wakely : > > > On Wed, 18 Jul 2018 at 11:56, David Malcolm wrote: > > > > Python 2.6 onwards is broadly compatible with Python 3.*. and is > about > > > > to be 10 years old. (IIRC it was the system python implementation in > > > > RHEL 6). > > > > > > It is indeed. Without some regular testing with Python 2.6 it could be > > > easy to introduce code that doesn't actually work on that old version. > > > I did that recently, see PR 86112. > > > > > > This isn't an objection to using Python (I like it, and anyway I don't > > > touch the parts of GCC that you're talking about using it for). Just a > > > caution that trying to restrict yourself to a portable subset isn't > > > always easy for casual users of a language (also a problem with C++98 > > > vs C++11 vs C++14 as I'm sure many GCC devs are aware). > > > > It's not very difficult to write "polyglot" Python that is indifferent > > to which version it runs under. I had to solve this problem for > > reposurgeon; techniques documented here... > > I don't see any mention of avoiding dict comprehensions (not supported > until 2.7, so unusable on RHEL6/CentOS6 and SLES 11). > > I maintain it's easy to unwittingly use a feature (such as dict > comprehensions) which works fine on your machine, but aren't supported > by all versions you intend to support. Regular testing with the oldest > version is needed to prevent that (which was the point I was making). > I think the RTEMS Community may be a good precedence here. RTEMS is always cross compiled and we are as host agnostic as possible. We use as close to the latest release of GCC, binutils, gdb, and newlib as possible. Our host side tools are in a combination of Python and C++. We use Sphinx for documentation. We are careful to use the Python on RHEL 6 as a baseline. You can build an RTEMS environment there. But at least one of the Sphinx pieces requires a Python of at least RHEL 7 vintage. We have a lot of what I will politely call institutional and large organization users who have to adhere to strict IT policies. I think RHEL 7 is common but can't swear there is no RHEL 6 out there and because of that, we set the Python 2.x as a minimum. Yes these are old. And for native new distribution use, it doesn't matter. But for cross and local upgrades, old distributions matter. Particularly those targeting enterprise users. And those are glacially slow. As an aside, it was not being able to build the RTEMS documentation that pushed me off RHEL 6 as my primary personal environment last year. I wanted to be using the oldest distribution I thought was in use in our community. --joel RTEMS >
Re: How to easily identify that a FUNCTION_DECL is a lambda
On 07/18/2018 02:40 PM, Jason Merrill wrote:
> On Wed, Jul 18, 2018 at 7:03 PM, Martin Liška wrote:
>> On 07/18/2018 03:45 AM, Jason Merrill wrote:
>>> On Mon, Jul 16, 2018 at 5:23 PM, Martin Liška wrote:
For purpose of --coverage I would like to distinguish lambda functions
among DECL_ARTIFICIAL functions.
>>>
>>> I'm curious, why?
>>
>> It's important for GCOV to report coverage for functions that are
>> really present in a source file. Lambdas are such functions.
>> On the other hand functions like
>> _Z41__static_initialization_and_destruction_*
>> or some implicit constructors (Centering<3>::Centering(Centering<3> const&))
>> should not be reported. It confuses a user. Note that both have a valid
>> location and one can't distinguish them.
>
> Those other functions may have *a* location, but only
> DECL_SOURCE_LOCATION; could we distinguish on that basis?
>
> Jason
>
I don't know how to distinguish them. Consider:
reference_to_this
>
QI
size
unit-size
align:8 warn_if_not_align:0 symtab:0 alias-set -1 canonical-type
0x7fffe82f62a0 method basetype
arg-types
chain >>
throws >
pointer_to_this >
nothrow public static weak autoinline decl_3 decl_5 QI tramp3d-v4.cpp:867:7
align:16 warn_if_not_align:0 context
initial abstract_origin
result
ignored VOID tramp3d-v4.cpp:867:7
align:8 warn_if_not_align:0 context >
full-name "BinaryNode >, double, BrickView>,
Scalar >::~BinaryNode() noexcept"
pending-inline-info 0x7fffe8341510
arguments
readonly unsigned DI
size
unit-size
align:64 warn_if_not_align:0 symtab:0 alias-set -1 canonical-type
0x7fffe82af150>
readonly unsigned read DI tramp3d-v4.cpp:34712:13 size unit-size
align:64 warn_if_not_align:0 context abstract_origin arg-type
>
struct-function 0x7fffe833e580 chain >
it's DECL_SOURCE_LOCATION is:
$2 = 200862249
and expand_location (DECL_SOURCE_LOCATION (current_function_decl)) is:
(gdb) p xloc
$1 = {
file = 0x2a58df0 "tramp3d-v4.cpp",
line = 867,
column = 7,
data = 0x0,
sysp = false
}
Martin
Re: [RFC] Adding Python as a possible language and it's usage
On 18.07.2018 14:49, Joel Sherrill wrote: > On Wed, Jul 18, 2018, 7:15 AM Jonathan Wakely wrote: > >> On Wed, 18 Jul 2018 at 13:06, Eric S. Raymond wrote: >>> >>> Jonathan Wakely : On Wed, 18 Jul 2018 at 11:56, David Malcolm wrote: > Python 2.6 onwards is broadly compatible with Python 3.*. and is >> about > to be 10 years old. (IIRC it was the system python implementation in > RHEL 6). It is indeed. Without some regular testing with Python 2.6 it could be easy to introduce code that doesn't actually work on that old version. I did that recently, see PR 86112. This isn't an objection to using Python (I like it, and anyway I don't touch the parts of GCC that you're talking about using it for). Just a caution that trying to restrict yourself to a portable subset isn't always easy for casual users of a language (also a problem with C++98 vs C++11 vs C++14 as I'm sure many GCC devs are aware). >>> >>> It's not very difficult to write "polyglot" Python that is indifferent >>> to which version it runs under. I had to solve this problem for >>> reposurgeon; techniques documented here... >> >> I don't see any mention of avoiding dict comprehensions (not supported >> until 2.7, so unusable on RHEL6/CentOS6 and SLES 11). >> >> I maintain it's easy to unwittingly use a feature (such as dict >> comprehensions) which works fine on your machine, but aren't supported >> by all versions you intend to support. Regular testing with the oldest >> version is needed to prevent that (which was the point I was making). >> > > I think the RTEMS Community may be a good precedence here. RTEMS is always > cross compiled and we are as host agnostic as possible. We use as close to > the latest release of GCC, binutils, gdb, and newlib as possible. Our host > side tools are in a combination of Python and C++. We use Sphinx for > documentation. > > We are careful to use the Python on RHEL 6 as a baseline. You can build an > RTEMS environment there. But at least one of the Sphinx pieces requires a > Python of at least RHEL 7 vintage. > > We have a lot of what I will politely call institutional and large > organization users who have to adhere to strict IT policies. I think RHEL 7 > is common but can't swear there is no RHEL 6 out there and because of that, > we set the Python 2.x as a minimum. > > Yes these are old. And for native new distribution use, it doesn't matter. > But for cross and local upgrades, old distributions matter. Particularly > those targeting enterprise users. And those are glacially slow. > > As an aside, it was not being able to build the RTEMS documentation that > pushed me off RHEL 6 as my primary personal environment last year. I wanted > to be using the oldest distribution I thought was in use in our community. doesn't RHEL 6 has overlays for that very reason to install a newer Python3? Please don't start with Python2 anymore. It's discontinued in less than two years and then you'll have distributions not having Python2 anymore. If you don't have a recent Python3, then you probably can build it for your platform itself. Python3 is also cross-buildable, and much easier to cross-build than guile or perl. Matthias
Re: [RFC] Adding Python as a possible language and it's usage
On Wed, Jul 18, 2018 at 5:29 PM, Matthias Klose wrote: > On 18.07.2018 14:49, Joel Sherrill wrote: > > On Wed, Jul 18, 2018, 7:15 AM Jonathan Wakely > wrote: > > > >> On Wed, 18 Jul 2018 at 13:06, Eric S. Raymond wrote: > >>> > >>> Jonathan Wakely : > On Wed, 18 Jul 2018 at 11:56, David Malcolm wrote: > > Python 2.6 onwards is broadly compatible with Python 3.*. and is > >> about > > to be 10 years old. (IIRC it was the system python implementation in > > RHEL 6). > > It is indeed. Without some regular testing with Python 2.6 it could be > easy to introduce code that doesn't actually work on that old version. > I did that recently, see PR 86112. > > This isn't an objection to using Python (I like it, and anyway I don't > touch the parts of GCC that you're talking about using it for). Just a > caution that trying to restrict yourself to a portable subset isn't > always easy for casual users of a language (also a problem with C++98 > vs C++11 vs C++14 as I'm sure many GCC devs are aware). > >>> > >>> It's not very difficult to write "polyglot" Python that is indifferent > >>> to which version it runs under. I had to solve this problem for > >>> reposurgeon; techniques documented here... > >> > >> I don't see any mention of avoiding dict comprehensions (not supported > >> until 2.7, so unusable on RHEL6/CentOS6 and SLES 11). > >> > >> I maintain it's easy to unwittingly use a feature (such as dict > >> comprehensions) which works fine on your machine, but aren't supported > >> by all versions you intend to support. Regular testing with the oldest > >> version is needed to prevent that (which was the point I was making). > >> > > > > I think the RTEMS Community may be a good precedence here. RTEMS is > always > > cross compiled and we are as host agnostic as possible. We use as close > to > > the latest release of GCC, binutils, gdb, and newlib as possible. Our > host > > side tools are in a combination of Python and C++. We use Sphinx for > > documentation. > > > > We are careful to use the Python on RHEL 6 as a baseline. You can build > an > > RTEMS environment there. But at least one of the Sphinx pieces requires a > > Python of at least RHEL 7 vintage. > > > > We have a lot of what I will politely call institutional and large > > organization users who have to adhere to strict IT policies. I think > RHEL 7 > > is common but can't swear there is no RHEL 6 out there and because of > that, > > we set the Python 2.x as a minimum. > > > > Yes these are old. And for native new distribution use, it doesn't > matter. > > But for cross and local upgrades, old distributions matter. Particularly > > those targeting enterprise users. And those are glacially slow. > > > > As an aside, it was not being able to build the RTEMS documentation that > > pushed me off RHEL 6 as my primary personal environment last year. I > wanted > > to be using the oldest distribution I thought was in use in our > community. > > doesn't RHEL 6 has overlays for that very reason to install a newer > Python3? > EPEL provides python 3.4 for RHEL6. (EPEL is a non-official add-on repository, but I suspect the vast majority who aren't running some single-task server have it enabled) Don't know if there's something equivalent for SLES. > Please don't start with Python2 anymore. It's discontinued in less than two > years and then you'll have distributions not having Python2 anymore. +1 -- Janne Blomqvist
Re: [RFC] Adding Python as a possible language and it's usage
On Tue, 2018-07-17 at 14:49 +0200, Martin Liška wrote: > My question is simple: can we starting using a scripting language like > Python and replace usage of the AWK scripts? I wonder what will be the expected way to obtain a suitable version of Python if one is not available on the build machine? With awk I can build it from source pretty much anywhere. Is building newer versions of Python on older targets a similarly straightforward process (somehow I doubt it)? What about Windows?
Re: [RFC] Adding Python as a possible language and it's usage
On Tue, 17 Jul 2018 20:23:36 -0400 David Malcolm wrote: > On Tue, 2018-07-17 at 16:37 -0400, David Niklas wrote: > > > Hi. > > > > > > I've recently touched AWK option generate machinery and it's quite > > > unpleasant to make any adjustments. My question is simple: can we > > > starting using a scripting language like Python and replace usage > > > of > > > the AWK scripts? It's probably question for Steering committee, but > > > I > > > would like to see feedback from community. > [disclosure: I'm a CPython core developer, albeit a rather dormant one, > and have made contributions to PyPy] Very good. > > As a FLOSS dev and someone who is familiar with both languages in > > question, I'd like to point out that python is an unstable > > language. > > > It > > has matured and changed a lot over the years. > > Depends on your meaning of "unstable". The changes are, IMHO, > extremely well-documented e.g.: > > https://docs.python.org/3/whatsnew/3.7.html > > and the documentation tells you precisely in which version each feature > became available; see e.g.: > https://docs.python.org/3/library/re.html#re.subn > for examples of this. And that is what I mean. I changes. I have compiled C code from 20 years ago and it works as expected. Many Python packages are still awaiting migration from 2 to 3 and 3.x series does change things. My argument is based on the fact that maintaining python code requires much more work than some other langs. > > The tools like python's > > 2to3 tool have gained an infamous reputation. > > OTOH, awk is very stable. I have been on the GNU variant's ML for > > some > > time and I have noticed that when a question over implementation > > arises > > they go looking at and, when necessary, consulting what the other > > awks are > > doing. For Python there is only one implementation, thus only one way > > of > > thinking about how it works unless you want to change something in > > the > > core language. > > There are multiple implementations of Python. > > CPython is the original one, but of the actively-developed > implementations there's also PyPy and IronPython, along with Jython, > and others. And yes, people talk to each other. If memory serves, ~1 year ago PyPy was not recommended by the Gentoo devs for a python implementation because it was considered unstable. Jython is integrating python with java so I did not consider it a "pure" python implimentation. I did not know that CPython was the original. I seem to remember that it was intended to convert python to C and was not yet complete. I can't comment about the IronPython, but it is good to know that crosstalk does occur. I use python3 when I need python. > > Gentoo's portage is an excellent example of a good language gone bad > > through less than ideal programming in python and it seems to me > > that, > > based on the description above, the awk code in gcc needs a code base > > cleanup and decrustification, not rewritten in the latest and > > greatest > > language simply because it is *the fad* of the day. > > I get the impression you've had a bad experience with Python in the > past, and that this is why you sent this email. Not really... For the curios my story is this: I wanted to learn to program and C was the dreaded language of the day. Ruby and Python3 were recommended. I tried to learn first ruby and then python with little success. I decided to try the hardest language I could find, since 2 years in, the "easy" ones were not working out. I leaned C in no time, even a perfect understanding of pointers came to me in 6 months time and I realized that OO and my brain did not like each other. I can program in python and other OO langs, but I am always running into 2 vs. 3 problems and each version seems to add something that I know other users might not have the correct version of python to support or breaks something that may or may not require changing ones program. Awk (my 4th lang), is a scripting language that I am also quite good at. I learned it because I needed to develop simple things faster. > If it's "the fad of the day", then according to: > https://www.tiobe.com/tiobe-index/python/ > it's been the fad of the year in 2007 and 2010, and is current the #4 > programming language. Maybe there's some inherent quality underlying > that long-term popularity that makes it more than, say just a "fad". Not to argue your point, but I have sadly witnessed as language after language is promoted by employers and educators such that I fear that the numbers of devs interested in a particular language is often times skewed instead of developers developing their interests organically. > Using a popular programming language will make it easier for GCC to get > new contributors. Until it becomes less popular... And gcc is for compiling C code, so we need more C devs than any other language :) > > And yes, by spelling > > python out as *the* language of choice without any other options Mr.
Re: [RFC] Adding Python as a possible language and it's usage
> On Jul 18, 2018, at 11:13 AM, Boris Kolpackov wrote: > > On Tue, 2018-07-17 at 14:49 +0200, Martin Liška wrote: > >> My question is simple: can we starting using a scripting language like >> Python and replace usage of the AWK scripts? > > I wonder what will be the expected way to obtain a suitable version of > Python if one is not available on the build machine? With awk I can > build it from source pretty much anywhere. Is building newer versions > of Python on older targets a similarly straightforward process (somehow > I doubt it)? What about Windows? It's the same sort of thing: untar the sources, configure, make, make install. The code is larger than awk but the process is no more difficult. For Windows there are pre-built kits. Ditto for a number of other popular operating systems. paul
Re: [RFC] Adding Python as a possible language and it's usage
Paul Koning writes: > > On Jul 18, 2018, at 11:13 AM, Boris Kolpackov > > wrote: > > > > I wonder what will be the expected way to obtain a suitable version of > > Python if one is not available on the build machine? With awk I can > > build it from source pretty much anywhere. Is building newer versions > > of Python on older targets a similarly straightforward process (somehow > > I doubt it)? What about Windows? > > It's the same sort of thing: untar the sources, configure, make, make > install. Will this also install all the Python packages one might plausible want to use in GCC?
Re: [RFC] Adding Python as a possible language and it's usage
> On Jul 18, 2018, at 1:22 PM, Boris Kolpackov wrote: > > Paul Koning writes: > >>> On Jul 18, 2018, at 11:13 AM, Boris Kolpackov >>> wrote: >>> >>> I wonder what will be the expected way to obtain a suitable version of >>> Python if one is not available on the build machine? With awk I can >>> build it from source pretty much anywhere. Is building newer versions >>> of Python on older targets a similarly straightforward process (somehow >>> I doubt it)? What about Windows? >> >> It's the same sort of thing: untar the sources, configure, make, make >> install. > > Will this also install all the Python packages one might plausible want > to use in GCC? It installs the entire standard Python library (corresponding to the 1800+ pages of the library manual). I expect that will easily cover anything GCC might want to do. paul
Re: [GSOC] LTO dump tool project
Hi,
I tried doing as suggested
+ dfi.pflags = 0;
+ dump_switch_p_1 (arg, &dfi, false);
1.> the value of dfi.pflags is not changing even if different command
line options are passed like -fdump-blocks or -fdump-vops
2.> what is the significance of bool doglob?
Please find the diff file attached herewith.
Please advise.
Hrishikesh
On Tue, Jul 17, 2018 at 6:30 PM, Martin Liška wrote:
> On 07/16/2018 05:44 PM, Hrishikesh Kulkarni wrote:
>> Hi,
>>
>> As suggested I have created command line option -optimized=[none,
>> blocks, stats, vops] to dump the respective gimple bodies of all
>> functions.
>>
>> for example:
>>
>> -optimized=blocks will dump
>
>
> Hi.
>
> The name does not make much sense and you removed option that
> limited function name. So instead I would create something like:
>
> -fdump-body=foo
> and
> -fdump-level=optimized
>
> I don't like how you do manual parsing in:
> + if (!strcmp (flag_optimized, "blocks"))
> +flags = TDF_BLOCKS;
> + else if (!strcmp (flag_optimized, "stats"))
> +flags = TDF_STATS;
> + else if (!strcmp (flag_optimized, "vops"))
> +flags = TDF_VOPS;
> + else if (!strcmp (flag_optimized, "none"))
> +flags = TDF_NONE;
>
> As I wrote in previous email, take a look how it's done
> in dump_switch_p_1 (const char *arg, struct dump_file_info *dfi, bool doglob)
> and reuse that functionality.
>
> Martin
>
>>
>> Gimple body of function: main
>> main ()
>> {
>> ;; basic block 2, loop depth 0
>> ;;pred: ENTRY
>> printf ("%d", 8);
>> return 0;
>> ;;succ: EXIT
>>
>> }
>>
>>
>> Gimple body of function: bar
>> bar (int a, int b, int cond)
>> {
>> int ret;
>>
>> ;; basic block 2, loop depth 0
>> ;;pred: ENTRY
>> if (cond_2(D) != 0)
>> goto ; [50.00%]
>> else
>> goto ; [50.00%]
>> ;;succ: 3
>> ;;4
>>
>> ;; basic block 3, loop depth 0
>> ;;pred: 2
>> ret_5 = a_3(D) + b_4(D);
>> goto ; [100.00%]
>> ;;succ: 5
>>
>> ;; basic block 4, loop depth 0
>> ;;pred: 2
>> ret_6 = a_3(D) - b_4(D);
>> ;;succ: 5
>>
>> ;; basic block 5, loop depth 0
>> ;;pred: 3
>> ;;4
>> # ret_1 = PHI
>> return ret_1;
>> ;;succ: EXIT
>>
>> }
>>
>>
>> Gimple body of function: foo
>> foo (int a, int b)
>> {
>> ;; basic block 2, loop depth 0
>> ;;pred: ENTRY
>> _3 = a_1(D) + b_2(D);
>> return _3;
>> ;;succ: EXIT
>>
>> }
>>
>> I have pushed the changes to the repo. Please find the diff file
>> attached herewith.
>>
>> Regards,
>>
>> Hrishikesh
>>
>> On Fri, Jul 13, 2018 at 2:47 PM, Martin Liška wrote:
>>> On 07/12/2018 08:05 PM, Hrishikesh Kulkarni wrote:
Hi,
I have added command line options:
-body=
To dump gimple body (TDF_NONE) of specific function.
-optimized-blocks=
To dump gimple body (TDF_BLOCKS) of specific function.
-optimized-stats=
To dump gimple body (TDF_STATS) of specific function.
-optimized-vops=
To dump gimple body (TDF_VOPS) of specific function.
>>>
>>> Hi.
>>>
>>> Thanks for it. However I would expect something more smart:
>>> -optimized=[vops,stats,block]. Note that you should do similar
>>> what's done in: gcc/dumpfile.c
>>>
>>> int
>>> gcc::dump_manager::
>>> dump_switch_p_1 (const char *arg, struct dump_file_info *dfi, bool doglob)
>>>
>>> that will automatically parse dump_flags_t flags;
>>>
>>> Then the copy&parse will not be needed.
>>>
for example:
-body=foo will dump
Gimple body of function: foo
foo (int a, int b)
{
[local count: 1073741825]:
_3 = a_1(D) + b_2(D);
return _3;
}
-optimized-blocks=foo will dump
Gimple body of function: foo
foo (int a, int b)
{
;; basic block 2, loop depth 0
;;pred: ENTRY
_3 = a_1(D) + b_2(D);
return _3;
;;succ: EXIT
}
-optimized-stats=foo will dump
Gimple body of function: foo
foo (int a, int b)
{
[local count: 1073741825]:
_3 = a_1(D) + b_2(D);
return _3;
}
-optimized-vops=foo will dump
Gimple body of function: foo
foo (int a, int b)
{
[local count: 1073741825]:
_3 = a_1(D) + b_2(D);
# VUSE <.MEM_4(D)>
return _3;
}
I have pushed the changes to the current branch. Please find the diff
file attached herewith.
I have tried to follow the coding conventions as far as possible in
this patch. Please let me know if I am missing out something so that I
can improve the same while refactoring and clean up as suggested in
the previous mail.
>>>
>>> As mentioned in the previous email, indentation level is 2. And every 8
>>> spaces
>>> are replaced with a tabular. In our patch, you use indentation level equal
>>> to
>>> one tab, which
Re: [RFC] Adding Python as a possible language and it's usage
On 18.07.2018 19:29, Paul Koning wrote: > > >> On Jul 18, 2018, at 1:22 PM, Boris Kolpackov wrote: >> >> Paul Koning writes: >> On Jul 18, 2018, at 11:13 AM, Boris Kolpackov wrote: I wonder what will be the expected way to obtain a suitable version of Python if one is not available on the build machine? With awk I can build it from source pretty much anywhere. Is building newer versions of Python on older targets a similarly straightforward process (somehow I doubt it)? What about Windows? >>> >>> It's the same sort of thing: untar the sources, configure, make, make >>> install. Windows binaries and MacOSX binaries are available from upstream. The build process on *ix targets is autoconf based and easy as for awk/gawk. >> Will this also install all the Python packages one might plausible want >> to use in GCC? some extension modules depend on external libraries, but even if those don't exist, the build succeeds without building these extension modules. The sources come with embedded libs for zlib, libmpdec, libexpat. They don't include libffi (only in 3.7), libsqlite, libgdbm, libbluetooth, libdb. I suppose the usage of such modules should be banned by policy. The only needed thing is any of libdb (Berkley/SleepyCat) or gdbm to build the anydbm module which might be necessary. > It installs the entire standard Python library (corresponding to the 1800+ > pages of the library manual). I expect that will easily cover anything GCC > might want to do. The current usage of awk and perl doesn't include any third party libraries. That's where the usage of Python should start with. Matthias
Re: [RFC] Adding Python as a possible language and it's usage
Jonathan Wakely : > I don't see any mention of avoiding dict comprehensions (not supported > until 2.7, so unusable on RHEL6/CentOS6 and SLES 11). That is correct. The HOWTO introduction does say that its techniques won't guarantee 2.6 compatibility. That would have been a great deal more difficult - some 3.x syntax backported into 2.7.2 makes a large difference here. In practice, no deployment of reposurgeon or src or doclifter or any of the other polyglot Python code I maintain has tripped over this, or at least I'm not seeing issue reports about it. Python devteam support for Python 2.6 terminated in 2013. > I maintain it's easy to unwittingly use a feature (such as dict > comprehensions) which works fine on your machine, but aren't supported > by all versions you intend to support. Regular testing with the oldest > version is needed to prevent that (which was the point I was making). Yes. This is why reposurgeon, doclifter, and cvs-fast-export both have regression-test suites that exercise all Python code under both 2 and 3, a practice I strongly recommend. Python 2.7 is scheduled for EOL in 2020. My plan is to retain 2.7 support in my code until 2022. I report that my practices are keeping the frequency of Python port defects I hear about to zero. I understand that GCC may have different constraints. -- http://www.catb.org/~esr/";>Eric S. Raymond My work is funded by the Internet Civil Engineering Institute: https://icei.org Please visit their site and donate: the civilization you save might be your own.
Re: [RFC] Adding Python as a possible language and it's usage
On Wed, Jul 18, 2018 at 11:51:36AM +0200, Richard Biener wrote: > We already conditionally require Perl for building for some targets so I > wonder > if using perl would be better ... At least perl is GPL (Python is not). What would the advantage of using Python be? I haven't heard any yet. Awk may be a bit clunky but at least it is easily readable for anyone. Segher
gcc-6-20180718 is now available
Snapshot gcc-6-20180718 is now available on ftp://gcc.gnu.org/pub/gcc/snapshots/6-20180718/ and on various mirrors, see http://gcc.gnu.org/mirrors.html for details. This snapshot has been generated from the GCC 6 SVN branch with the following options: svn://gcc.gnu.org/svn/gcc/branches/gcc-6-branch revision 262865 You'll find: gcc-6-20180718.tar.xzComplete GCC SHA256=b65e05ecae9295c4ddf23969b91b9b32ca19f1e7dce7b5781a742c8b1268aee1 SHA1=8d103af160a3b3157c331ad6ef67accf49bb07ac Diffs from 6-20180711 are available in the diffs/ subdirectory. When a particular snapshot is ready for public consumption the LATEST-6 link is updated and a message is sent to the gcc list. Please do not use a snapshot before it has been announced that way.
