On Fri, 15 Mar 2024 at 15:13, Eric Gallager <eg...@gwmail.gwu.edu> wrote: > > On Fri, Mar 15, 2024 at 4:53 AM Christophe Lyon via Gcc <gcc@gcc.gnu.org> > wrote: > > > > On Thu, 14 Mar 2024 at 19:10, Simon Marchi <sim...@simark.ca> wrote: > > > > > > > > > > > > On 2024-03-13 04:02, Christophe Lyon via Gdb wrote: > > > > Hi! > > > > > > > > After recent discussions on IRC and on the lists about maintainer-mode > > > > and various problems with auto-generated source files, I've written > > > > this small prototype. > > > > > > > > Based on those discussions, I assumed that people generally want to > > > > update autotools files using a script similar to autoregen.py, which > > > > takes care of running aclocal, autoheader, automake and autoconf as > > > > appropriate. > > > > > > > > What is currently missing is a "simple" way of regenerating other > > > > files, which happens normally with --enable-maintainer-mode (which is > > > > reportedly broken). This patch as a "regenerate" Makefile target > > > > which can be called to update those files, provided > > > > --enable-maintainer-mode is used. > > > > > > > > I tried this approach with the following workflow for binutils/gdb: > > > > - run autoregen.py in srcdir > > > > - cd builddir > > > > - configure --enable-maintainer-mode > > > > - make all-bfd all-libiberty regenerate -j1 > > > > - for gdb: make all -C gdb/data-directory -j1 > > > > - make all -jXXX > > > > > > > > Making 'all' in bfd and libiberty is needed by some XXX-gen host > > > > programs in opcodes. > > > > > > > > The advantage (for instance for CI) is that we can regenerate files at > > > > -j1, thus avoiding the existing race conditions, and build the rest > > > > with -j XXX. > > > > > > > > Among drawbacks: > > > > - most sub-components use Makefile.am, but gdb does not: this may make > > > > maintenance more complex (different rules for different projects) > > > > - maintaining such ad-hoc "regenerate" rules would require special > > > > attention from maintainers/reviewers > > > > - dependency on -all-bfd and all-libiberty is probably not fully > > > > intuitive, but should not be a problem if the "regenerate" rules > > > > are used after a full build for instance > > > > > > > > Of course Makefile.def/Makefile.tpl would need further cleanup as I > > > > didn't try to take gcc into account is this patch. > > > > > > > > Thoughts? > > > > > > My first thought it: why is it a Makefile target, instead of some script > > > on the side (like autoregen.sh). It would be nice / useful to be > > > able to it without configuring / building anything. For instance, the > > > autoregen buildbot job could run it without configuring anything. > > > Ideally, the buildbot wouldn't maintain its own autoregen.py file on the > > > side, it would just use whatever is in the repo. > > > > Firstly because of what you mention later: some regeneration steps > > require building host tools first, like the XXX-gen in opcodes. > > > > Since the existing Makefiles already contain the rules to autoregen > > all these files, it seemed natural to me to reuse them, to avoid > > reinventing the wheel with the risk of introducing new bugs. > > > > This involves changes in places where I've never looked at before, so > > I'd rather reuse as much existing support as possible. > > > > For instance, there are the generators in opcodes/, but also things in > > sim/, bfd/, updates to the docs and potfiles. In gcc, there's also > > something "unusual" in fixincludes/ and libgfortran/ > > > > In fact, I considered also including 'configure', 'Makefile.in', > > etc... in the 'regenerate' target, it does not seem natural to me to > > invoke a script on the side, where you have to replicate the behaviour > > of existing Makefiles, possibly getting out-of-sync when someone > > forgets to update either Makefile or autoregen.py. What is currently > > missing is a way to easily regenerate files without having to run a > > full 'make all' (which currently takes care of calling autoconf & > > friends to update configure/Makefile.in). > > > > But yeah, having to configure before being able to regenerate files is > > a bit awkward too :-) > > > > > > > > > > Looking at the rule to re-generate copying.c in gdb for instance: > > > > > > # Make copying.c from COPYING > > > $(srcdir)/copying.c: @MAINTAINER_MODE_TRUE@ $(srcdir)/../COPYING3 > > > $(srcdir)/copying.awk > > > awk -f $(srcdir)/copying.awk \ > > > < $(srcdir)/../COPYING3 > $(srcdir)/copying.tmp > > > mv $(srcdir)/copying.tmp $(srcdir)/copying.c > > > > > > There is nothing in this code that requires having configured the source > > > tree. This code could for instance be moved to some > > > generate-copying-c.sh script. generate-copying-c.sh could be called by > > > an hypothetical autoregen.sh script, as well as the copying.c Makefile > > > target, if we want to continue supporting the maintainer mode. > > Wouldn't it be more obscure than now? Currently such build rules are > > all in the relevant Makefile. You'd have to open several scripts to > > discover what's involved with updating copying.c > > > > Yeah I agree that it's good to keep all build rules in the Makefile; > if there's a possibility of something changing, things that depend > upon it need to know, and the best way to express those dependencies > is in the Makefile. > > > > > > > Much like your regenerate targets, an autoregen.sh script in a given > > > directory would be responsible to re-generate all the files in this > > > directory that are generated and checked in git. It would also be > > > responsible to call any autoregen.sh file in subdirectories. > > Makefiles already have all that in place :-) > > Except if you consider that you'd want to ignore timestamps and always > > regenerate things? > > > > > > > There's just the issue of files that are generated using tools that are > > > compiled. When experimenting with maintainer mode the other day, I > > > stumbled on the opcodes/i386-gen, for instance. I don't have a good > > > solution to that, except to rewrite these tools in a scripting language > > > like Python. > > > > So for opcodes, it currently means rewriting such programs for i386, > > aarch64, ia64 and luckily msp430/rl78/rx share the same opc2c > > generator. > > Also there are the files generated by cgen, too, which no one seems to
I had heard about cgen ages ago but I had forgotten about them. And your message made me discover configure's option --enable-cgen-maint :-) > know how to regenerate, either. And then in bfd there's that chew > program in the doc subdir. And then in the binutils subdirectory > proper there's that sysinfo tool for generating sysroff.[ch]. > > > Not sure how to find volunteers? > > > > Christophe > > > > > > > > Simon
