Hi Gaetan, Thanks for the extensive answer. I hope with my replies below things should be much clearer.
On 30 January 2017 at 15:18, Gaetan Nadon <[email protected]> wrote: > I no longer have a development environment so I cannot try anything. > > Maybe I missed the core idea of the patches. The srcdir is read only, > correct? > s/is read-only/is read-only outside of the initial make dist/ To elaborate a bit - afaict make distcheck does the following - initial make dist - extract tarball, and run make all, make check, make dist (I'll refer to this secondary one) et al. > Unless I remember it incorrectly, the INSTALL and ChangeLog files are > generated in the srcdir as they would normally be checked-in in git. > Automake expects them to be there. It is the Xorg project who decided to > generate them, not Automake. This situation is not "normal" to begin with. > Agreed. > I assume you tried to build a git repo (AKA module) with the source code > sitting on a read-only filesystem. Correct? > Not quite - I've reworked (simplified?) release.sh which pointed out the (ab)use here. > What are the reported failures? Did the build stop because an O/S error? Or > was it just that both files did not get generated. > In a gist "make distcheck failed" - upon the second `make dist' there was an IO error. Note: the INSTALL case is extra picky, so let's assume that all cases below are dealing with ChangeLog. > Currently: the INSTALL_CMD copies the INSTALL file from the datadir to > top_srcdir/.INSTALL.tmp which is RO. That should fail on a RO filesystem, > correct? IS that the problem you are trying to solve? > Precisely. > Proposed patch: the INSTALL_CMD copies the INSTALL file from the datadir to > top_builddir/.INSTALL.tmp/ which is RW. That should work. A .INSTALL.tmp > file now sits among the object code created by the compiler. > Correct. > Next the top_builddir/.INSTALL.tmp is moved to the top_srcdir/INSTALL on a > filesystem which is RO. That should equally fail, correct? > > I am assuming that if one cannot copy a file onto a RO filesystem, then one > cannot move a file in there as well. Correct? > This part will succeed during the initial "make dist". The secondary one has srcdir as RO so we'll end up in the "else" statement - rm .FOO.tmp; touch FOO. Note that if we reach the "else", we're a) RO/RW file system and the file is present - touch is a no-op/succeeds, respectively or b) RW file system and the file is missing (building from a tarball and manually removed the file) - touch will create a dummy one. > On another topic, any change in the macros affect all modules (or git repos) > in the X project. Extensive testing is required. The fix must also be > backwards compatible with existing tarballs floating in the field.To get > started quickly, try a few modules that have generated code and > documentation. These are good test cases. As I remember it, all modules > listed in build.sh built successfully with make dist. > I might be overly confident, but barring crazy hacks elsewhere things should work across the board. Then again, I was expecting xorg-macros to be missing blatant bugs so I'm updating the script and pulling $world just in case. Related questions: Do you know of a script/semi-automated way to create a modfile to feed into release.sh ? I'm likely blind, but it seems that build.sh does not have an option to only pull/clone the repos, without building, does it ? I would be nice to do so and use release.sh which already does enough build/etc. checking/testing. Looking at build.sh I ended up adding 90 missing repos with fonts, uitils, docs, others still to check/add. Not sure if/how much are supposed to be legacy/deprecated but we shall see - if the newly added ones start failing I'll just pull them out. > A more general question, would a module generating source using lex and yacc > would build on a RO filesystem? > > The requirement here is that the generated code requires lex and yacc > programs to generate some C code when the repo is "configured" (autogen.sh). > That generated code is then shipped in a tarball where the tarball builder > only compiles the C generated code without the need to have the lex and yacc > programs installed. A tarball builder may not run autogen.sh as this is a > module configuration step and not a build step. > IMHO [definitely _not_ an expert wrt generated files] these ought to be handled like INSTALL/ChangeLog - generate once at initial [non RO] make dist, etc. Such that building from tarball is OK, even if one does make clean. Tl;Dr; Fully agree. <thinking out loud> IIRC in mesa we generate those files in builddir and think that we purge them in make clean stage. It's been on my list of things to check/fix but other distractions keep on coming up ;-) </thinking out loud> > If your organization has a build strategy to use shared shared RO > filesystems, this may not work for some modules. I had never seen this tried > before. The INSTALL and ChangeLog files are not the only files created in > the srcdir during the configuration step. > > An alternative would be to "configure" (autogen.sh) the module (without > running make) on a RW filesystem and then move the code to a shared RO > filesystem where it would be just "built" (make all). > Not sure what you mean with "your organisation", but let's leave this out for a moment. Afaict being able to do `git clone ...foo && cd foo && ./autogen.sh && make distcheck' should just work, shouldn't it ? That's what (barring a unrelated sidestep) my earlier changes to release.sh did, which in turn pointed to the issues here. I think that you'll be able to quickly see the issue in question, if you give the above one line (git clone..) a try. Atm it seems that you're making quite nice, yet not strictly unrelated assumptions. Can I bother you with that one, please ? In reminds me of our first experience, where I came across another funny build-related predicament are your deduction seemed, ahem, way off [1] [2]. Although admittedly my wording was ... short, to put it mildly. Thanks Emil [1] https://lists.freedesktop.org/archives/xorg-devel/2012-October/034131.html [2] https://cgit.freedesktop.org/xorg/xserver/commit/?id=ea3b7db4b747f883ef7b8b524ee13bf8a63add14 _______________________________________________ [email protected]: X.Org development Archives: http://lists.x.org/archives/xorg-devel Info: https://lists.x.org/mailman/listinfo/xorg-devel
