Re: [PATCH v5 4/5] c++modules: report imported CMI files as dependencies
On Fri, Jun 23, 2023 at 08:12:41 -0400, Nathan Sidwell wrote: > On 6/22/23 22:45, Ben Boeckel wrote: > > On Thu, Jun 22, 2023 at 17:21:42 -0400, Jason Merrill wrote: > >> On 1/25/23 16:06, Ben Boeckel wrote: > >>> They affect the build, so report them via `-MF` mechanisms. > >> > >> Why isn't this covered by the existing code in preprocessed_module? > > > > It appears as though it is neutered in patch 3 where > > `write_make_modules_deps` is used in `make_write` (or will use that name > > Why do you want to record the transitive modules? I would expect just noting > the > ones with imports directly in the TU would suffice (i.e check the 'outermost' > arg) FWIW, only GCC has "fat" modules. MSVC and Clang both require the transitive closure to be passed. The idea there is to minimize the size of individual module files. If GCC only reads the "fat" modules, then only those should be recorded. If it reads other modules, they should be recorded as well. --Ben
Re: [PATCH v5 5/5] c++modules: report module mapper files as a dependency
On Fri, Jun 23, 2023 at 10:44:11 -0400, Jason Merrill wrote: > On 1/25/23 16:06, Ben Boeckel wrote: > > It affects the build, and if used as a static file, can reliably be > > tracked using the `-MF` mechanism. > > Hmm, this seems a bit like making all .o depend on the Makefile; it Technically this is true: the command line for the TU lives in said Makefile; if I updated it, a new TU would be really nice. This is a long-standing limitation of `make` though. FWIW, `ninja` fixes it by tracking the command line used and CMake's Makefiles generator handles it by storing TU flags in an included file and depending on that file from the TU output. > shouldn't be necessary to rebuild all TUs that use modules when we add > another module to the mapper file. If I change it from: ``` mod.a mod.a.cmi ``` to: ``` mod.a mod.a.replace.cmi ``` I'd expect a recompile. As with anything, this depends on the granularity of the mapper files. A global mapper file is very similar to a global response file and given that we don't have line-change granularity dependency tracking… > What is your expected use case for > this dependency? CMake, at least, uses a per-TU mapper file, so any build system using a similar strategy handling the above case would only affect TUs that actually list `mod.a`. --Ben
Re: [PATCH v5 3/5] p1689r5: initial support
On Fri, Jun 23, 2023 at 14:31:17 -0400, Jason Merrill wrote: > On 6/20/23 15:46, Ben Boeckel wrote: > > On Tue, Feb 14, 2023 at 16:50:27 -0500, Jason Merrill wrote: > >> On 1/25/23 13:06, Ben Boeckel wrote: > > >>> Header units (including the standard library headers) are 100% > >>> unsupported right now because the `-E` mechanism wants to import their > >>> BMIs. A new mode (i.e., something more workable than existing `-E` > >>> behavior) that mocks up header units as if they were imported purely > >>> from their path and content would be required. > >> >> I notice that the cpp dependency generation tries (in open_file_failed) > >> to continue after encountering a missing file, is that not sufficient > >> for header units? Or adjustable to be sufficient? > > > > No. Header units can introduce macros which can be used to modify the > > set of modules that are imported. Included headers are "discovered" > > dependencies and don't modify the build graph (just add more files that > > trigger a rebuild) and can be collected during compilation. Module > > dependencies are needed to get the build correct in the first place in > > order to: > > > > - order module compilations in the build graph so that imported modules > > are ready before anything using them; and > > - computing the set of flags needed for telling the compiler where > > imported modules' CMI files should be located. > > So if the header unit CMI isn't available during dependency generation, > would it be better to just #include the header? It's not so simple: the preprocessor state needs to isolate out `LOCAL_ONLY` from this case: ``` #define LOCAL_ONLY 1 import ; // The preprocessing of this should *not* see // `LOCAL_ONLY`. ``` > > Hmm. But `stdout` is probably fine to use for both though. Basically: > > > > if (fdeps_stream == out_stream && fdeps_stream != stdout) > >make_diagnostic_noise (); > > (fdeps_stream == deps_stream, but sure, that's reasonable. Done. > >> So, I take it this is the common use case you have in mind, generating > >> Make dependencies for the p1689 file? When are you thinking the Make > >> dependencies for the .o are generated? At build time? > > > > Yes. If an included file changes, the scanning should be performed > > again. The compilation will have its own `-MF` as well (which should > > point to the same files plus the CMI files it ends up reading). > > > >> I'm a bit surprised you're using .json rather than an extension that > >> indicates what the information is. > > > > I can change that; the filename doesn't *really* matter (e.g., CMake > > uses `.ddi` for "dynamic dependency information"). > > That works. Done. > >>> `-M` is about discovered dependencies: those that you find out while > >>> doing work. `-fdep-*` is about ordering dependencies: extracting > >>> information from file content in order to even order future work around. > >> > >> I'm not sure I see the distinction; Makefiles also express ordering > >> dependencies. In both cases, you want to find out from the files what > >> order you will want to process them in when building the project. > > > > Makefiles can express ordering dependencies, but not the `-M` snippets; > > these are for files that, if changed, should trigger a rebuild. This is > > > fundamentally different than module dependencies which instead indicate > > which *compiles* (or CMI generation if using a 2-phase setup) need to > > complete before compilation (or CMI generation) of the scanned TU can be > > performed. Generally generated headers will be ordered manually in the > > build system description. However, maintaining that same level for > > in-source dependency information on a per-source level is a *far* higher > > burden. > > The main difference I see is that the CMI might not exist yet. As you > say, we don't want to require people to write all the dependencies by > hand, but that just means we need to be able to generate the > dependencies automatically. In the Make-only model I'm thinking of, one > would collect dependencies on an initial failing build, and then start > over from the beginning again with the dependencies we discovered. It's > the same two-phase scan and build, but one that uses the same compile > commands for both phases. It's a potentially unbounded set of phases: - 2 phases per tool that is built that generates other module-using code for other tools: - scan files for toolA - build files for toolA - scan files written by toolA (for toolB) - build files written by toolA (for toolB) - … - if a referenced module does not exist, knowing when one is "done" is difficult (an import cycle would appear like this because while module X does exist, it depends on Y which itself claims a dependency on X). *Something* needs to interpret the information being provided in the `-fdeps-file=` and communicate back to the build tool what is going on. This requires:
RE: Issue generating GCC coverage report since r14-1625-geba3565ce6d766
Hi Roger,
sorry for late reply, rather unexpectedly I found myself traveling last
week.
On Fri, Jun 16 2023, Roger Sayle wrote:
> Hi Martin,
> It's great to hear from you. My apologies for the inconvenience.
> I believe that the problem has been solved by Jakub's patch:
> https://gcc.gnu.org/git/?p=gcc.git;a=commit;f=gcc/config/i386/i386.md;h=43a3252c42af12ad90082e4088ea58eecd0bf582
Unfortunately the situation remains the same, even with the above patch
in master.
The workaround I hoped for (adding --ignore-errors mismatch) does not
help either, the whole process then fails later on. Perhaps adding
another --ignore-errors elsewhere may fix this, but at the moment I need
to defer playing with this for a few weeks.
Thanks,
Martin
>
> I strongly suspect that the problem was that my patch was emitting
> "(const_int 0)" as
> an instruction into the RTL stream, which I'd misunderstood to be recognized
> as a
> no-op by the middle-end. This isn't the case and the correct idiom is to
> (also) use:
> emit_note (NOTE_INSN_DELETED); DONE;
>
> I can easily believe that this unintended behaviour is/was interfering with
> your code
> coverage scripts (I should study your posted results).
>
> I hope this explains things. Please let me know if things really are not
> fixed (or not).
> Cheers,
> Roger
> --
>
>> -Original Message-
>> From: Martin Jambor
>> Sent: 16 June 2023 13:51
>> To: GCC Mailing List
>> Cc: Roger Sayle
>> Subject: Issue generating GCC coverage report since r14-1625-geba3565ce6d766
>>
>> Hello,
>>
>> we try to build coverage info for GCC for our testsuite and upload it to
>> https://gcc.opensuse.org/gcc-lcov/ every weekend. But since patch
>> r14-1625-geba3565ce6d766 (Add support for stc and cmc instructions in
>> i386.md) the generation broke down. However, I don't think there is
>> something
>> necessarily wrong with that particular commit, at least I don't see anything
>> suspicious.
>>
>> I inherited the generating script from Martin Liška and have not really
>> looked
>> much into it much, but it simply does the following after a fresh GCC master
>> checkout (I added the --disable-multilib and reduced the number of languages
>> to
>> reproduce this more quickly):
>>
>>
>> ../src/configure --prefix=/home/mjambor/gcc/mine/inst --enable-
>> languages=c,c++ --disable-bootstrap --enable-host-shared
>> --enable-coverage=opt
>> --disable-multilib
>> make -j64 && make -j64 -k check
>> find gcc/testsuite/ -name '*.gcda' -exec rm -rvf {} \; # I don't know why
>> the
>> script does this
>> lcov -d . --capture --output-file gcc.info
>>
>>
>> and this last step, since the commit, when processing file
>> ./gcc/insn-attrtab.gcda
>> fails with error:
>>
>> geninfo: ERROR: mismatched end line for _Z12get_attr_isaP8rtx_insn at
>> /home/mjambor/gcc/mine/src/gcc/config/i386/i386.md:5776: 5776 -> 8963
>> (use "geninfo --ignore-errors mismatch ..." to bypass this error)
>>
>> I tried looking briefly into the perl sources of lcov and geninfo but I am
>> afraid I
>> don't have the necessary knowledge of the language and the tool or the time
>> to
>> properly debug this. So I am inclined to simply add --ignore-errors
>> mismatch to
>> lcov options, which avoids the issue, and be done with it. Nevertheless, I
>> thought
>> I'd mention this here in case anyone here has any ideas what can be going
>> wrong.
>>
>> Thanks,
>>
>> Martin
gcc-14-20230625 is now available
Snapshot gcc-14-20230625 is now available on https://gcc.gnu.org/pub/gcc/snapshots/14-20230625/ and on various mirrors, see http://gcc.gnu.org/mirrors.html for details. This snapshot has been generated from the GCC 14 git branch with the following options: git://gcc.gnu.org/git/gcc.git branch master revision 9326a49c9e9d6316bc960505a55da2dd06bc10af You'll find: gcc-14-20230625.tar.xz Complete GCC SHA256=e2637b73cb805f7f2a34a87ab1d3115515752e3a3fb9bf5114e016ae84bbde60 SHA1=7fd5697993793432cfa432a908ffd7e56f27afe5 Diffs from 14-20230618 are available in the diffs/ subdirectory. When a particular snapshot is ready for public consumption the LATEST-14 link is updated and a message is sent to the gcc list. Please do not use a snapshot before it has been announced that way.
