llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT--> @llvm/pr-subscribers-flang-parser Author: Thirumalai Shaktivel (Thirumalai-Shaktivel) <details> <summary>Changes</summary> Implement the GCC/Clang dependency-file flags for flang, which it previously rejected as unknown arguments: -M, -MM, -MD, -MMD, -MF, -MT and -MQ. Feature: - -MD/-MMD : write a .d file alongside a normal compile - -M/-MM : prescan only; emit deps to stdout by default - -MF : dependency-file path - -MT/-MQ : target name (verbatim / Make-quoted) (-M==-MM and -MD==-MMD; Fortran has no system/user header split.) Design: - `-M`/`-MM`: driver passes `-Eonly -dependency-file - -MT <target>` to `-fc1`. `-Eonly` selects `RunPreprocessorOnly`, which runs only the prescanner and produces no compiled output. The dependency file is the sole result, written to stdout by default. - `-MD`/`-MMD`: driver passes `-dependency-file foo.d -MT <target>` alongside the normal compile action. The object file is produced as usual, and the dependency file is written as a side effect. - `-MF <path>` overrides where the dependency file is written. `-MT <target>` sets the target name (the part before the colon in the Make rule) verbatim. `-MQ` does the same but additionally quotes characters that are special to Make. - In `-fc1`, `-dependency-file` and `-MT` are stored in the frontend options. After the prescan, `writeDependencyFile()` is called unconditionally regardless of the action. It collects every file opened during prescan — the source plus all `INCLUDE`/`#include` files — via `GetIncludedFilePaths()`, then writes the `target: source includes...` Make rule to the specified file, or stdout if the path is `-`. --- Patch is 41.63 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/209379.diff 18 Files Affected: - (modified) clang/include/clang/Basic/MakeSupport.h (+20) - (modified) clang/include/clang/Frontend/DependencyOutputOptions.h (+1-3) - (modified) clang/include/clang/Options/Options.td (+13-5) - (modified) clang/lib/Basic/MakeSupport.cpp (+144) - (modified) clang/lib/Driver/ToolChains/Flang.cpp (+104-3) - (modified) clang/lib/Frontend/DependencyFile.cpp (+3-134) - (modified) flang/docs/FlangDriver.md (+15) - (modified) flang/include/flang/Frontend/FrontendAction.h (+2) - (modified) flang/include/flang/Frontend/FrontendActions.h (+6) - (modified) flang/include/flang/Frontend/FrontendOptions.h (+9) - (modified) flang/include/flang/Parser/provenance.h (+5) - (modified) flang/lib/Frontend/CompilerInvocation.cpp (+6) - (modified) flang/lib/Frontend/FrontendAction.cpp (+42) - (modified) flang/lib/FrontendTool/ExecuteCompilerInvocation.cpp (+2) - (modified) flang/lib/Parser/provenance.cpp (+15) - (added) flang/test/Driver/dependency-file-gen-mod.f90 (+33) - (added) flang/test/Driver/dependency-file-gen.f90 (+44) - (added) flang/test/Driver/dependency-file.f90 (+106) ``````````diff diff --git a/clang/include/clang/Basic/MakeSupport.h b/clang/include/clang/Basic/MakeSupport.h index c663014ba7bcf..5cd5dd92c5476 100644 --- a/clang/include/clang/Basic/MakeSupport.h +++ b/clang/include/clang/Basic/MakeSupport.h @@ -10,6 +10,7 @@ #define LLVM_CLANG_BASIC_MAKESUPPORT_H #include "clang/Basic/LLVM.h" +#include "llvm/ADT/ArrayRef.h" #include "llvm/ADT/StringRef.h" namespace clang { @@ -18,6 +19,25 @@ namespace clang { /// Only the characters '$', '#', ' ', '\t' are quoted. void quoteMakeTarget(StringRef Target, SmallVectorImpl<char> &Res); +/// DependencyOutputFormat - Format for the compiler dependency file. +enum class DependencyOutputFormat { Make, NMake }; + +/// Write Make-style dependency output to the output stream, in the form: +/// target1 target2 ...: prereq1 prereq2 ... +/// +/// \param Targets The targets, already quoted for Make via quoteMakeTarget(). +/// \param Files The prerequisites; each is escaped for \p Format when written. +/// \param Format Escape prerequisites for GNU Make or NMake. +/// \param PhonyTargets If true, also emit an empty "prereq:" line for each +/// prerequisite (except \p InputFileIndex), so later deleting a prerequisite +/// doesn't break the build. +/// \param InputFileIndex Index in \p Files of the main input, skipped above. +void printMakeDependencyFile( + llvm::raw_ostream &OS, llvm::ArrayRef<std::string> Targets, + llvm::ArrayRef<std::string> Files, + DependencyOutputFormat Format = DependencyOutputFormat::Make, + bool PhonyTargets = false, unsigned InputFileIndex = 0); + } // namespace clang #endif // LLVM_CLANG_BASIC_MAKESUPPORT_H diff --git a/clang/include/clang/Frontend/DependencyOutputOptions.h b/clang/include/clang/Frontend/DependencyOutputOptions.h index 98728f4d53b43..c2607b833a9e4 100644 --- a/clang/include/clang/Frontend/DependencyOutputOptions.h +++ b/clang/include/clang/Frontend/DependencyOutputOptions.h @@ -10,6 +10,7 @@ #define LLVM_CLANG_FRONTEND_DEPENDENCYOUTPUTOPTIONS_H #include "clang/Basic/HeaderInclude.h" +#include "clang/Basic/MakeSupport.h" #include <string> #include <vector> @@ -18,9 +19,6 @@ namespace clang { /// ShowIncludesDestination - Destination for /showIncludes output. enum class ShowIncludesDestination { None, Stdout, Stderr }; -/// DependencyOutputFormat - Format for the compiler dependency file. -enum class DependencyOutputFormat { Make, NMake }; - /// ExtraDepKind - The kind of extra dependency file. enum ExtraDepKind { EDK_SanitizeIgnorelist, diff --git a/clang/include/clang/Options/Options.td b/clang/include/clang/Options/Options.td index 4974209b8db30..767c9044e44e3 100644 --- a/clang/include/clang/Options/Options.td +++ b/clang/include/clang/Options/Options.td @@ -807,14 +807,19 @@ def embed_dir_EQ : Joined<["--"], "embed-dir=">, Group<Preprocessor_Group>, Visibility<[ClangOption, CC1Option]>, MetaVarName<"<dir>">, HelpText<"Add directory to embed search path">; def MD : Flag<["-"], "MD">, Group<M_Group>, + Visibility<[ClangOption, FlangOption]>, HelpText<"Write a depfile containing user and system headers">; def MMD : Flag<["-"], "MMD">, Group<M_Group>, + Visibility<[ClangOption, FlangOption]>, HelpText<"Write a depfile containing user headers">; def M : Flag<["-"], "M">, Group<M_Group>, + Visibility<[ClangOption, FlangOption]>, HelpText<"Like -MD, but also implies -E and writes to stdout by default">; def MM : Flag<["-"], "MM">, Group<M_Group>, + Visibility<[ClangOption, FlangOption]>, HelpText<"Like -MMD, but also implies -E and writes to stdout by default">; def MF : JoinedOrSeparate<["-"], "MF">, Group<M_Group>, + Visibility<[ClangOption, FlangOption]>, HelpText<"Write depfile output from -MMD, -MD, -MM, or -M to <file>">, MetaVarName<"<file>">; def MG : Flag<["-"], "MG">, Group<M_Group>, Visibility<[ClangOption, CC1Option]>, @@ -826,10 +831,10 @@ def MP : Flag<["-"], "MP">, Group<M_Group>, Visibility<[ClangOption, CC1Option]> HelpText<"Create phony target for each dependency (other than main file)">, MarshallingInfoFlag<DependencyOutputOpts<"UsePhonyTargets">>; def MQ : JoinedOrSeparate<["-"], "MQ">, Group<M_Group>, - Visibility<[ClangOption, CC1Option]>, + Visibility<[ClangOption, CC1Option, FlangOption]>, HelpText<"Specify name of main file output to quote in depfile">; def MT : JoinedOrSeparate<["-"], "MT">, Group<M_Group>, - Visibility<[ClangOption, CC1Option]>, + Visibility<[ClangOption, CC1Option, FC1Option, FlangOption]>, HelpText<"Specify name of main file output in depfile">, MarshallingInfoStringVector<DependencyOutputOpts<"Targets">>; def MV : Flag<["-"], "MV">, Group<M_Group>, Visibility<[ClangOption, CC1Option]>, @@ -1528,7 +1533,7 @@ def dM : Flag<["-"], "dM">, Group<d_Group>, Visibility<[ClangOption, CC1Option, HelpText<"Print macro definitions in -E mode instead of normal output">; def dead__strip : Flag<["-"], "dead_strip">; def dependency_file : Separate<["-"], "dependency-file">, - Visibility<[ClangOption, CC1Option]>, + Visibility<[ClangOption, CC1Option, FC1Option]>, HelpText<"Filename (or -) to write dependency output to">, MarshallingInfoString<DependencyOutputOpts<"OutputFile">>; def dependency_dot : Separate<["-"], "dependency-dot">, @@ -8529,8 +8534,6 @@ defm recovery_ast_type : BoolOption<"f", "recovery-ast-type", let Group = Action_Group in { -def Eonly : Flag<["-"], "Eonly">, - HelpText<"Just run preprocessor, no output (for timings)">; def dump_raw_tokens : Flag<["-"], "dump-raw-tokens">, HelpText<"Lex file in raw mode and dump raw tokens">; def analyze : Flag<["-"], "analyze">, @@ -8673,6 +8676,11 @@ def finitial_counter_value_EQ : Joined<["-"], "finitial-counter-value=">, } // let Visibility = [CC1Option] +def Eonly : Flag<["-"], "Eonly">, Group<Action_Group>, + Visibility<[CC1Option, FC1Option]>, + HelpText<"Just run preprocessor, no output (for timings or dependency " + "generation)">; + //===----------------------------------------------------------------------===// // Language Options //===----------------------------------------------------------------------===// diff --git a/clang/lib/Basic/MakeSupport.cpp b/clang/lib/Basic/MakeSupport.cpp index 4ddfcc350410c..afc22493aff0d 100644 --- a/clang/lib/Basic/MakeSupport.cpp +++ b/clang/lib/Basic/MakeSupport.cpp @@ -7,6 +7,9 @@ //===----------------------------------------------------------------------===// #include "clang/Basic/MakeSupport.h" +#include "llvm/ADT/SmallString.h" +#include "llvm/Support/Path.h" +#include "llvm/Support/raw_ostream.h" void clang::quoteMakeTarget(StringRef Target, SmallVectorImpl<char> &Res) { for (unsigned i = 0, e = Target.size(); i != e; ++i) { @@ -33,3 +36,144 @@ void clang::quoteMakeTarget(StringRef Target, SmallVectorImpl<char> &Res) { Res.push_back(Target[i]); } } + +/// Print the filename, with escaping or quoting that accommodates the three +/// most likely tools that use dependency files: GNU Make, BSD Make, and +/// NMake/Jom. +/// +/// BSD Make is the simplest case: It does no escaping at all. This means +/// characters that are normally delimiters, i.e. space and # (the comment +/// character) simply aren't supported in filenames. +/// +/// GNU Make does allow space and # in filenames, but to avoid being treated +/// as a delimiter or comment, these must be escaped with a backslash. Because +/// backslash is itself the escape character, if a backslash appears in a +/// filename, it should be escaped as well. (As a special case, $ is escaped +/// as $$, which is the normal Make way to handle the $ character.) +/// For compatibility with BSD Make and historical practice, if GNU Make +/// un-escapes characters in a filename but doesn't find a match, it will +/// retry with the unmodified original string. +/// +/// GCC tries to accommodate both Make formats by escaping any space or # +/// characters in the original filename, but not escaping backslashes. The +/// apparent intent is so that filenames with backslashes will be handled +/// correctly by BSD Make, and by GNU Make in its fallback mode of using the +/// unmodified original string; filenames with # or space characters aren't +/// supported by BSD Make at all, but will be handled correctly by GNU Make +/// due to the escaping. +/// +/// A corner case that GCC gets only partly right is when the original filename +/// has a backslash immediately followed by space or #. GNU Make would expect +/// this backslash to be escaped; however GCC escapes the original backslash +/// only when followed by space, not #. It will therefore take a dependency +/// from a directive such as +/// #include "a\ b\#c.h" +/// and emit it as +/// a\\\ b\\#c.h +/// which GNU Make will interpret as +/// a\ b\ +/// followed by a comment. Failing to find this file, it will fall back to the +/// original string, which probably doesn't exist either; in any case it won't +/// find +/// a\ b\#c.h +/// which is the actual filename specified by the include directive. +/// +/// Clang does what GCC does, rather than what GNU Make expects. +/// +/// NMake/Jom has a different set of scary characters, but wraps filespecs in +/// double-quotes to avoid misinterpreting them; see +/// https://msdn.microsoft.com/en-us/library/dd9y37ha.aspx for NMake info, +/// https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx +/// for Windows file-naming info. +static void printFilename(llvm::raw_ostream &OS, llvm::StringRef Filename, + clang::DependencyOutputFormat OutputFormat) { + // Convert filename to platform native path + llvm::SmallString<256> NativePath; + llvm::sys::path::native(Filename, NativePath); + + if (OutputFormat == clang::DependencyOutputFormat::NMake) { + // Add quotes if needed. These are the characters listed as "special" to + // NMake, that are legal in a Windows filespec, and that could cause + // misinterpretation of the dependency string. + if (NativePath.find_first_of(" #${}^!") != llvm::StringRef::npos) + OS << '\"' << NativePath << '\"'; + else + OS << NativePath; + return; + } + assert(OutputFormat == clang::DependencyOutputFormat::Make); + for (unsigned i = 0, e = NativePath.size(); i != e; ++i) { + if (NativePath[i] == '#') // Handle '#' the broken gcc way. + OS << '\\'; + else if (NativePath[i] == ' ') { // Handle space correctly. + OS << '\\'; + unsigned j = i; + while (j > 0 && NativePath[--j] == '\\') + OS << '\\'; + } else if (NativePath[i] == '$') // $ is escaped by $$. + OS << '$'; + OS << NativePath[i]; + } +} + +void clang::printMakeDependencyFile(llvm::raw_ostream &OS, + llvm::ArrayRef<std::string> Targets, + llvm::ArrayRef<std::string> Files, + DependencyOutputFormat Format, + bool PhonyTargets, + unsigned InputFileIndex) { + // Write out the dependency targets, trying to avoid overly long + // lines when possible. We try our best to emit exactly the same + // dependency file as GCC>=10, assuming the included files are the + // same. + const unsigned MaxColumns = 75; + unsigned Columns = 0; + + for (StringRef Target : Targets) { + unsigned N = Target.size(); + if (Columns == 0) { + Columns += N; + } else if (Columns + N + 2 > MaxColumns) { + Columns = N + 2; + OS << " \\\n "; + } else { + Columns += N + 1; + OS << ' '; + } + // Targets already quoted as needed. + OS << Target; + } + + OS << ':'; + Columns += 1; + + // Now add each dependency in the order it was seen, but avoiding + // duplicates. + for (StringRef File : Files) { + if (File == "<stdin>") + continue; + // Start a new line if this would exceed the column limit. Make + // sure to leave space for a trailing " \" in case we need to + // break the line on the next iteration. + unsigned N = File.size(); + if (Columns + (N + 1) + 2 > MaxColumns) { + OS << " \\\n "; + Columns = 2; + } + OS << ' '; + printFilename(OS, File, Format); + Columns += N + 1; + } + OS << '\n'; + + // Create phony targets if requested. + if (PhonyTargets && !Files.empty()) { + unsigned Index = 0; + for (auto I = Files.begin(), E = Files.end(); I != E; ++I) { + if (Index++ == InputFileIndex) + continue; + printFilename(OS, *I, Format); + OS << ":\n"; + } + } +} diff --git a/clang/lib/Driver/ToolChains/Flang.cpp b/clang/lib/Driver/ToolChains/Flang.cpp index d900037230f20..a69913fc4872e 100644 --- a/clang/lib/Driver/ToolChains/Flang.cpp +++ b/clang/lib/Driver/ToolChains/Flang.cpp @@ -11,6 +11,7 @@ #include "Cuda.h" #include "clang/Basic/CodeGenOptions.h" +#include "clang/Basic/MakeSupport.h" #include "clang/Driver/CommonArgs.h" #include "clang/Options/OptionUtils.h" #include "clang/Options/Options.h" @@ -35,6 +36,100 @@ static void addDashXForInput(const ArgList &Args, const InputInfo &Input, CmdArgs.push_back(types::getTypeName(Input.getType())); } +// Translate the dependency-file options into the arguments understood by +// `flang -fc1`. The options handled here: +// -M Emit only the dependencies and skip code generation. They are +// written to stdout unless -MF redirects them. +// -MM Treated identically to -M. The -MM/-M split exists to omit system +// headers, but Fortran has no notion of system vs user headers, so +// there is nothing for -MM to exclude. +// -MD Compile normally and produce the object file, while also writing the +// dependency file. Its name defaults to the -o value, or the input +// file name when -o is absent, with the extension replaced by .d. +// -MMD Treated identically to -MD, for the same reason -MM equals -M. +// -MF Set the path of the dependency file to write. +// -MT Set the dependency target name (the part before the colon). +// -MQ Like -MT, but additionally quotes characters special to Make. +static void renderDependencyGenerationOptions(Compilation &C, + const JobAction &JA, + const ArgList &Args, + const InputInfo &Output, + const InputInfoList &Inputs, + ArgStringList &CmdArgs) { + Arg *ArgM = Args.getLastArg(options::OPT_M, options::OPT_MM); + Arg *ArgMD = Args.getLastArg(options::OPT_MD, options::OPT_MMD); + + // Drop warnings for -M/-MM so they don't mix into the dependency output. + if (ArgM) { + CmdArgs.push_back("-w"); + } else { + ArgM = ArgMD; + } + + if (!ArgM) { + return; + } + + // Emit "-MT <target>", quoting Make metacharacters when requested. + auto addTarget = [&](StringRef Target, bool Quote) { + SmallString<128> Quoted; + if (Quote) { + clang::quoteMakeTarget(Target, Quoted); + Target = Quoted; + } + CmdArgs.push_back("-MT"); + CmdArgs.push_back(Args.MakeArgString(Target)); + }; + + // Decide where to write the dependency file. + const char *DepFile; + if (Arg *MF = Args.getLastArg(options::OPT_MF)) { + // -MF gives the path explicitly. + DepFile = MF->getValue(); + C.addFailureResultFile(DepFile, &JA); + } else if (Output.getType() == types::TY_Dependencies) { + // Plain -M/-MM: the dependency file is the output, so use its name + DepFile = Output.getFilename(); + } else if (!ArgMD) { + // -M/-MM with no -o: write the dependencies to stdout. + DepFile = "-"; + } else { + // -MD/-MMD: name it after -o, else the input, with a .d extension. + SmallString<128> P; + if (Arg *OutputOpt = Args.getLastArg(options::OPT_o)) { + P = OutputOpt->getValue(); + } else { + P = llvm::sys::path::filename(Inputs[0].getBaseInput()); + } + llvm::sys::path::replace_extension(P, "d"); + DepFile = Args.MakeArgString(P); + C.addFailureResultFile(DepFile, &JA); + } + CmdArgs.push_back("-dependency-file"); + CmdArgs.push_back(DepFile); + + // Render the explicit target(s). -MT is verbatim, -MQ is Make-quoted. + bool HasTarget = false; + for (const Arg *A : Args.filtered(options::OPT_MT, options::OPT_MQ)) { + HasTarget = true; + A->claim(); + addTarget(A->getValue(), A->getOption().matches(options::OPT_MQ)); + } + + // With no explicit target, default to the object file. In -M/-MM mode -o + // names the dependency file, not the target, so derive <base>.o instead. + if (!HasTarget) { + if (Arg *OutputOpt = Args.getLastArg(options::OPT_o); + OutputOpt && Output.getType() != types::TY_Dependencies) { + addTarget(OutputOpt->getValue(), /*Quote=*/true); + } else { + SmallString<128> P(llvm::sys::path::filename(Inputs[0].getBaseInput())); + llvm::sys::path::replace_extension(P, "o"); + addTarget(P, /*Quote=*/true); + } + } +} + void Flang::addFortranDialectOptions(const ArgList &Args, ArgStringList &CmdArgs) const { Args.addAllArgs(CmdArgs, {options::OPT_ffixed_form, @@ -1052,9 +1147,13 @@ void Flang::ConstructJob(Compilation &C, const JobAction &JA, CmdArgs.push_back(Args.MakeArgString(TripleStr)); if (isa<PreprocessJobAction>(JA)) { - CmdArgs.push_back("-E"); - if (Args.getLastArg(options::OPT_dM)) { - CmdArgs.push_back("-dM"); + if (Output.getType() == types::TY_Dependencies) { + CmdArgs.push_back("-Eonly"); + } else { + CmdArgs.push_back("-E"); + if (Args.getLastArg(options::OPT_dM)) { + CmdArgs.push_back("-dM"); + } } } else if (isa<CompileJobAction>(JA) || isa<BackendJobAction>(JA)) { if (JA.getType() == types::TY_Nothing) { @@ -1310,6 +1409,8 @@ void Flang::ConstructJob(Compilation &C, const JobAction &JA, if (Args.getLastArg(options::OPT_save_temps_EQ)) Args.AddLastArg(CmdArgs, options::OPT_save_temps_EQ); + renderDependencyGenerationOptions(C, JA, Args, Output, Inputs, CmdArgs); + addDashXForInput(Args, Input, CmdArgs); bool FRecordCmdLine = false; diff --git a/clang/lib/Frontend/DependencyFile.cpp b/clang/lib/Frontend/DependencyFile.cpp index 00f4a54269cfa..5702d1dabcc8b 100644 --- a/clang/lib/Frontend/DependencyFile.cpp +++ b/clang/lib/Frontend/DependencyFile.cpp @@ -12,6 +12,7 @@ #include "clang/Basic/DiagnosticFrontend.h" #include "clang/Basic/FileManager.h" +#include "clang/Basic/MakeSupport.h" #include "clang/Basic/SourceManager.h" #include "clang/Frontend/DependencyOutputOptions.h" #include "clang/Frontend/Utils.h" @@ -293,85 +294,6 @@ void DependencyFileGenerator::finishedMainFile(DiagnosticsEngine &Diags) { outputDependencyFile(Diags); } -/// Print the filename, with escaping or quoting that accommodates the three -/// most likely tools that use dependency files: GNU Make, BSD Make, and -/// NMake/Jom. -/// -/// BSD Make is the simplest case: It does no escaping at all. This means -/// characters that are normally delimiters, i.e. space and # (the comment -/// character) simply aren't supported in filenames. -/// -/// GNU Make does allow space and # in filenames, but to avoid being treated -/// as a delimiter or comment, these must be escaped with a backslash. Because -/// backslash is itself the escape character, if a backslash appears in a -/// filename, it should be escaped as well. (As a special case, $ is escaped -/// as $$, which is the normal Make way to handle the $ character.) -/// For compatibility with BSD Make and historical practice, if GNU Make -/// un-escapes characters in a filename but doesn't find a match, it will -/// retry with the unmodified original string. -/// -/// GCC tries to accommodate both Make formats by escaping any space or # -/// characters in the original filename, but not escaping backslashes. The -/// apparent intent is so that filenames with backslashes will be handled -/// correctly by BSD Make, and by GNU Make in its fallback mode of using the -/// unmodified original string; filenames with # or space characters aren't -/// supported by BSD Make at all, but will be handled correctly by GNU Make -/// due to the escaping. -/// -/// A corner case that GC... [truncated] `````````` </details> https://github.com/llvm/llvm-project/pull/209379 _______________________________________________ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
