llvmbot wrote:
<!--LLVM PR SUMMARY COMMENT--> @llvm/pr-subscribers-clang Author: Josh Dowell (Slartibarty) <details> <summary>Changes</summary> This adds a new compiler flag called `-fms-preprocessor-compat` which may be used to control whether the preprocessor enables full Microsoft compatibility mode, which was previously rolled into `-fms-compatibility`. This means that `/Zc:preprocessor` for clang-cl can be implemented as an alias for `-fno-ms-preprocessor-compat`. It also sets the predefined macro `_MSVC_TRADITIONAL` under appropriate circumstances. This can be merged as-is, but I would greatly appreciate feedback on this from those more experienced with working with the Clang frontend, specifically regarding the following points: Under MSVC, `/Zc:preprocessor` is implied when using `/std:c11` or newer, however I did not implement this due to there not being a straightforward or future proof way to get the current C standard in use when determining arguments for the Clang driver, there is already code in place which checks for C++20 or newer in the same function, for something else, but it is hacky and checks for every possible combination of C++20 arguments, of which there are many more for C11 and newer. Is this an okay divergence? There is currently not a way to opt-out of the two preprocessor extensions enabled by `-fms-extensions` unless you have also specified `-fms-compatibility` as well as `-fno-ms-preprocessor-compat`. I implemented it this way so that using clang-cl like MSVC will indeed match MSVC exactly, where the charize operator and /##/ trick are disabled. I could percieve a situtation where it would be desirable to continue using these extensions under clang-cl, but I preferred matching MSVC. Is this okay? Fixes #<!-- -->147304 --- Full diff: https://github.com/llvm/llvm-project/pull/167200.diff 14 Files Affected: - (modified) clang/docs/ReleaseNotes.rst (+7) - (modified) clang/docs/UsersManual.rst (+2) - (modified) clang/include/clang/Basic/LangOptions.def (+1) - (modified) clang/include/clang/Basic/LangOptions.h (+20) - (modified) clang/include/clang/Driver/Options.td (+12) - (modified) clang/lib/Driver/ToolChains/Clang.cpp (+4) - (modified) clang/lib/Frontend/InitPreprocessor.cpp (+11) - (modified) clang/lib/Lex/Lexer.cpp (+3-3) - (modified) clang/lib/Lex/TokenLexer.cpp (+7-7) - (modified) clang/test/Driver/cl-options.c (+6) - (modified) clang/test/Lexer/ms-compatibility.c (+1-1) - (modified) clang/test/Preprocessor/macro_fn_comma_swallow2.c (+1-1) - (modified) clang/test/Preprocessor/microsoft-ext.c (+1-1) - (added) clang/test/Preprocessor/microsoft-preprocessor-compat.c (+36) ``````````diff diff --git a/clang/docs/ReleaseNotes.rst b/clang/docs/ReleaseNotes.rst index 3a4e1fce2511e..a0db2a8794e14 100644 --- a/clang/docs/ReleaseNotes.rst +++ b/clang/docs/ReleaseNotes.rst @@ -306,12 +306,17 @@ Non-comprehensive list of changes in this release - Clang now rejects the invalid use of ``constexpr`` with ``auto`` and an explicit type in C. (#GH163090) +- ``_MSVC_TRADITIONAL`` is now defined to ``0`` when using ``-fms-extensions``. The charize operator and ``/##/`` paste trick remain as supported extensions, which is a divergence from MSVC. + New Compiler Flags ------------------ - New option ``-fno-sanitize-debug-trap-reasons`` added to disable emitting trap reasons into the debug info when compiling with trapping UBSan (e.g. ``-fsanitize-trap=undefined``). - New option ``-fsanitize-debug-trap-reasons=`` added to control emitting trap reasons into the debug info when compiling with trapping UBSan (e.g. ``-fsanitize-trap=undefined``). - New options for enabling allocation token instrumentation: ``-fsanitize=alloc-token``, ``-falloc-token-max=``, ``-fsanitize-alloc-token-fast-abi``, ``-fsanitize-alloc-token-extended``. - The ``-resource-dir`` option is now displayed in the list of options shown by ``--help``. +- The ``-fms-preprocessor-compat`` flag has been added to control whether full emulation of the MSVC traditional preprocessor is performed. + Using ``-fms-preprocessor-compat`` will define ``_MSVC_TRADITIONAL`` to ``1``, and enables the full set of functionality required to emulate + the old MSVC character buffer based preprocessor as closely as possible. It is enabled by default when using ``-fms-compatibility``, which is the case for clang-cl. Lanai Support ^^^^^^^^^^^^^^ @@ -581,6 +586,8 @@ Android Support Windows Support ^^^^^^^^^^^^^^^ +- clang-cl now supports ``/Zc:preprocessor``, which is an alias for ``-fno-ms-preprocessor-compat``. + LoongArch Support ^^^^^^^^^^^^^^^^^ - Enable linker relaxation by default for loongarch64. diff --git a/clang/docs/UsersManual.rst b/clang/docs/UsersManual.rst index fb22ad3c90af4..2d5208ed317e9 100644 --- a/clang/docs/UsersManual.rst +++ b/clang/docs/UsersManual.rst @@ -5055,6 +5055,8 @@ Execute ``clang-cl /?`` to see a list of supported options: /Zc:trigraphs Enable trigraphs /Zc:twoPhase- Disable two-phase name lookup in templates /Zc:twoPhase Enable two-phase name lookup in templates + /Zc:preprocessor- Use the traditional (non-conforming) preprocessor (default) + /Zc:preprocessor Use the standard conforming preprocessor /Zi Alias for /Z7. Does not produce PDBs. /Zl Don't mention any default libraries in the object file /Zp Set the default maximum struct packing alignment to 1 diff --git a/clang/include/clang/Basic/LangOptions.def b/clang/include/clang/Basic/LangOptions.def index 40fc66ea12e34..33dc642f7a280 100644 --- a/clang/include/clang/Basic/LangOptions.def +++ b/clang/include/clang/Basic/LangOptions.def @@ -269,6 +269,7 @@ LANGOPT(HIPStdParInterposeAlloc, 1, 0, NotCompatible, "Replace allocations / dea LANGOPT(OpenACC , 1, 0, NotCompatible, "OpenACC Enabled") +LANGOPT(MSVCPreprocessor , 1, 0, NotCompatible, "Fully emulate the Microsoft Visual C++ traditional (non-conforming) preprocessor") LANGOPT(MSVCEnableStdcMacro , 1, 0, NotCompatible, "Define __STDC__ with '-fms-compatibility'") LANGOPT(SizedDeallocation , 1, 0, NotCompatible, "sized deallocation") LANGOPT(AlignedAllocation , 1, 0, NotCompatible, "aligned allocation") diff --git a/clang/include/clang/Basic/LangOptions.h b/clang/include/clang/Basic/LangOptions.h index 8aa89d8c8c807..02d958878088f 100644 --- a/clang/include/clang/Basic/LangOptions.h +++ b/clang/include/clang/Basic/LangOptions.h @@ -627,6 +627,26 @@ class LangOptions : public LangOptionsBase { return MSCompatibilityVersion >= MajorVersion * 100000U; } + /// Returns true if we want to enable lightweight Microsoft preprocessor + /// extensions. + /// Historically, MicrosoftExt and MSVCCompat enabled different extensions + /// to the preprocessor, but we want to disable a majority of them when + /// MSVCPreprocessor is set to false. + bool wantsMSVCPreprocessorExtensions() const { + // If we want full compatibility then we need the extensions + if (MSVCPreprocessor) + return true; + // If MS extensions are turned off, we don't want the extensions + if (!MicrosoftExt) + return false; + // If we're in full MSVC compat mode, we know MSVCPreprocessor is false, + // thus, we don't want the extensions + if (MSVCCompat) + return false; + // We just want lightweight extensions + return true; + } + bool isOverflowPatternExcluded(OverflowPatternExclusionKind Kind) const { if (OverflowPatternExclusionMask & OverflowPatternExclusionKind::None) return false; diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td index 11e81e032d5fc..795b1d4cbf9f1 100644 --- a/clang/include/clang/Driver/Options.td +++ b/clang/include/clang/Driver/Options.td @@ -3238,6 +3238,10 @@ def fms_compatibility : Flag<["-"], "fms-compatibility">, Group<f_Group>, Visibility<[ClangOption, CC1Option, CLOption]>, HelpText<"Enable full Microsoft Visual C++ compatibility">, MarshallingInfoFlag<LangOpts<"MSVCCompat">>; +def fms_preprocessor_compat : Flag<["-"], "fms-preprocessor-compat">, Group<f_Group>, + Visibility<[ClangOption, CC1Option]>, + HelpText<"Enable full emulation of the traditional MSVC preprocesor">, + MarshallingInfoFlag<LangOpts<"MSVCPreprocessor">>; def fms_define_stdc : Flag<["-"], "fms-define-stdc">, Group<f_Group>, Visibility<[ClangOption, CC1Option, CLOption]>, HelpText<"Define '__STDC__' to '1' in MSVC Compatibility mode">, @@ -3614,6 +3618,8 @@ def fno_ms_extensions : Flag<["-"], "fno-ms-extensions">, Group<f_Group>, Visibility<[ClangOption, CLOption]>; def fno_ms_compatibility : Flag<["-"], "fno-ms-compatibility">, Group<f_Group>, Visibility<[ClangOption, CLOption]>; +def fno_ms_preprocessor_compat : Flag<["-"], "fno-ms-preprocessor-compat">, Group<f_Group>, + Visibility<[ClangOption]>; def fno_objc_legacy_dispatch : Flag<["-"], "fno-objc-legacy-dispatch">, Group<f_Group>; def fno_objc_weak : Flag<["-"], "fno-objc-weak">, Group<f_Group>, Visibility<[ClangOption, CC1Option]>; @@ -9123,6 +9129,12 @@ def _SLASH_vd : CLJoined<"vd">, HelpText<"Control vtordisp placement">, Alias<vtordisp_mode_EQ>; def _SLASH_X : CLFlag<"X">, HelpText<"Do not add %INCLUDE% to include search path">, Alias<nostdlibinc>; +def _SLASH_Zc_preprocessor : CLFlag<"Zc:preprocessor">, + HelpText<"Use the standard conforming preprocessor">, + Alias<fno_ms_preprocessor_compat>; +def _SLASH_Zc_preprocessor_ : CLFlag<"Zc:preprocessor-">, + HelpText<"Use the traditional (non-conforming) preprocessor (default)">, + Alias<fms_preprocessor_compat>; def _SLASH_Zc___STDC__ : CLFlag<"Zc:__STDC__">, HelpText<"Define __STDC__">, Alias<fms_define_stdc>; diff --git a/clang/lib/Driver/ToolChains/Clang.cpp b/clang/lib/Driver/ToolChains/Clang.cpp index 30d3e5293a31b..dd32c0bd92b7e 100644 --- a/clang/lib/Driver/ToolChains/Clang.cpp +++ b/clang/lib/Driver/ToolChains/Clang.cpp @@ -7056,6 +7056,10 @@ void Clang::ConstructJob(Compilation &C, const JobAction &JA, if (!types::isCXX(Input.getType()) && Args.hasArg(options::OPT_fms_define_stdc)) CmdArgs.push_back("-fms-define-stdc"); + // FIXME: This should be disabled by default in C11 and newer. + if (Args.hasFlag(options::OPT_fms_preprocessor_compat, + options::OPT_fno_ms_preprocessor_compat, true)) + CmdArgs.push_back("-fms-preprocessor-compat"); } if (Triple.isWindowsMSVCEnvironment() && !D.IsCLMode() && diff --git a/clang/lib/Frontend/InitPreprocessor.cpp b/clang/lib/Frontend/InitPreprocessor.cpp index b88d9f89c5f71..d9571c57cfe86 100644 --- a/clang/lib/Frontend/InitPreprocessor.cpp +++ b/clang/lib/Frontend/InitPreprocessor.cpp @@ -1022,6 +1022,17 @@ static void InitializePredefinedMacros(const TargetInfo &TI, } } + if (LangOpts.MSVCPreprocessor) { + Builder.defineMacro("_MSVC_TRADITIONAL", "1"); + } else if (LangOpts.MicrosoftExt) { + // Note: when set to 0 this macro implies, under cl.exe, that the + // charize operator and /##/ no longer function as they used to. + // However, we have special code that cleanly deals with these patterns, + // so we'll allow them to exist as extensions, even though we are stating + // that we do not support the "traditional" preprocessor. + Builder.defineMacro("_MSVC_TRADITIONAL", "0"); + } + // Macros to help identify the narrow and wide character sets // FIXME: clang currently ignores -fexec-charset=. If this changes, // then this may need to be updated. diff --git a/clang/lib/Lex/Lexer.cpp b/clang/lib/Lex/Lexer.cpp index b282a600c0e56..94bb5a8446e55 100644 --- a/clang/lib/Lex/Lexer.cpp +++ b/clang/lib/Lex/Lexer.cpp @@ -2181,7 +2181,7 @@ const char *Lexer::LexUDSuffix(Token &Result, const char *CurPtr, if (!IsUDSuffix) { if (!isLexingRawMode()) - Diag(CurPtr, LangOpts.MSVCCompat + Diag(CurPtr, LangOpts.MSVCPreprocessor ? diag::ext_ms_reserved_user_defined_literal : diag::ext_reserved_user_defined_literal) << FixItHint::CreateInsertion(getSourceLocation(CurPtr), " "); @@ -4217,7 +4217,7 @@ bool Lexer::LexTokenInternal(Token &Result, bool TokAtPhysicalStartOfLine) { Kind = tok::hashhash; // '%:%:' -> '##' CurPtr = ConsumeChar(ConsumeChar(CurPtr, SizeTmp, Result), SizeTmp2, Result); - } else if (Char == '@' && LangOpts.MicrosoftExt) {// %:@ -> #@ -> Charize + } else if (Char == '@' && LangOpts.wantsMSVCPreprocessorExtensions()) {// %:@ -> #@ -> Charize CurPtr = ConsumeChar(CurPtr, SizeTmp, Result); if (!isLexingRawMode()) Diag(BufferPtr, diag::ext_charize_microsoft); @@ -4405,7 +4405,7 @@ bool Lexer::LexTokenInternal(Token &Result, bool TokAtPhysicalStartOfLine) { if (Char == '#') { Kind = tok::hashhash; CurPtr = ConsumeChar(CurPtr, SizeTmp, Result); - } else if (Char == '@' && LangOpts.MicrosoftExt) { // #@ -> Charize + } else if (Char == '@' && LangOpts.wantsMSVCPreprocessorExtensions()) { // #@ -> Charize Kind = tok::hashat; if (!isLexingRawMode()) Diag(BufferPtr, diag::ext_charize_microsoft); diff --git a/clang/lib/Lex/TokenLexer.cpp b/clang/lib/Lex/TokenLexer.cpp index 47f4134fb1465..cbe6ba1d56e1f 100644 --- a/clang/lib/Lex/TokenLexer.cpp +++ b/clang/lib/Lex/TokenLexer.cpp @@ -145,7 +145,7 @@ bool TokenLexer::MaybeRemoveCommaBeforeVaArgs( // In Microsoft-compatibility mode, a comma is removed in the expansion // of " ... , __VA_ARGS__ " if __VA_ARGS__ is empty. This extension is // not supported by gcc. - if (!HasPasteOperator && !PP.getLangOpts().MSVCCompat) + if (!HasPasteOperator && !PP.getLangOpts().MSVCPreprocessor) return false; // GCC removes the comma in the expansion of " ... , ## __VA_ARGS__ " if @@ -467,7 +467,7 @@ void TokenLexer::ExpandFunctionArguments() { // behavior by not considering single commas from nested macro // expansions as argument separators. Set a flag on the token so we can // test for this later when the macro expansion is processed. - if (PP.getLangOpts().MSVCCompat && NumToks == 1 && + if (PP.getLangOpts().MSVCPreprocessor && NumToks == 1 && ResultToks.back().is(tok::comma)) ResultToks.back().setFlag(Token::IgnoredComma); @@ -653,7 +653,7 @@ bool TokenLexer::Lex(Token &Tok) { // Special processing of L#x macros in -fms-compatibility mode. // Microsoft compiler is able to form a wide string literal from // 'L#macro_arg' construct in a function-like macro. - (PP.getLangOpts().MSVCCompat && + (PP.getLangOpts().MSVCPreprocessor && isWideStringLiteralFromMacro(Tok, Tokens[CurTokenIdx])))) { // When handling the microsoft /##/ extension, the final token is // returned by pasteTokens, not the pasted token. @@ -732,7 +732,7 @@ bool TokenLexer::pasteTokens(Token &LHSTok, ArrayRef<Token> TokenStream, unsigned int &CurIdx) { assert(CurIdx > 0 && "## can not be the first token within tokens"); assert((TokenStream[CurIdx].is(tok::hashhash) || - (PP.getLangOpts().MSVCCompat && + (PP.getLangOpts().MSVCPreprocessor && isWideStringLiteralFromMacro(LHSTok, TokenStream[CurIdx]))) && "Token at this Index must be ## or part of the MSVC 'L " "#macro-arg' pasting pair"); @@ -740,7 +740,7 @@ bool TokenLexer::pasteTokens(Token &LHSTok, ArrayRef<Token> TokenStream, // MSVC: If previous token was pasted, this must be a recovery from an invalid // paste operation. Ignore spaces before this token to mimic MSVC output. // Required for generating valid UUID strings in some MS headers. - if (PP.getLangOpts().MicrosoftExt && (CurIdx >= 2) && + if (PP.getLangOpts().wantsMSVCPreprocessorExtensions() && (CurIdx >= 2) && TokenStream[CurIdx - 2].is(tok::hashhash)) LHSTok.clearFlag(Token::LeadingSpace); @@ -854,8 +854,8 @@ bool TokenLexer::pasteTokens(Token &LHSTok, ArrayRef<Token> TokenStream, // Test for the Microsoft extension of /##/ turning into // here on the // error path. - if (PP.getLangOpts().MicrosoftExt && LHSTok.is(tok::slash) && - RHS.is(tok::slash)) { + if (PP.getLangOpts().wantsMSVCPreprocessorExtensions() && + LHSTok.is(tok::slash) && RHS.is(tok::slash)) { HandleMicrosoftCommentPaste(LHSTok, Loc); return true; } diff --git a/clang/test/Driver/cl-options.c b/clang/test/Driver/cl-options.c index 1b1169b71554a..fb137d48c0fe8 100644 --- a/clang/test/Driver/cl-options.c +++ b/clang/test/Driver/cl-options.c @@ -375,6 +375,12 @@ // RUN: %clang_cl -c -### /std:clatest -- %s 2>&1 | FileCheck -check-prefix CHECK-CLATEST %s // CHECK-CLATEST: -std=c23 +// RUN: %clang_cl -c -### /Zc:preprocessor- -- %s 2>&1 | FileCheck -check-prefix CHECK-ZC-PREPROCESSOR-NO %s +// CHECK-ZC-PREPROCESSOR-NO: -fms-preprocessor-compat + +// RUN: %clang_cl -c -### /Zc:preprocessor -- %s 2>&1 | FileCheck -check-prefix CHECK-ZC-PREPROCESSOR %s +// CHECK-ZC-PREPROCESSOR-NOT: -fms-preprocessor-compat + // For some warning ids, we can map from MSVC warning to Clang warning. // RUN: %clang_cl -wd4005 -wd4100 -wd4910 -wd4996 -wd12345678 -### -- %s 2>&1 | FileCheck -check-prefix=Wno %s // Wno: "-cc1" diff --git a/clang/test/Lexer/ms-compatibility.c b/clang/test/Lexer/ms-compatibility.c index 2981b8e062e93..ebaf89a7c4738 100644 --- a/clang/test/Lexer/ms-compatibility.c +++ b/clang/test/Lexer/ms-compatibility.c @@ -1,4 +1,4 @@ -// RUN: %clang_cc1 -E -fms-compatibility %s | FileCheck --check-prefix=CHECK-MS-COMPAT %s +// RUN: %clang_cc1 -E -fms-preprocessor-compat %s | FileCheck --check-prefix=CHECK-MS-COMPAT %s // RUN: %clang_cc1 -E %s | FileCheck --check-prefix=CHECK-NO-MS-COMPAT %s #define FN(x) L#x diff --git a/clang/test/Preprocessor/macro_fn_comma_swallow2.c b/clang/test/Preprocessor/macro_fn_comma_swallow2.c index 93ab2b83664a1..f594e4139bb33 100644 --- a/clang/test/Preprocessor/macro_fn_comma_swallow2.c +++ b/clang/test/Preprocessor/macro_fn_comma_swallow2.c @@ -5,7 +5,7 @@ // RUN: %clang_cc1 -E -std=c11 %s | FileCheck -check-prefix=C99 -strict-whitespace %s // RUN: %clang_cc1 -E -x c++ %s | FileCheck -check-prefix=GCC -strict-whitespace %s // RUN: %clang_cc1 -E -std=gnu99 %s | FileCheck -check-prefix=GCC -strict-whitespace %s -// RUN: %clang_cc1 -E -fms-compatibility %s | FileCheck -check-prefix=MS -strict-whitespace %s +// RUN: %clang_cc1 -E -fms-preprocessor-compat %s | FileCheck -check-prefix=MS -strict-whitespace %s // RUN: %clang_cc1 -E -DNAMED %s | FileCheck -check-prefix=GCC -strict-whitespace %s // RUN: %clang_cc1 -E -std=c99 -DNAMED %s | FileCheck -check-prefix=C99 -strict-whitespace %s diff --git a/clang/test/Preprocessor/microsoft-ext.c b/clang/test/Preprocessor/microsoft-ext.c index aae52a84b62f1..51e2f9aff234d 100644 --- a/clang/test/Preprocessor/microsoft-ext.c +++ b/clang/test/Preprocessor/microsoft-ext.c @@ -1,4 +1,4 @@ -// RUN: %clang_cc1 -E -fms-compatibility %s -o %t +// RUN: %clang_cc1 -E -fms-preprocessor-compat %s -o %t // RUN: FileCheck %s < %t # define M2(x, y) x + y diff --git a/clang/test/Preprocessor/microsoft-preprocessor-compat.c b/clang/test/Preprocessor/microsoft-preprocessor-compat.c new file mode 100644 index 0000000000000..29883409a85b8 --- /dev/null +++ b/clang/test/Preprocessor/microsoft-preprocessor-compat.c @@ -0,0 +1,36 @@ +// This test verifies that _MSVC_TRADITIONAL is defined under the right +// circumstances, microsoft-ext.c is responsible for testing the implementation +// details of the traditional preprocessor. +// FIXME: C11 should eventually use the conforming preprocessor by default. +// +// RUN: %clang_cc1 -E %s | FileCheck --check-prefix=CHECK-DEFAULT %s +// RUN: %clang_cc1 -E -fms-extensions %s | FileCheck --check-prefix=CHECK-MS-EXT %s +// RUN: %clang -E -fms-compatibility %s | FileCheck --check-prefix=CHECK-MS-COMPAT %s +// RUN: %clang_cc1 -E -fms-preprocessor-compat %s | FileCheck --check-prefix=CHECK-MS-PREPRO-COMPAT %s +// RUN: %clang_cl -E %s | FileCheck --check-prefix=CHECK-CL %s +// RUN: %clang_cl -E /std:c11 %s | FileCheck --check-prefix=CHECK-C11 %s +// RUN: %clang_cl -E /Zc:preprocessor %s | FileCheck --check-prefix=CHECK-ZC-PREPRO %s +// RUN: %clang_cl -E /clang:-fno-ms-preprocessor-compat %s | FileCheck --check-prefix=CHECK-NO-PREPRO-COMPAT %s + +typedef enum { + NOT_DEFINED, + IS_ZERO, + IS_ONE +} State; + +#if !defined(_MSVC_TRADITIONAL) +State state = NOT_DEFINED; +#elif _MSVC_TRADITIONAL == 0 +State state = IS_ZERO; +#elif _MSVC_TRADITIONAL == 1 +State state = IS_ONE; +#endif + +// CHECK-DEFAULT: State state = NOT_DEFINED; +// CHECK-MS-EXT: State state = IS_ZERO; +// CHECK-MS-COMPAT: State state = IS_ONE; +// CHECK-MS-PREPRO-COMPAT: State state = IS_ONE; +// CHECK-CL: State state = IS_ONE; +// CHECK-C11: State state = IS_ONE; +// CHECK-ZC-PREPRO: State state = IS_ZERO; +// CHECK-NO-PREPRO-COMPAT: State state = IS_ZERO; `````````` </details> https://github.com/llvm/llvm-project/pull/167200 _______________________________________________ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
