rsandifo-arm created this revision. rsandifo-arm added reviewers: erichkeane, aaron.ballman. Herald added subscribers: kristof.beyls, dschuff. Herald added a project: All. rsandifo-arm requested review of this revision. Herald added subscribers: cfe-commits, aheejin. Herald added projects: clang, clang-tools-extra.
Platform-specific language extensions often want to provide a way of indicating that certain functions should be called in a different way, compiled in a different way, or otherwise treated differently from a “normal” function. Honoring these indications is often required for correctness, rather being than an optimization/QoI thing. If a function declaration has a property P that matters for correctness, it will be ODR-incompatible with a function that does not have property P. If a function type has a property P that affects the calling convention, it will not be two-way compatible with a function type that does not have property P. These properties therefore affect language semantics. That in turn means that they cannot be treated as standard [[]] attributes. Until now, many of these properties have been specified using GNU-style attributes instead. GNU attributes have traditionally been more lax than standard attributes, with many of them having semantic meaning. Examples include calling conventions and the vector_size attribute. However, there is a big drawback to using GNU attributes for semantic information: compilers that don't understand the attributes will (by default) emit a warning rather than an error. They will go on to compile the code as though the attributes weren't present, which will inevitably lead to wrong code in most cases. For users who live dangerously and disable the warning, this wrong code could even be generated silently. A more robust approach would be to specify the properties using keywords, which older compilers would then reject. Some vendor-specific extensions have already taken this approach. But traditionally, each such keyword has been treated as a language extension in its own right. This has three major drawbacks: (1) The parsing rules need to be kept up-to-date as the language evolves. (2) There are often corner cases that similar extensions handle differently. (3) Each extension requires more custom code than a standard attribute. The underlying problem for all three is that, unlike for true attributes, there is no established template that extensions can reuse. The purpose of this patch series is to try to provide such a template. One option would have been to pick an existing keyword and do whatever that keyword does. The problem with that is that most keywords only apply to specific kinds of types, kinds of decls, etc., and so the parsing rules are (for good reason) not generally applicable to all types and decls. Really, the “only” thing wrong with using standard attributes is that standard attributes cannot affect semantics. In all other respects they provide exactly what we need: a well-defined grammar that evolves with the language, clear rules about what an attribute appertains to, and so on. This series therefore adds keyword “attributes” that can appear exactly where a standard attribute can appear and that appertain to exactly what a standard attribute would appertain to. The link is mechanical and no opt-outs or variations are allowed. This should make the keywords predictable for programmers who are already familiar with standard attributes. This does mean that these keywords will be accepted for parsing purposes in many more places than necessary. Inappropriate uses will then be diagnosed during semantic analysis. However, the compiler would need to reject the keywords in those positions whatever happens, and treating them as ostensible attributes shouldn't be any worse than the alternative. In some cases it might even be better. For example, SME's __arm_streaming attribute would make conceptual sense as a statement attribute, so someone who takes a “try-it-and-see” approach might write: __arm_streaming { …block-of-code…; } In fact, we did consider supporting this originally. The reason for rejecting it was that it was too difficult to implement, rather than because it didn't make conceptual sense. One slight disadvantage of the keyword-based approach is that it isn't possible to use #pragma clang attribute with the keywords. Perhaps we could add support for that in future, if it turns out to be useful. For want of a better term, I've called the new attributes "regular" keyword attributes (in the sense that their parsing is regular wrt standard attributes), as opposed to "custom" keyword attributes that have their own parsing rules. This patch adds the Attr.td support for regular keyword attributes. Adding an attribute with a RegularKeyword spelling causes tablegen to define the associated tokens and to record that attributes created with that syntax are regular keyword attributes rather than custom keyword attributes. A follow-on patch contains the main Parse and Sema support, which is enabled automatically by the Attr.td definition. Other notes: - The series does not allow regular keyword attributes to take arguments, but this could be added in future. - I wondered about trying to use tablegen for TypePrinter::printAttributedAfter too, but decided against it. RegularKeyword is really a spelling-level classification rather than an attribute-level classification, and in general, an attribute could have both GNU and RegularKeyword spellings. In contrast, printAttributedAfter is only given the attribute kind and the type that results from applying the attribute. AFAIK, it doesn't have access to the original attribute spelling. This means that some attribute-specific or type-specific knowledge might be needed to print the attribute in the best way. - Generating the tokens automatically from Attr.td means that pseudo's libgrammar does now depend on tablegen. - The patch uses the SME __arm_streaming attribute as an example for testing purposes. The attribute does not do anything at this stage. Later SME-specific patches will add proper semantics for it, and add other SME-related keyword attributes. Repository: rG LLVM Github Monorepo https://reviews.llvm.org/D148700 Files: clang-tools-extra/pseudo/lib/grammar/CMakeLists.txt clang/include/clang/Basic/Attr.td clang/include/clang/Basic/AttributeCommonInfo.h clang/include/clang/Basic/CMakeLists.txt clang/include/clang/Basic/TokenKinds.def clang/include/clang/Basic/TokenKinds.h clang/include/clang/Lex/Token.h clang/lib/AST/TypePrinter.cpp clang/lib/Sema/SemaDeclAttr.cpp clang/lib/Sema/SemaType.cpp clang/lib/Serialization/ASTReaderDecl.cpp clang/lib/Serialization/ASTWriter.cpp clang/unittests/AST/AttrTest.cpp clang/utils/TableGen/ClangAttrEmitter.cpp clang/utils/TableGen/TableGen.cpp clang/utils/TableGen/TableGenBackends.h
Index: clang/utils/TableGen/TableGenBackends.h =================================================================== --- clang/utils/TableGen/TableGenBackends.h +++ clang/utils/TableGen/TableGenBackends.h @@ -43,6 +43,8 @@ llvm::raw_ostream &OS); void EmitClangAttrPCHRead(llvm::RecordKeeper &Records, llvm::raw_ostream &OS); void EmitClangAttrPCHWrite(llvm::RecordKeeper &Records, llvm::raw_ostream &OS); +void EmitClangAttrTokenKinds(llvm::RecordKeeper &Records, + llvm::raw_ostream &OS); void EmitClangAttrHasAttrImpl(llvm::RecordKeeper &Records, llvm::raw_ostream &OS); void EmitClangAttrSpellingListIndex(llvm::RecordKeeper &Records, Index: clang/utils/TableGen/TableGen.cpp =================================================================== --- clang/utils/TableGen/TableGen.cpp +++ clang/utils/TableGen/TableGen.cpp @@ -35,6 +35,7 @@ GenClangAttrSubjectMatchRuleList, GenClangAttrPCHRead, GenClangAttrPCHWrite, + GenClangAttrTokenKinds, GenClangAttrHasAttributeImpl, GenClangAttrSpellingListIndex, GenClangAttrASTVisitor, @@ -128,6 +129,8 @@ "Generate clang PCH attribute reader"), clEnumValN(GenClangAttrPCHWrite, "gen-clang-attr-pch-write", "Generate clang PCH attribute writer"), + clEnumValN(GenClangAttrTokenKinds, "gen-clang-attr-token-kinds", + "Generate a list of attribute-related clang tokens"), clEnumValN(GenClangAttrHasAttributeImpl, "gen-clang-attr-has-attribute-impl", "Generate a clang attribute spelling list"), @@ -303,6 +306,9 @@ case GenClangAttrPCHWrite: EmitClangAttrPCHWrite(Records, OS); break; + case GenClangAttrTokenKinds: + EmitClangAttrTokenKinds(Records, OS); + break; case GenClangAttrHasAttributeImpl: EmitClangAttrHasAttrImpl(Records, OS); break; Index: clang/utils/TableGen/ClangAttrEmitter.cpp =================================================================== --- clang/utils/TableGen/ClangAttrEmitter.cpp +++ clang/utils/TableGen/ClangAttrEmitter.cpp @@ -2378,6 +2378,11 @@ OS << "#endif // CLANG_ATTR_ACCEPTS_EXPR_PACK\n\n"; } +static bool isRegularKeywordAttribute(const FlattenedSpelling &S) { + return (S.variety() == "Keyword" && + !S.getSpellingRecord().getValueAsBit("HasOwnParseRules")); +} + static void emitFormInitializer(raw_ostream &OS, const FlattenedSpelling &Spelling, StringRef SpellingIndex) { @@ -2385,7 +2390,9 @@ (Spelling.variety() == "Keyword" && Spelling.name() == "alignas"); OS << "{AttributeCommonInfo::AS_" << Spelling.variety() << ", " << SpellingIndex << ", " << (IsAlignas ? "true" : "false") - << " /*IsAlignas*/}"; + << " /*IsAlignas*/, " + << (isRegularKeywordAttribute(Spelling) ? "true" : "false") + << " /*IsRegularKeywordAttribute*/}"; } static void emitAttributes(RecordKeeper &Records, raw_ostream &OS, @@ -3404,6 +3411,25 @@ OS << " .Default(0);\n"; } +// Emits the list of tokens for regular keyword attributes. +void EmitClangAttrTokenKinds(RecordKeeper &Records, raw_ostream &OS) { + emitSourceFileHeader("A list of tokens generated from the attribute" + " definitions", + OS); + // Assume for now that the same token is not used in multiple regular + // keyword attributes. + for (auto *R : Records.getAllDerivedDefinitions("Attr")) + for (const auto &S : GetFlattenedSpellings(*R)) + if (isRegularKeywordAttribute(S)) { + if (!R->getValueAsListOfDefs("Args").empty()) + PrintError(R->getLoc(), + "RegularKeyword attributes with arguments are not " + "yet supported"); + OS << "KEYWORD(" << S.getSpellingRecord().getValueAsString("Name") + << ", KEYALL)\n"; + } +} + // Emits the list of spellings for attributes. void EmitClangAttrHasAttrImpl(RecordKeeper &Records, raw_ostream &OS) { emitSourceFileHeader("Code to implement the __has_attribute logic", OS); Index: clang/unittests/AST/AttrTest.cpp =================================================================== --- clang/unittests/AST/AttrTest.cpp +++ clang/unittests/AST/AttrTest.cpp @@ -168,4 +168,16 @@ } } +TEST(Attr, RegularKeywordAttribute) { + auto AST = clang::tooling::buildASTFromCode(""); + auto &Ctx = AST->getASTContext(); + auto Funcref = clang::WebAssemblyFuncrefAttr::CreateImplicit(Ctx); + EXPECT_EQ(Funcref->getSyntax(), clang::AttributeCommonInfo::AS_Keyword); + ASSERT_FALSE(Funcref->isRegularKeywordAttribute()); + + auto Streaming = clang::ArmStreamingAttr::CreateImplicit(Ctx); + EXPECT_EQ(Streaming->getSyntax(), clang::AttributeCommonInfo::AS_Keyword); + ASSERT_TRUE(Streaming->isRegularKeywordAttribute()); +} + } // namespace Index: clang/lib/Serialization/ASTWriter.cpp =================================================================== --- clang/lib/Serialization/ASTWriter.cpp +++ clang/lib/Serialization/ASTWriter.cpp @@ -4348,6 +4348,7 @@ Record.push_back(A->getParsedKind()); Record.push_back(A->getSyntax()); Record.push_back(A->getAttributeSpellingListIndexRaw()); + Record.push_back(A->isRegularKeywordAttribute()); #include "clang/Serialization/AttrPCHWrite.inc" } Index: clang/lib/Serialization/ASTReaderDecl.cpp =================================================================== --- clang/lib/Serialization/ASTReaderDecl.cpp +++ clang/lib/Serialization/ASTReaderDecl.cpp @@ -3094,11 +3094,12 @@ bool IsAlignas = (ParsedKind == AttributeCommonInfo::AT_Aligned && Syntax == AttributeCommonInfo::AS_Keyword && SpellingIndex == AlignedAttr::Keyword_alignas); + bool IsRegularKeywordAttribute = Record.readBool(); - AttributeCommonInfo Info( - AttrName, ScopeName, AttrRange, ScopeLoc, - AttributeCommonInfo::Kind(ParsedKind), - {AttributeCommonInfo::Syntax(Syntax), SpellingIndex, IsAlignas}); + AttributeCommonInfo Info(AttrName, ScopeName, AttrRange, ScopeLoc, + AttributeCommonInfo::Kind(ParsedKind), + {AttributeCommonInfo::Syntax(Syntax), SpellingIndex, + IsAlignas, IsRegularKeywordAttribute}); #include "clang/Serialization/AttrPCHRead.inc" Index: clang/lib/Sema/SemaType.cpp =================================================================== --- clang/lib/Sema/SemaType.cpp +++ clang/lib/Sema/SemaType.cpp @@ -125,6 +125,7 @@ case ParsedAttr::AT_VectorCall: \ case ParsedAttr::AT_AArch64VectorPcs: \ case ParsedAttr::AT_AArch64SVEPcs: \ + case ParsedAttr::AT_ArmStreaming: \ case ParsedAttr::AT_AMDGPUKernelCall: \ case ParsedAttr::AT_MSABI: \ case ParsedAttr::AT_SysVABI: \ @@ -4894,8 +4895,10 @@ // If we're supposed to infer nullability, do so now. if (inferNullability && !inferNullabilityInnerOnlyComplete) { ParsedAttr::Form form = - inferNullabilityCS ? ParsedAttr::Form::ContextSensitiveKeyword() - : ParsedAttr::Form::Keyword(false /*IsAlignAs*/); + inferNullabilityCS + ? ParsedAttr::Form::ContextSensitiveKeyword() + : ParsedAttr::Form::Keyword(false /*IsAlignAs*/, + false /*IsRegularKeywordAttribute*/); ParsedAttr *nullabilityAttr = Pool.create( S.getNullabilityKeyword(*inferNullability), SourceRange(pointerLoc), nullptr, SourceLocation(), nullptr, 0, form); @@ -7709,6 +7712,8 @@ return createSimpleAttr<AArch64VectorPcsAttr>(Ctx, Attr); case ParsedAttr::AT_AArch64SVEPcs: return createSimpleAttr<AArch64SVEPcsAttr>(Ctx, Attr); + case ParsedAttr::AT_ArmStreaming: + return createSimpleAttr<ArmStreamingAttr>(Ctx, Attr); case ParsedAttr::AT_AMDGPUKernelCall: return createSimpleAttr<AMDGPUKernelCallAttr>(Ctx, Attr); case ParsedAttr::AT_Pcs: { Index: clang/lib/Sema/SemaDeclAttr.cpp =================================================================== --- clang/lib/Sema/SemaDeclAttr.cpp +++ clang/lib/Sema/SemaDeclAttr.cpp @@ -5258,6 +5258,9 @@ case ParsedAttr::AT_AArch64SVEPcs: CC = CC_AArch64SVEPCS; break; + case ParsedAttr::AT_ArmStreaming: + CC = CC_C; // Placeholder until real SME support is added. + break; case ParsedAttr::AT_AMDGPUKernelCall: CC = CC_AMDGPUKernelCall; break; Index: clang/lib/AST/TypePrinter.cpp =================================================================== --- clang/lib/AST/TypePrinter.cpp +++ clang/lib/AST/TypePrinter.cpp @@ -1721,6 +1721,11 @@ return; } + if (T->getAttrKind() == attr::ArmStreaming) { + OS << "__arm_streaming"; + return; + } + OS << " __attribute__(("; switch (T->getAttrKind()) { #define TYPE_ATTR(NAME) @@ -1761,6 +1766,7 @@ case attr::CmseNSCall: case attr::AnnotateType: case attr::WebAssemblyFuncref: + case attr::ArmStreaming: llvm_unreachable("This attribute should have been handled already"); case attr::NSReturnsRetained: Index: clang/include/clang/Lex/Token.h =================================================================== --- clang/include/clang/Lex/Token.h +++ clang/include/clang/Lex/Token.h @@ -117,8 +117,13 @@ } /// Return true if this is any of tok::annot_* kind tokens. - bool isAnnotation() const { - return tok::isAnnotation(getKind()); + bool isAnnotation() const { return tok::isAnnotation(getKind()); } + + /// Return true if K is a keyword that is parsed in the same position as + /// a standard attribute, but that has semantic meaning and so cannot be + /// a true attribute. + bool isRegularKeywordAttribute() const { + return tok::isRegularKeywordAttribute(getKind()); } /// Return a source location identifier for the specified Index: clang/include/clang/Basic/TokenKinds.h =================================================================== --- clang/include/clang/Basic/TokenKinds.h +++ clang/include/clang/Basic/TokenKinds.h @@ -99,6 +99,14 @@ /// Return true if this is an annotation token representing a pragma. bool isPragmaAnnotation(TokenKind K); +inline constexpr bool isRegularKeywordAttribute(TokenKind K) { + return (false +#define KEYWORD(X, Y) || (K == tok::kw_##X) +#include "clang/Basic/AttrTokenKinds.inc" +#undef KEYWORD + ); +} + } // end namespace tok } // end namespace clang Index: clang/include/clang/Basic/TokenKinds.def =================================================================== --- clang/include/clang/Basic/TokenKinds.def +++ clang/include/clang/Basic/TokenKinds.def @@ -747,6 +747,9 @@ KEYWORD(__builtin_available , KEYALL) KEYWORD(__builtin_sycl_unique_stable_name, KEYSYCL) +// Keywords defined by Attr.td. +#include "clang/Basic/AttrTokenKinds.inc" + // Clang-specific keywords enabled only in testing. TESTING_KEYWORD(__unknown_anytype , KEYALL) Index: clang/include/clang/Basic/CMakeLists.txt =================================================================== --- clang/include/clang/Basic/CMakeLists.txt +++ clang/include/clang/Basic/CMakeLists.txt @@ -35,6 +35,12 @@ SOURCE Attr.td TARGET ClangAttrSubjectMatchRuleList) +clang_tablegen(AttrTokenKinds.inc -gen-clang-attr-token-kinds + -I ${CMAKE_CURRENT_SOURCE_DIR}/../../ + SOURCE Attr.td + TARGET ClangAttrTokenKinds + ) + clang_tablegen(AttrHasAttributeImpl.inc -gen-clang-attr-has-attribute-impl -I ${CMAKE_CURRENT_SOURCE_DIR}/../../ SOURCE Attr.td Index: clang/include/clang/Basic/AttributeCommonInfo.h =================================================================== --- clang/include/clang/Basic/AttributeCommonInfo.h +++ clang/include/clang/Basic/AttributeCommonInfo.h @@ -77,6 +77,7 @@ unsigned SyntaxUsed : 4; unsigned SpellingIndex : 4; unsigned IsAlignas : 1; + unsigned IsRegularKeywordAttribute : 1; protected: static constexpr unsigned SpellingNotCalculated = 0xf; @@ -86,24 +87,29 @@ /// including its syntax and spelling. class Form { public: - constexpr Form(Syntax SyntaxUsed, unsigned SpellingIndex, bool IsAlignas) + constexpr Form(Syntax SyntaxUsed, unsigned SpellingIndex, bool IsAlignas, + bool IsRegularKeywordAttribute) : SyntaxUsed(SyntaxUsed), SpellingIndex(SpellingIndex), - IsAlignas(IsAlignas) {} + IsAlignas(IsAlignas), + IsRegularKeywordAttribute(IsRegularKeywordAttribute) {} constexpr Form(tok::TokenKind Tok) : SyntaxUsed(AS_Keyword), SpellingIndex(SpellingNotCalculated), - IsAlignas(Tok == tok::kw_alignas) {} + IsAlignas(Tok == tok::kw_alignas), + IsRegularKeywordAttribute(tok::isRegularKeywordAttribute(Tok)) {} Syntax getSyntax() const { return Syntax(SyntaxUsed); } unsigned getSpellingIndex() const { return SpellingIndex; } bool isAlignas() const { return IsAlignas; } + bool isRegularKeywordAttribute() const { return IsRegularKeywordAttribute; } static Form GNU() { return AS_GNU; } static Form CXX11() { return AS_CXX11; } static Form C2x() { return AS_C2x; } static Form Declspec() { return AS_Declspec; } static Form Microsoft() { return AS_Microsoft; } - static Form Keyword(bool IsAlignas) { - return Form(AS_Keyword, SpellingNotCalculated, IsAlignas); + static Form Keyword(bool IsAlignas, bool IsRegularKeywordAttribute) { + return Form(AS_Keyword, SpellingNotCalculated, IsAlignas, + IsRegularKeywordAttribute); } static Form Pragma() { return AS_Pragma; } static Form ContextSensitiveKeyword() { return AS_ContextSensitiveKeyword; } @@ -113,11 +119,12 @@ private: constexpr Form(Syntax SyntaxUsed) : SyntaxUsed(SyntaxUsed), SpellingIndex(SpellingNotCalculated), - IsAlignas(0) {} + IsAlignas(0), IsRegularKeywordAttribute(0) {} unsigned SyntaxUsed : 4; unsigned SpellingIndex : 4; unsigned IsAlignas : 1; + unsigned IsRegularKeywordAttribute : 1; }; AttributeCommonInfo(const IdentifierInfo *AttrName, @@ -127,7 +134,8 @@ ScopeLoc(ScopeLoc), AttrKind(AttrKind), SyntaxUsed(FormUsed.getSyntax()), SpellingIndex(FormUsed.getSpellingIndex()), - IsAlignas(FormUsed.isAlignas()) { + IsAlignas(FormUsed.isAlignas()), + IsRegularKeywordAttribute(FormUsed.isRegularKeywordAttribute()) { assert(SyntaxUsed >= AS_GNU && SyntaxUsed <= AS_Implicit && "Invalid syntax!"); } @@ -154,7 +162,10 @@ Kind getParsedKind() const { return Kind(AttrKind); } Syntax getSyntax() const { return Syntax(SyntaxUsed); } - Form getForm() const { return Form(getSyntax(), SpellingIndex, IsAlignas); } + Form getForm() const { + return Form(getSyntax(), SpellingIndex, IsAlignas, + IsRegularKeywordAttribute); + } const IdentifierInfo *getAttrName() const { return AttrName; } SourceLocation getLoc() const { return AttrRange.getBegin(); } SourceRange getRange() const { return AttrRange; } @@ -191,6 +202,8 @@ return SyntaxUsed == AS_Keyword || SyntaxUsed == AS_ContextSensitiveKeyword; } + bool isRegularKeywordAttribute() const { return IsRegularKeywordAttribute; } + bool isContextSensitiveKeywordAttribute() const { return SyntaxUsed == AS_ContextSensitiveKeyword; } Index: clang/include/clang/Basic/Attr.td =================================================================== --- clang/include/clang/Basic/Attr.td +++ clang/include/clang/Basic/Attr.td @@ -315,6 +315,13 @@ : Spelling<name, "Keyword"> { bit HasOwnParseRules = hasOwnParseRules; } + +// A keyword that can appear wherever a standard attribute can appear, +// and that appertains to whatever a standard attribute would appertain to. +// This is useful for things that affect semantics but that should otherwise +// be treated like standard attributes. +class RegularKeyword<string name> : Keyword<name, 0> {} + // A keyword that has its own individual parsing rules. class CustomKeyword<string name> : Keyword<name, 1> {} @@ -2417,6 +2424,11 @@ let Documentation = [AArch64SVEPcsDocs]; } +def ArmStreaming : TypeAttr, TargetSpecificAttr<TargetAArch64> { + let Spellings = [RegularKeyword<"__arm_streaming">]; + let Documentation = [Undocumented]; +} + def Pure : InheritableAttr { let Spellings = [GCC<"pure">]; let Documentation = [Undocumented]; Index: clang-tools-extra/pseudo/lib/grammar/CMakeLists.txt =================================================================== --- clang-tools-extra/pseudo/lib/grammar/CMakeLists.txt +++ clang-tools-extra/pseudo/lib/grammar/CMakeLists.txt @@ -1,10 +1,5 @@ set(LLVM_LINK_COMPONENTS Support) -# This library is used by the clang-pseudo-gen tool which runs at build time. -# Dependencies should be minimal to avoid long dep paths in the build graph. -# It does use clangBasic headers (tok::TokenKind), but linking is not needed. -# We have no transitive dependencies on tablegen files. -list(REMOVE_ITEM LLVM_COMMON_DEPENDS clang-tablegen-targets) add_clang_library(clangPseudoGrammar Grammar.cpp GrammarBNF.cpp
_______________________________________________ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits