> On Mar 27, 2025, at 9:17 AM, Qing Zhao <qing.z...@oracle.com> wrote:
>
> Yeoul,
>
> Thanks for the writeup.
>
> So, basically, This writeup insisted on introducing a new “structure scope”
> (similar as the instance scope in C++) into C language ONLY for counted_by
> attribute:
>
> 1. Inside counted_by attribute, the name lookup starts:
>
> A. Inside the current structure first (the NEW structure scope added to C);
> B. Then outside the structure; (other current C scopes, local scope or
> global scope)
>
> 2. When trying to reference a variable outside of the structure scope that
> name_conflicts with
> a structure member, a new builtin function “__builtin_global_ref” is
> introduced for such
> purpose.
>
> ( I think that __builtin_global_ref might not accurate, because the outer
> scope might be either global scope or local scope)
Clarification: __builtin_global_ref will see the global scope directly. This is
similar to global scope resolution syntax (‘::’) in C++.
constexpr int len = 10;
void foo (void)
{
const int len = 20;
struct s {
int len;
int *__counted_by(__builtin_global_ref(len)) buf; // refers to global ‘len'
};
}
Here are some reasons why we chose to provide a global scope resolution
builtin, not a builtin to see an outer scope or just a local scope:
1) The builtin is a substitute for some “scope resolution specifier”. Scope
specifiers typically meant to choose a “specific" scope.
2) To the best of my knowledge there is no precedence in any other C family
language to provide a scope resolution for local scopes.
3) Name conflicts with local variables can be easily renamed.
4) If we provide a builtin that selects outer scope instead, there is no way to
choose a global ‘len' if it’s shadowed by a local variable, so then the member
name has to be renamed anyway in order to choose a global `len`.
5) This way, code can be written compatibly both in C and C++.
>
> 3. Where there is confliction between counted_by and VLA such as:
>
> constexpr int len = 10;
>
> struct s {
> int len;
> int *__counted_by(len) buf; // refers to struct member `len`.
> int arr[len]; // refers to global constexpr `len`
> };
>
> Issue compiler warning to user to ask the user to use __builtin_global_ref to
> distinguish.
Additionally, our proposal suggests __builtin_member_ref to explicitly use a
member in a similar situation.
The builtin could be replaced by ‘__self' or some other syntax once the
standard committee decides in the future, but earlier in the thread JeanHeyd
pointed out that:
"I would like to gently push back about __self__, or __self, or self,
because all of these identifiers are fairly common identifiers in code. When I
writing the paper for __self_func (
https://thephd.dev/_vendor/future_cxx/papers/C%20-%20__self_func.html ), I
searched GitHub and other source code indexing and repository services: __self,
__self__, and self has a substantial amount of uses. If there's an alternative
spelling to consider, I think that would be helpful."
Thus, I think instead of trying to stick to a certain syntax right now, using
some builtin will allow us to easily migrate to a new syntax by guarding the
current usage under a macro.
Writing the builtin could be cumbersome but this shall be written only when
there is an ambiguity. Btw, I’m open to any other name suggestions for the
builtins!
>
> Are the above the correct understanding of your writeup?
Yes, it’s mostly correct, except some clarifications I made above. Thank you!
>
>
> From my understanding:
>
> 1. This design started from the C++’s point of view by adding a new
> “structure scope” to C;
> 2. This design conflicts with the current VLA default scope rule (which based
> on the default C scopes) in C.
> In the above example that mixes counted_by and VLA, it’s so weird that
> there are two difference name
> lookup rules inside the same structure.
> It’s clearly a design bug. Either VLA or counted_by need to be fixed to
> make them consistent.
>
>
> I personally do not completely object to introduce a new “structure scope”
> into C, but it’s so hard for me to accept
> that there are two different name lookup rules inside the same structure: one
> rule for VLA, another rule for counted_by
> attribute. (If introducing a new “structure scope” to C, I think it’s
> better to change VLA to “structure scope” too, not sure
> whether this is feasible or not)
>
> I still think that introduce a new keyword “__self” for referring member
> variable inside structure without adding
> a new “structure scope" should be the best approach to resolve this issue in
> C.
>
> However, I am really hoping that the discussion can be converged soon. So, I
> am okay with adding a new “structure scope”
> If most of people agreed on that approach.
Thanks for the flexibility!
>
> Qing
>
>
>> On Mar 26, 2025, at 12:59, Yeoul Na <yeoul...@apple.com> wrote:
>>
>> Hi all,
>>
>> Thanks for all the discussions.
>>
>> I posted the design rationale for our current approach in
>> https://discourse.llvm.org/t/rfc-forward-referencing-a-struct-member-within-bounds-annotations/85510.
>> This clarifies some of the questions that are asked in this thread. The
>> document also proposes diagnostics to mitigate potential ambiguity, and
>> propose new builtins that can be used as a suppression and disambiguation
>> mechanism.
>>
>> Best regards,
>> Yeoul
>>
>>> On Mar 26, 2025, at 9:11 AM, Yeoul Na <yeoul...@apple.com> wrote:
>>>
>>> Sorry for the delay.
>>>
>>> I’m planning on sending out our design rationale of the current approach
>>> without the new syntax today.
>>>
>>> - Yeoul
>>>
>>>> On Mar 14, 2025, at 9:22 PM, John McCall <rjmcc...@apple.com> wrote:
>>>>
>>>> On 14 Mar 2025, at 15:18, Martin Uecker wrote:
>>>> Am Freitag, dem 14.03.2025 um 14:42 -0400 schrieb John McCall:
>>>> On 14 Mar 2025, at 14:13, Martin Uecker wrote:
>>>> Am Freitag, dem 14.03.2025 um 10:11 -0700 schrieb David Tarditi:
>>>> Hi Martin,
>>>> The C design of VLAs misunderstood dependent typing.
>>>> They probably did not care about theory, but the design is
>>>> not inconsistent with theory.
>>>> This is almost true, but for bad reasons. The theory of dependent types is
>>>> heavily concerned with deciding whether two types are the same, and C
>>>> simply sidesteps this question because type identity is largely
>>>> meaningless in C. Every value of variably-modified type is (or decays to)
>>>> a pointer, and all pointers in C freely convert to one another (within the
>>>> object/function categories). _Generic is based on type compatibility, not
>>>> equality. So in that sense, the standard doesn’t say anything inconsistent
>>>> with theory because it doesn’t even try to say anything.
>>>> The reason it is not quite true is that C does have rules for compatible
>>>> and composite types, and alas, those rules for variably-modified types are
>>>> not consistent with theory. Two VLA types of compatible element type are
>>>> always statically considered compatible, and it’s simply UB if the sizes
>>>> aren’t the same. The composite type of a VLA and a fixed-size array type
>>>> is always the fixed-size array type. The standard is literally incomplete
>>>> about the composite type of two VLAs; if you use a ternary operator where
>>>> both operands are casts to VLA types, the standard just says it’s
>>>> straight-up just undefined behavior (because one of the types has a bound
>>>> that’s unevaluated) and doesn’t even bother telling us what the static
>>>> type is supposed to be.
>>>> Yes, I guess this is all true.
>>>> But let's rephrase my point a bit more precisely: One could take
>>>> a strict subset of C that includes variably modified types but
>>>> obviously has to forbid a lot other things (e.g. arbitrary pointer
>>>> conversions or unsafe down-casts and much more) and make this a
>>>> memory-safe language with dependent types. This would also
>>>> require adding run-time checks at certain places where there
>>>> is now UB, in particular where two VLA types need to be compatible.
>>>> Mmm. You can certainly subset C to the point that it’s memory-safe, but
>>>> it wouldn’t really be anything like C anymore. As long as C has a heap,
>>>> I don’t see any path to achieving temporal safety without significant
>>>> extensions to the language. But if we’re just talking about spatial safety,
>>>> then sure, that could be a lot closer to C today.
>>>> Is that your vision, then, that you’d like to see the same sort of checks
>>>> that -fbounds-safety does, but you want them based firmly in the language
>>>> as a dynamic check triggered by pointer type conversion, with bounds
>>>> specified using variably-modified types? It’s a pretty elegant vision, and
>>>> I can see the attraction. It has some real merits, which I’ll get to below.
>>>> I do see at least two significant challenges, though.
>>>> The first and biggest problem is that, in general, array bounds can only be
>>>> expressed on a pointer value if it’s got pointer to array type. Most C
>>>> array
>>>> code today works primarily with pointers to elements; programmers just use
>>>> array types to create concrete arrays, and they very rarely use pointers to
>>>> array type at all. There are a bunch of reasons for that:
>>>> • Pointers to arrays have to be dereferenced twice: (*ptr)[idx] instead
>>>> of ptr[idx].
>>>> • That makes them more error-prone, because it is easy to do pointer
>>>> arithmetic at the wrong level, e.g. by writing ptr[idx], which will
>>>> stride by multiples of the entire array size. That may even pass the
>>>> compiler without complaint because of C’s laxness about conversions.
>>>> • Keeping the bound around in the pointer type is more work and doesn’t
>>>> do
>>>> anything useful right now.
>>>> • A lot of C programmers dislike nested declarator syntax and can’t
>>>> remember
>>>> how it works. Those of us who can write it off the top of our heads are
>>>> quite atypical.
>>>> Now, there is an exception: you can write a parameter using an array type,
>>>> and it actually declares a pointer parameter. You could imagine using this
>>>> as a syntax for an enforceable array bound for arguments, although the
>>>> committee did already decide that these bounds were meaningless without
>>>> static. Unfortunately, you can’t do this in any other position and still
>>>> end up with just a pointer, so it’s not helpful as a general syntax for
>>>> associating bounds with pointers.
>>>> The upshot is that this isn’t really something people can just adopt by
>>>> adding annotations. It’s not just a significant rewrite, it’s a rewrite
>>>> that
>>>> programmers will have very legitimate objections to. I think that makes
>>>> this
>>>> at best a complement to the “sidecar” approach taken by -fbounds-safety
>>>> where we can track top-level bounds to a specific pointer value.
>>>> The second problem is that there are some extralingual problems that
>>>> -fbounds-safety has to solve around bounds that aren’t just local
>>>> evaluations of bounds expressions, and a type-conversion-driven approach
>>>> doesn’t help with any of them.
>>>> As you mentioned, the design of variably-modified types is based on
>>>> evaluating the bounds expression at some specific point in the program
>>>> execution. Since these types can only be written locally, the evaluation
>>>> point is obvious. If we wanted to dynamically enforce bounds during
>>>> initialization, it would simply be another use of the same computed bound:
>>>> int count = ...;
>>>> int (*ptr)[count * 10] = source_ptr;
>>>>
>>>> Here we would evaluate count * 10 exactly once and use it both as (1) part
>>>> of the destination type when initializing ptr with source_ptr and (2)
>>>> part of the type of ptr for all uses of it. For example, if source_ptr
>>>> were of type int (*)[100], we would dynamically check that
>>>> count * 10 <= 100. This all works perfectly with an arbitrary bounds
>>>> expression; it could even contain an opaque function call.
>>>> Note that we don’t need any special behavior specifically for
>>>> initialization. If we later assign a new value into ptr, we will still be
>>>> converting the new value to the type int (*)[< count * 10 >], using the
>>>> value computed at the time of declaration of the variable. This model would
>>>> simply require that conversion to validate the bounds during assignment
>>>> just
>>>> as it would during initialization.
>>>> Now, with nested arrays, variance does become a problem. Let’s reduce
>>>> bounds expression to their evaluated bounds to make this easier to write.
>>>> • int (*)[11] can be converted to int(*)[10] because we’re simply
>>>> allowing fewer elements to be used.
>>>> • By the same token, int (*(*)[11])[5] can be converted to
>>>> int (*(*)[10])[5]. This is the same logic as the above, just with an
>>>> element type that happens to be a pointer to array type.
>>>> • But int (*(*)[11])[5] cannot be safely converted to int (*(*)[11])[4],
>>>> because while it’s safe to read an int (*)[4] from this array, it’s
>>>> not safe to assign one into it.
>>>> • int (* const (*)[11])[5] can be safely converted to
>>>> int (* const (*)[11])[4], but only if this dialect also enforces const-
>>>> correctness, at least on array pointers.
>>>> Anyway, a lot of this changes if we want to use the same concept for
>>>> non-local pointers to arrays, because we no longer have an obvious point of
>>>> execution at which to evaluate the bounds expression. Instead, we are
>>>> forced
>>>> into re-evaluating it every time we access the variable holding the array.
>>>> Consider:
>>>> struct X {
>>>> int count;
>>>> int (*ptr)[count * 10]; // using my preferred syntax
>>>> };
>>>>
>>>> void test(struct X *xp) {
>>>> // For the purposes of the conversion check here, the
>>>> // source type is int (*)[< xp->count * 10 >], freshly
>>>> // evaluated as part of the member access.
>>>> int (*local)[100] = xp->ptr;
>>>> }
>>>>
>>>> This has several immediate consequences.
>>>> Firstly, we need to already be able to compute the correct bound when we do
>>>> the dynamic checks for assignments into this field. For local variably-
>>>> modified types, everything in the expression was already in scope and
>>>> presumably initialized, so this wasn’t a problem. Here, we’re not helped
>>>> by scope, and we are dependent on the count field already having been
>>>> initialized.
>>>> Secondly, we must be very concerned about anything that could change the
>>>> result of this evaluation. So we cannot allow an arbitrary expression;
>>>> it must be something that we can fully analyze for what could change it.
>>>> And if refers to variables or fields (which it presumably always will), we
>>>> must prevent assignments to those, or at least validate that any
>>>> assignments aren’t causing unsound changes to the bound expression.
>>>> Thirdly, that concern must apply non-locally: if we allow the address of
>>>> the
>>>> pointer field to be taken (which is totally fine in the local case!),
>>>> we can no directly reason about mutations through that pointer, so we
>>>> have to prevent changes to the bounds variables/fields while the pointer is
>>>> outstanding.
>>>> And finally, we must be able to recognize combinations of assignments,
>>>> because when we’re initializing (or completely rewriting) this structure,
>>>> we will need to able to assign to both count and ptr and not have the
>>>> same restrictions in place that we would for separate assignments.
>>>> None of this falls out naturally from separate, local language rules; it
>>>> all has to be invented for the purpose of serving this dynamic check. And
>>>> in fact, -fbounds-safety has to do all of this already just to make
>>>> basic checks involving pointers in structs work.
>>>> If that can all be established, though, I think the type-conversion-based
>>>> approach using variably-modified types has some very nice properties as a
>>>> complement to what we’re doing in -fbounds-safety.
>>>> For one, it interacts with the -fbounds-safety analysis very cleanly. If
>>>> bounds in types are dynamically enforced (which is not true in normal C,
>>>> but could be in this dialect), then the type becomes a source for reliable
>>>> reliable information for the bounds-safety analysis. Conversely, if
>>>> a pointer is converted to a variably-modified type, the analysis done
>>>> by -bounds-safety could be used as an input to the conversion check.
>>>> For another, I think it may lead towards an cleaner story for arrays of
>>>> pointers to arrays than -fbounds-safety can achieve today, as long as
>>>> the inner arrays are of uniform length.
>>>> But ultimately, I think it’s still at best a complement to the attributes
>>>> we need for -fbounds-safety.
>>>> John.
>>>
>>
>
Yeoul
